Hi all, I'm wondering if someone could put me on the right path to using the "qvalue" package correctly. I have an actual P value of 0.05.  I have a list of 1,000 randomised p values: range of these randomised p values is 0.002 to 0.795, average of the randomised p values is 0.399 and the median of the randomised p values is 0.45. 869 of the randomised p values are > 0.05.

I want to work out the false discovery rate (FDR) (Q; as described by Storey and Tibshriani in 2003) for my original p value, defined as the number of expected false positives over the number of significant results for my original P value. 

So, for my original P value, I want one Q value, that has been calculated as described above based on the 1,000 random p values.

I wrote this code:

pvals <- c(list_of_p_values_obtained_from_randomisations)

qobj <-qvalue(p=pvals)

r_output <- qobj$pi0

In this case, r_output (i.e. the Q value) is 0.0062. So I thought it would be reasonable to expect the FDR Q Value (i.e the number of expected false positives over the number of significant results) to be at least over 0.05, given that 869 of the randomised p values are > 0.05? . That's why I think qobj$pi0 isn't the right variable to be looking at? So my problem (or my mis-understanding) is that I have an actual P value of 0.05, but then a Q value that is lower, 0.006?

Could someone please tell me where I'm going wrong.

Thanks

Tom

It's a red flag that your largest p-value 0.795.  The assumption is that true null p-values are Uniform(0,1), so the largest p-value should be close to 1 when this assumption is met and there are a reasonable number of tests being performed.

From FAQ #1 (https://github.com/StoreyLab/qvalue/blob/master/README.md#frequently-asked-questions):

1. This package produces "adjusted p-values", so how is it possible that my adjusted p-values are smaller than my original p-values?

The q-value is not an adjusted p-value, but rather a population quantity with an explicit definition. The package produces estimates of q-values and the local FDR, both of which are very different from p-values. The package does not perform a Bonferroni correction on p-values, which returns "adjusted p-values" that are larger than the original p-values. The maximum possible q-value is pi_0, the proportion of true null hypotheses. The maximum possible p-value is 1. When considering a large number of hypothesis tests where there is a nontrivial fraction of true alternative p-values, we will have both an estimate pi_0 < 1 and we will have some large p-values close to 1. Therefore, the maximal estimated q-value will be less than or equal to the estimated pi_0 but there will also be a number of p-values larger than the estimated pi_0. It must be the case then that at some point p-values become larger than estimated q-values.