I know there have been several messages comparing Excel SAM vs siggenes SAM, and several others asking about how the FDR is calculated in siggenes SAM, but none of these have answered a question I have about what to believe in the SAM output table. My table from siggenes: Delta p0 FALSE Called FDR 0.25 0.147 2023.5 4220 0.07 0.5 0.147 1211.5 2686 0.066 0.75 0.147 789 1617 0.072 1 0.147 390 866 0.066 1.25 0.147 197 496 0.058 1.5 0.147 120.5 337 0.053 1.75 0.147 69.5 228 0.045 2 0.147 38.5 150 0.038 2.25 0.147 22 94 0.034 2.5 0.147 13 65 0.029 2.75 0.147 7 30 0.034 3 0.147 4 16 0.037 3.25 0.147 1 7 0.021 3.5 0.147 0.5 4 0.018 3.75 0.147 0 0 0 The FDR here is Pi hat * (false/called). I'm not sure what that is supposed to mean. Which number of false am I supposed to believe? The number false as calculated by multiplying the FDR by the # called? (This makes sense to me, for example: 0.07*4220=297.5 false) Or the # false as reported in the false column? (This doesn't make sense to me...what's the point of the FDR as calculated if it doesn't jibe with the # false and the # called?) Excel SAM seems to circumvent this problem by multiplying the number false by Pi hat (and reporting only that product, not the number false before being multiplied), and then calculating the FDR as false/ called, this FDR then implicitly has Pi hat in it (or so it seems to me). This way, # false, # called, and the observed FDR all correspond correctly, unlike siggenes SAM. Thanks in advance for any help. --Jake [[alternative HTML version deleted]]

Thanks for the reply, Holger. Let me just restate what I understand from what you said, just to make sure I'm understanding this correctly (it's probably pretty obvious that I'm no statistician): The column "False" in the SAM table gives a kind of "worst-case scenario" regarding the number of false at the given number of called significant. To get a more realistic expectation of how many of those are indeed falsely called, we multiply the number in the false column by the proportion pi hat (denoted here as p0). Is there a reason why this "adjusted" number of false calls is not directly reported in the output table? Does this practical interpretation make sense? The reason I believe that Excel is multiplying their number of false calls by pi hat is that (at least in my data) the median number false converges (as the number of total called decreases) to the exact value of pi hat. The exact value of pi hat is reported as the median number false for everything below a certain value. The number false never goes below pi hat. I thought for a long time as to why that would be and it occurred to me that maybe the minimum # false that Excel reports is 1. This minimum is then multiplied by pi hat to give... pi hat. That's the only way I can make sense of the numbers Excel gives. Thanks again for the help. mfg --Jake I On Jun 6, 2005, at 2:47 PM, Holger Schwender wrote: > Hi Jacob, > > the FDR is estimated in almost exactly the same way as in Excel > SAM. The > difference is that by default siggenes uses the mean number of falsely > called genes (as described, e.g., in the Tusher paper) instead of > the median > number of falsely called genes. You can however set med=TRUE in sam > and the > median number will be calculated. The other difference is the > computation of > pi0. While Excel SAM uses an ad hoc estimate, siggenes uses by > default a > natural cubic spline based estimate proposed by John Storey, a > former PhD > student of Tishirani and Co-author of the Tusher paper. You can, > however, > set lambda=.5 in sam(...) and you will get the adhoc estimate of > Excel SAM. > So changing these two defaults will give you exactly the estimates > that are > given to you by Excel SAM. > > To explain the table: False is the mean/median number of falsely > called > genes. If you divide False by Called you will get an estimate which > corresponds to the Benjamini-Hochberg estimate of the FDR. As > authors like > John Storey, Brad Efron etc. have mentioned, this is a conservative > estimate > that assumes that all null hypotheses are TRUE, i.e. none of the > genes are > differentially expressed. In this case, the number of falsely > called genes > is equal to the number of False positives. We however know that > some of the > genes are differentially expressed, so we estimate their proportion > by 1 - > pi0 and this leads to an estimate for the number of false positives > of pi0 * > number of falsely called genes. And the FDR is loosely spoken the > expected > ratio of the number of false positives to the number of rejected null > hypotheses / genes called differentially expressed, and hence FDR = > pi0 * > False / Called. > > So the column False can be calculated by multiplying FDR by Called and > dividing this by pi0, i.e. in your example: False = 0.07*4220/.147. > > And this is the way both Excel SAM and siggenes compute the FDR. > The reason > for this is that the implementation of siggenes follows almost > exactly the > implementation of Excel SAM (almost since some parts of Excel SAM > are not > known, namely the exact choice of the possible quantiles for the fudge > factor and the q-value calculation) as described in Tusher et al. > and the > Excel SAM manual. > > HTH, > Holger > > >> >> I know there have been several messages comparing Excel SAM vs >> siggenes SAM, and several others asking about how the FDR is >> calculated in siggenes SAM, but none of these have answered a >> question I have about what to believe in the SAM output table. My >> table from siggenes: >> >> >> Delta p0 FALSE Called FDR >> 0.25 0.147 2023.5 4220 0.07 >> 0.5 0.147 1211.5 2686 0.066 >> 0.75 0.147 789 1617 0.072 >> 1 0.147 390 866 0.066 >> 1.25 0.147 197 496 0.058 >> 1.5 0.147 120.5 337 0.053 >> 1.75 0.147 69.5 228 0.045 >> 2 0.147 38.5 150 0.038 >> 2.25 0.147 22 94 0.034 >> 2.5 0.147 13 65 0.029 >> 2.75 0.147 7 30 0.034 >> 3 0.147 4 16 0.037 >> 3.25 0.147 1 7 0.021 >> 3.5 0.147 0.5 4 0.018 >> 3.75 0.147 0 0 0 >> >> The FDR here is Pi hat * (false/called). I'm not sure what that is >> supposed to mean. Which number of false am I supposed to believe? >> The number false as calculated by multiplying the FDR by the # >> called? (This makes sense to me, for example: 0.07*4220=297.5 >> false) Or the # false as reported in the false column? (This doesn't >> make sense to me...what's the point of the FDR as calculated if it >> doesn't jibe with the # false and the # called?) >> >> Excel SAM seems to circumvent this problem by multiplying the number >> false by Pi hat (and reporting only that product, not the number >> false before being multiplied), and then calculating the FDR as >> false/ >> called, this FDR then implicitly has Pi hat in it (or so it seems to >> me). This way, # false, # called, and the observed FDR all >> correspond correctly, unlike siggenes SAM. >> >> Thanks in advance for any help. >> >> --Jake >> >> >> [[alternative HTML version deleted]] >> >> _______________________________________________ >> Bioconductor mailing list >> Bioconductor@stat.math.ethz.ch >> https://stat.ethz.ch/mailman/listinfo/bioconductor >> >> > > -- > Geschenkt: 3 Monate GMX ProMail gratis + 3 Ausgaben stern gratis > ++ Jetzt anmelden & testen ++ http://www.gmx.net/de/go/promail ++ >