Hello, I'm quite new to Bioconductor/affy, and I was wondering if there's a simple way to include the absent/present call for a gene in the outputfile generated with write.exprs(eset, file='boo') in theaffy package. the eset was generated with eset <- expresso(cel, bgcorrect.method = 'rma', normalize.method = 'qspline', pmcorrect.method = 'pmonly', summary.method='liwong') For further analyses I'd like to exclude genes that are absent in all chips. thanks a lot for your help, Arne

<arne.muller@aventis.com> writes: > Hello, > > I'm quite new to Bioconductor/affy, and I was wondering if there's a simple > way to include the absent/present call for a gene in the outputfile generated > with write.exprs(eset, file='boo') in theaffy package. > > the eset was generated with > > eset <- expresso(cel, bgcorrect.method = 'rma', normalize.method = > 'qspline', pmcorrect.method = 'pmonly', > summary.method='liwong') > > For further analyses I'd like to exclude genes that are absent in all chips. That's tough. It isn't clear what a sensible definition of absent is. Or present. Do you mean "expressed" ? "Differentially expressed" ? "sort of differentially expressed but not too weakly expressed?". For any of these, you'll need a precise definition (there isn't any in Bioconductor), and you can compute your own. (I know that MAS will make these calls; I'm only familiar with Rosetta Resolver's variant, and they don't really make sense to me -- to be precise, I know numerically how they are derived, but fail to why they realistically connect biologically or technologically without a great deal of assumptions and a wild imagination). best, -tony -- rossini@u.washington.edu http://www.analytics.washington.edu/ Biomedical and Health Informatics University of Washington Biostatistics, SCHARP/HVTN Fred Hutchinson Cancer Research Center UW (Tu/Th/F): 206-616-7630 FAX=206-543-3461 | Voicemail is unreliable FHCRC (M/W): 206-667-7025 FAX=206-667-4812 | use Email CONFIDENTIALITY NOTICE: This e-mail message and any attachme...{{dropped}}

Hi, I get your point with interpreting absent/present calls. Technically it's a nice feature, becasue one can just discard the majority of the genes on the chip for further analysis. In fact I think absent/present calls make sense in terms of biology, since just a fraction of the genes are realy expressed at a time. How to express this numerially is a different story (and I guess a difficult one). Anyway, with MAS the calls are calculated anyway, can't they? So, I'd be nice (at least for "completness") to add a "mascall" method to the exprSet objects generated by affy. What do you think? By the way, if you ignore the call, do you set an arbitrary intensity cutoff later in your analysis, or do just reley on the statistics (anova p-value or whatever)? regards, Arne > -----Original Message----- > From: A.J. Rossini [mailto:rossini@blindglobe.net] > Sent: 08 October 2003 15:33 > To: Muller, Arne PH/FR > Cc: bioconductor@stat.math.ethz.ch > Subject: Re: [BioC] Affy: Present calls in an eset > > > <arne.muller@aventis.com> writes: > > > Hello, > > > > I'm quite new to Bioconductor/affy, and I was wondering if > there's a simple > > way to include the absent/present call for a gene in the > outputfile generated > > with write.exprs(eset, file='boo') in theaffy package. > > > > the eset was generated with > > > > eset <- expresso(cel, bgcorrect.method = 'rma', > normalize.method = > > 'qspline', pmcorrect.method = 'pmonly', > > summary.method='liwong') > > > > For further analyses I'd like to exclude genes that are > absent in all chips. > > That's tough. It isn't clear what a sensible definition of absent > is. Or present. > > Do you mean "expressed" ? "Differentially expressed" ? "sort of > differentially expressed but not too weakly expressed?". For any of > these, you'll need a precise definition (there isn't any in > Bioconductor), and you can compute your own. > > (I know that MAS will make these calls; I'm only familiar with Rosetta > Resolver's variant, and they don't really make sense to me -- to be > precise, I know numerically how they are derived, but fail to why they > realistically connect biologically or technologically without a great > deal of assumptions and a wild imagination). > > best, > -tony > > -- > rossini@u.washington.edu > http://www.analytics.washington.edu/ > Biomedical and Health Informatics University of Washington > Biostatistics, SCHARP/HVTN Fred Hutchinson Cancer > Research Center > UW (Tu/Th/F): 206-616-7630 FAX=206-543-3461 | Voicemail is unreliable > FHCRC (M/W): 206-667-7025 FAX=206-667-4812 | use Email > > CONFIDENTIALITY NOTICE: This e-mail message and any attachments may be > confidential and privileged. If you received this message in error, > please destroy it and notify the sender. Thank you. >

Hi, my understading ot the A/P calls from Affy MAS5 is that they represent a way to "evaluate" the background signal. In that way you could have a high background, ie, a high MisM signal, and despite the PM signal is identical, for instance 500, the this gene is going to be considered as Absent, which could be that your gene is not expressed, but could also be due to the design of the probe set rather than to the fact that the gene is not expressed.... Detecting if a gene is expressed or not with Affy chips is for me almost another question, a difficult one ! my threshold is 70 (2^RMA value) but it is likely that at this value some genes are expressed it is hust that below tis value I consider, arbitrarly, that it is difficult to validate something because of the noise. Anybody can correct me if I m wrong with all this since I still new in ths field as well ! Hope it helps Philippe [[alternative HTML version deleted]]

<arne.muller@aventis.com> writes: > I get your point with interpreting absent/present calls. Technically it's a > nice feature, becasue one can just discard the majority of the genes on the > chip for further analysis. In fact I think absent/present calls make sense in > terms of biology, since just a fraction of the genes are realy expressed at a > time. How to express this numerially is a different story (and I guess a > difficult one). Sure, but equivalently, I could implement the "feature" with "Tony's Complete Crud Calls" (TCCC): rbinom(40000,1,0.2) Change the 20% expressed to a different number if it makes you feel better. I'm not being quite fair, but from my perspective, it's only slightly less realistic than the MAS or Resolver calls -- the difference being that the probability of response is an unclear function of the expression level, with a trend where the probabilty of the call goes up with the expression level (which is the technical component). And then, you've got the possibility of low levels of expression which are real but hard to pick out. > Anyway, with MAS the calls are calculated anyway, can't they? So, I'd be nice > (at least for "completness") to add a "mascall" method to the exprSet objects > generated by affy. What do you think? It would be great. As far as I know, the MAS calls are protected as a trade secret and non-disclosure agreements. Maybe not, but I happen to know for sure that the rosetta resolver calls are! (or I'd have implemented them for amusement) > By the way, if you ignore the call, do you set an arbitrary intensity cutoff > later in your analysis, or do just reley on the statistics (anova p-value or > whatever)? If you have biological replicates, then either, depending on what you are looking for. I've never found an approach which works all the time. If you have technical replicates, then the calls are necessary, but note that they don't have biological meaning, just technical (technology) meaning. best, -tony -- rossini@u.washington.edu http://www.analytics.washington.edu/ Biomedical and Health Informatics University of Washington Biostatistics, SCHARP/HVTN Fred Hutchinson Cancer Research Center UW (Tu/Th/F): 206-616-7630 FAX=206-543-3461 | Voicemail is unreliable FHCRC (M/W): 206-667-7025 FAX=206-667-4812 | use Email CONFIDENTIALITY NOTICE: This e-mail message and any attachme...{{dropped}}

Hi, my understading ot the A/P calls from Affy MAS5 is that they represent a way to "evaluate" the background signal. In that way you could have a high background, ie, a high MisM signal, and despite the PM signal is identical, for instance 500, the this gene is going to be considered as Absent, which could be that your gene is not expressed, but could also be due to the design of the probe set rather than to the fact that the gene is not expressed.... Detecting if a gene is expressed or not with Affy chips is for me almost another question, a difficult one ! my threshold is 70 (2^RMA value) but it is likely that at this value some genes are expressed it is hust that below tis value I consider, arbitrarly, that it is difficult to validate something because of the noise. Anybody can correct me if I m wrong with all this since I still new in ths field as well ! Hope it helps Philippe [[alternative HTML version deleted]]

Hi, MAS5.0 gives you two numbers, the expression level for each gene, and a detection p-value that tells you how confident MAS5.0 is that it got the expression level right. This p-value is then split into three categories (simply by thresholding) to give you Present, Marginal or Absent calls. Generally, thresholding by intensity does something similar to using the detection p-value, but not always, for every gene, in every situation. Keeping these two concepts distinct (accuracy and confidence), is IMHO not a bad idea (we can of course argue about how best to calculate metrics for these :-)). After all, this seems to be what we are doing when we generate not only a fold-change but also a p-score across replicates for each gene in a study. Crispin > -----Original Message----- > From: Phguardiol@aol.com [mailto:Phguardiol@aol.com] > Sent: 08 October 2003 15:29 > To: Arne.Muller@aventis.com; bioconductor@stat.math.ethz.ch > Subject: Re: [BioC] Affy: Present calls in an eset > > > Hi, > my understading ot the A/P calls from Affy MAS5 is that they > represent a way > to "evaluate" the background signal. > In that way you could have a high background, ie, a high MisM > signal, and > despite the PM signal is identical, for instance 500, the > this gene is going to > be considered as Absent, which could be that your gene is not > expressed, but > could also be due to the design of the probe set rather than > to the fact that > the gene is not expressed.... > > Detecting if a gene is expressed or not with Affy chips is > for me almost > another question, a difficult one ! my threshold is 70 (2^RMA > value) but it is > likely that at this value some genes are expressed it is hust > that below tis > value I consider, arbitrarly, that it is difficult to > validate something because > of the noise. > Anybody can correct me if I m wrong with all this since I > still new in ths > field as well ! > Hope it helps > Philippe > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioconductor mailing list > Bioconductor@stat.math.ethz.ch > https://www.stat.math.ethz.ch/mailman/listinfo> /bioconductor > -------------------------------------------------------- This email is confidential and intended solely for the use o...{{dropped}}

That's great news, Rafael! Thanks! The absent/present calls are already included in the TXT files from the Affy software (based on MAS). I was wondering if it's possible to getse files into the Bioconductor affy package. This's be some kind of high level analysis, since the "summary" per gene (the intensity) is already calculated by the Affymetrix software based on PM/MM. As understand the Bioconductor affy package, the normalisation is done one the probeset level (for each of the 40 oligos per gene), rather than the gene's intensity (affy normalisation is done *before* the probesets are summarised into genes). This is somehow different of what we (and others) do - although I guess it's a good idea to perform normalisation at the "lowest" end, ie he probesets. Is it possible to apply the "normalize.methods" to background + pm corrected and summarised genes; i.e. 1. read probe level data 2. background corredtion 3. probe specific bg correction (e.g. substracting MM) 4. Summarizing the probe set values into one expression value 5. normalize summarized expression values kind regards, Arne > -----Original Message----- > From: Rafael A. Irizarry [mailto:ririzarr@jhsph.edu] > Sent: 08 October 2003 16:54 > To: Muller, Arne PH/FR > Cc: rossini@u.washington.edu; bioconductor@stat.math.ethz.ch > Subject: RE: [BioC] Affy: Present calls in an eset > > > we will *try* to have mascall fro the next release. > > On Wed, 8 Oct 2003 Arne.Muller@aventis.com wrote: > > > Hi, > > > > I get your point with interpreting absent/present calls. > Technically it's a > > nice feature, becasue one can just discard the majority of > the genes on the > > chip for further analysis. In fact I think absent/present > calls make sense in > > terms of biology, since just a fraction of the genes are > realy expressed at a > > time. How to express this numerially is a different story > (and I guess a > > difficult one). > > > > Anyway, with MAS the calls are calculated anyway, can't > they? So, I'd be nice > > (at least for "completness") to add a "mascall" method to > the exprSet objects > > generated by affy. What do you think? > > > > By the way, if you ignore the call, do you set an arbitrary > intensity cutoff > > later in your analysis, or do just reley on the statistics > (anova p-value or > > whatever)? > > > > regards, > > > > Arne > > > > > > > -----Original Message----- > > > From: A.J. Rossini [mailto:rossini@blindglobe.net] > > > Sent: 08 October 2003 15:33 > > > To: Muller, Arne PH/FR > > > Cc: bioconductor@stat.math.ethz.ch > > > Subject: Re: [BioC] Affy: Present calls in an eset > > > > > > > > > <arne.muller@aventis.com> writes: > > > > > > > Hello, > > > > > > > > I'm quite new to Bioconductor/affy, and I was wondering if > > > there's a simple > > > > way to include the absent/present call for a gene in the > > > outputfile generated > > > > with write.exprs(eset, file='boo') in theaffy package. > > > > > > > > the eset was generated with > > > > > > > > eset <- expresso(cel, bgcorrect.method = 'rma', > > > normalize.method = > > > > 'qspline', pmcorrect.method = 'pmonly', > > > > summary.method='liwong') > > > > > > > > For further analyses I'd like to exclude genes that are > > > absent in all chips. > > > > > > That's tough. It isn't clear what a sensible definition of absent > > > is. Or present. > > > > > > Do you mean "expressed" ? "Differentially expressed" ? "sort of > > > differentially expressed but not too weakly expressed?". > For any of > > > these, you'll need a precise definition (there isn't any in > > > Bioconductor), and you can compute your own. > > > > > > (I know that MAS will make these calls; I'm only familiar > with Rosetta > > > Resolver's variant, and they don't really make sense to > me -- to be > > > precise, I know numerically how they are derived, but > fail to why they > > > realistically connect biologically or technologically > without a great > > > deal of assumptions and a wild imagination). > > > > > > best, > > > -tony > > > > > > -- > > > rossini@u.washington.edu > > > http://www.analytics.washington.edu/ > > > Biomedical and Health Informatics University of Washington > > > Biostatistics, SCHARP/HVTN Fred Hutchinson Cancer > > > Research Center > > > UW (Tu/Th/F): 206-616-7630 FAX=206-543-3461 | Voicemail > is unreliable > > > FHCRC (M/W): 206-667-7025 FAX=206-667-4812 | use Email > > > > > > CONFIDENTIALITY NOTICE: This e-mail message and any > attachments may be > > > confidential and privileged. If you received this message > in error, > > > please destroy it and notify the sender. Thank you. > > > > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > https://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > > >

I actually asked about this a while back: https://www.stat.math.ethz.ch/pipermail/bioconductor/2003-August/00208 1.html The algorithms aren't too hard to code, although I haven't found the time to try it yet. Overall, I don't think the calls are particularly valuable: as Tony says, if you read the algorithm you start to wonder at how arbitrary it all is. On the other hand, I agree that it would be nice to be replicate this part of MAS5 with Affy. Paul -----Original Message----- Date: Wed, 8 Oct 2003 15:47:42 +0200 From: <arne.muller@aventis.com> Subject: RE: [BioC] Affy: Present calls in an eset To: <rossini@u.washington.edu> Cc: bioconductor@stat.math.ethz.ch Message-ID: <c80ecafa2acc1b45be45d133ed660ade410b0e@crbsmxsusr04.pharma.aventis.co m=""> Content-Type: text/plain; charset="iso-8859-1" Hi, I get your point with interpreting absent/present calls. Technically it's a nice feature, becasue one can just discard the majority of the genes on the chip for further analysis. In fact I think absent/present calls make sense in terms of biology, since just a fraction of the genes are realy expressed at a time. How to express this numerially is a different story (and I guess a difficult one). Anyway, with MAS the calls are calculated anyway, can't they? So, I'd be nice (at least for "completness") to add a "mascall" method to the exprSet objects generated by affy. What do you think? By the way, if you ignore the call, do you set an arbitrary intensity cutoff later in your analysis, or do just reley on the statistics (anova p-value or whatever)? regards, Arne > -----Original Message----- > From: A.J. Rossini [mailto:rossini@blindglobe.net] > Sent: 08 October 2003 15:33 > To: Muller, Arne PH/FR > Cc: bioconductor@stat.math.ethz.ch > Subject: Re: [BioC] Affy: Present calls in an eset > > > <arne.muller@aventis.com> writes: > > > Hello, > > > > I'm quite new to Bioconductor/affy, and I was wondering if > there's a simple > > way to include the absent/present call for a gene in the > outputfile generated > > with write.exprs(eset, file='boo') in theaffy package. > > > > the eset was generated with > > > > eset <- expresso(cel, bgcorrect.method = 'rma', > normalize.method = > > 'qspline', pmcorrect.method = 'pmonly', > > summary.method='liwong') > > > > For further analyses I'd like to exclude genes that are > absent in all chips. > > That's tough. It isn't clear what a sensible definition of absent > is. Or present. > > Do you mean "expressed" ? "Differentially expressed" ? "sort of > differentially expressed but not too weakly expressed?". For any of > these, you'll need a precise definition (there isn't any in > Bioconductor), and you can compute your own. > > (I know that MAS will make these calls; I'm only familiar with Rosetta > Resolver's variant, and they don't really make sense to me -- to be > precise, I know numerically how they are derived, but fail to why they > realistically connect biologically or technologically without a great > deal of assumptions and a wild imagination). > > best, > -tony > > -- > rossini@u.washington.edu > http://www.analytics.washington.edu/ > Biomedical and Health Informatics University of Washington > Biostatistics, SCHARP/HVTN Fred Hutchinson Cancer > Research Center > UW (Tu/Th/F): 206-616-7630 FAX=206-543-3461 | Voicemail is unreliable > FHCRC (M/W): 206-667-7025 FAX=206-667-4812 | use Email > > CONFIDENTIALITY NOTICE: This e-mail message and any attachments may be > confidential and privileged. If you received this message in error, > please destroy it and notify the sender. Thank you. >

Hi, it's probably also worth remembering that %present calls are widely used by the Affy community as part of their standard QC proceedure to check that arrays within a project are performing sensibly. Crispin -------------------------------------------------------- This email is confidential and intended solely for the use o...{{dropped}}

As regards calculating P and A calls like MAS5--I would have imagined that this has all the problems inherent in calculating MAS5 values. The gcrma algorithm appears to do a better job of calculating real background from pooling MM probes of similar gc content. I would have thought this would be best for P and A calls too??? Stephen Henderson ********************************************************************** This email and any files transmitted with it are confidentia...{{dropped}}