Hi Noel, > Just so that I am clear- the point is that the > bimodality is not an artifact of the convolution, but > simply the fact that the number of modes of a > distribution is not conserved under monotonous > transformations. No, I did not say that, and I do not know how to understand this sentence, since "the convolution" is directly related to "the monotonous transformation" that we are talking about > This is why the paper points to the > fact that the histograms of log2 (PMs/MMs) stratified > by log2(PMs) is bimodal I leave the exegesis of the paper to its authors. > -so bimodality is a more > general property of the probe level data. As you have just said yourself, the number of modes is not a property of the data, but of the data plus the particular (non-linear) transformation that you choose to apply to them. Best wishes Wolfgang.

Hi all, I almost always see the bi-modality, and I think Claus has the right argument, that the first peak represents not expressed/very weakly expressed genes. Most Affy chips have fairly good coverage of the genome, and hence for almost any sample a good proportion of the genes will not be expressed. I find it to (GC)RMA's credit that it identifies the non-expressed genes so clearly. Additionally, I routinely filter out genes if they are called "Absent" on all arrays by the MAS 5 algorithm, and this greatly reduces the height of the first peak. Occasionally I even filtered out genes unless they were "Present" on all arrays just to see what would happen to the distribution, and low and behold, the bimodality disappears! So as opposed to seeing the bimodality as a problem, I view it as accurately representing the real expression distribution. Cheers, Jenny At 08:17 AM 6/6/2006, Claus Mayer wrote: >Hi Wolfgang (and everybody else)! As pointed out by you there are two >different issues here: a) the bi-modality of (GC)RMA normalized data on >many chips (which I have observed repeatedly now as well ), b) the >bi-modality of log(PM/MM) values as stated in the Irrizarry et al. paper. >In both cases the mathematical argument, that any continuous distribution >can be monotonely transformed into any other continuous distribution holds >(which is basically behind your statement that monotonous transformations >do not preserve the number of peaks/modes), but I still think, that the >observation a) of bi-modal distributions of gcrma normalized expression >values is worth to be discussed. Assuming GCRMA is good/perfect >normalisation method the normalised values should directly relate to the >"true" biological expressions and thus it is tempting to take such a >histogram as an indication of there being two classes of genes: i) genes >with no/small expression values (forming the first peak), ii) >truely/highly expressed genes (forming the second peak). If on the other >hand the bi-modality is an implicit by-product of the GCRMA- normalisation, >it doesn't make sense to interpret the bi-modality biologically in that >way. I have only limited experiences with Affy arrays so far, but at >least in one case the bi-modality also occured (but not so clearly) when >using MAS5 instead of GCRMA, which I took as an indication that in this >case, that GCRMA didn't create the two modes, but just made it easier to >distinguish between them. I would be interested to hear the experiences of >others in this respect. Best Wishes Claus Wolfgang Huber wrote: > Hi, > > >I am surprised why anybody is surprised about the different number of > >modes ("peaks"): the number of modes of a distribution is not conserved > >under monotonous transformations (such as the background correction in > >RMA), this simply follows from chain rule. > > See below for a simple >example with some "mock" microarray intensities z > and density of >log-transformed values before and after a (primitive) > background >background correction. > > Cheers > Wolfgang > > > set.seed(123) > > n = >100000 > z = 20 + exp(c(rnorm(n), 3+rnorm(n))) > > par(mfrow=c(1,2)) > >plot(density(log2(z))) > plot(density(log2(z-20))) > > > >noel0925 at sbcglobal.net wrote: > >> In the paper: Exploration, >Normalization and Summaries >> of High Density Oligonucleotide Array Probe >Level Data >> the following statement regarding the >> bimodality of >log2(PM) values and RMA background >> corrected PM values can be found- >"The same bimodal >> effect is seen when we stratisfy by log2(PM), thus >it >> is not an artifact of conditioning on sums." (p4). >> I am a little >confused by this as I thought that >> indeed an artifact of the >convolution! >> >> Clearly, the background corrected intensity >> values >are given by E(S | O) or the conditional >> expectation of the signal >given what we observe; where >> the observed signal is the convolution of >a normally >> distributed background (N) mean mu variance sigma^2 >> (B~ >N(u, ????^2)) and an exponentially distributed >> signal (S) with mean >alpha (S~ exp(????)). >> >> There have been several postings regarding >this matter >> in the Bioconductor archives and all seem to point to >> >this. Have I misunderstood? >> >> In particular was the following post: >> >https://stat.ethz.ch/pipermail/bioconductor/2004-August/005908.html >> >(See below the response from zwu at jhsph.edu >> >> The original question >I got was about the bimodal >> distribution of gcrma >> result from probe >intensities with unimodel >> distribution. My answer was >> that the >"change" was not necessarily surprising. >> >> For example , when you have >"true log signal" from a >> bimodal distribution >> >logS=c(rnorm(1000,3,1),rnorm(1000,8,2)) >> # You will see this has two >peaks >> par(mfrow=c(2,2)) >> plot(density(logS)) >> #if the background, >log(non-specific binding) come >> from >> logB=rnorm(2000,6,1) >> #then >when you plot the histogram of convolution in >> log scale, >> >plot(density(log(exp(logS)+exp(logB)))) >> #you see only one peak, and >this would be "before >> gcrma". >> >> This explanation made sense to me, >but seems to >> contradict what is stated in the paper. >> >> Also, can >someone explain the difference between RMA >> background version1 vs >version2? >> >> >> Best regards, >> Noel >> >> >_______________________________________________ >> Bioconductor mailing >list >> Bioconductor at stat.math.ethz.ch >> >https://stat.ethz.ch/mailman/listinfo/bioconductor >> Search the archives: >http://news.gmane.org/gmane.science.biology.informatics.conductor >> > > > > -- >********************************************************************* ************** >Dr Claus-D. Mayer | http://www.bioss.ac.uk >Biomathematics & Statistics Scotland | email: claus at bioss.ac.uk Rowett >Research Institute | Telephone: +44 (0) 1224 716652 Aberdeen >AB21 9SB, Scotland, UK. | Fax: +44 (0) 1224 715349 >_______________________________________________ Bioconductor mailing list >Bioconductor at stat.math.ethz.ch >https://stat.ethz.ch/mailman/listinfo/bioconductor Search the archives: >http://news.gmane.org/gmane.science.biology.informatics.conductor Jenny Drnevich, Ph.D. Functional Genomics Bioinformatics Specialist W.M. Keck Center for Comparative and Functional Genomics Roy J. Carver Biotechnology Center University of Illinois, Urbana-Champaign 330 ERML 1201 W. Gregory Dr. Urbana, IL 61801 USA ph: 217-244-7355 fax: 217-265-5066 e-mail: drnevich at uiuc.edu

Hi Claus, Regarding a), I think it is not helpful to talk about the number of peaks in the distribution of microarray data, normalized or unnormalized, unless you are very precise about what transformation, pre-processing, scanner settings, etc. you apply. That is, that number is unlikely to be of any biological significance. See below. b) is an entirely different issue, it is simply an artifact of the way MMs are defined, but (GC)RMA does not use PM/MM ratios. Please try out the example I gave, it is stronger than just the trivial observation that any distribution can be mapped into any other through a suitable non-linear mapping: set.seed(123) n = 100000 z = 20 + exp(c(rnorm(n), 3+rnorm(n))) par(mfrow=c(1,3)) plot(density(log2(z))) plot(density(log2(z-20))) x = seq(min(z), max(z), length=100) plot(x, log2(2^x-20), type="l") The function x -> log2(2^x-20) is concave and looks quite "well-behaved". The densities of log2(z) and log2(z-20) look quite different, and similar effects might result from subtly different preprocessing strategies, or different scanner settings etc. Best wishes Wolfgang ------------------------------------------------------------------ Wolfgang Huber EBI/EMBL Cambridge UK http://www.ebi.ac.uk/huber Claus Mayer wrote: > Hi Wolfgang (and everybody else)! > > As pointed out by you there are two different issues here: a) the > bi-modality of (GC)RMA normalized data on many chips (which I have > observed repeatedly now as well ), b) the bi-modality of log(PM/MM) > values as stated in the Irrizarry et al. paper. > > In both cases the mathematical argument, that any continuous > distribution can be monotonely transformed into any other continuous > distribution holds (which is basically behind your statement that > monotonous transformations do not preserve the number of peaks/modes), > but I still think, that the observation a) of bi-modal distributions of > gcrma normalized expression values is worth to be discussed. > Assuming GCRMA is good/perfect normalisation method the normalised > values should directly relate to the "true" biological expressions and > thus it is tempting to take such a histogram as an indication of there > being two classes of genes: i) genes with no/small expression values > (forming the first peak), ii) truely/highly expressed genes (forming the > second peak). > If on the other hand the bi-modality is an implicit by-product of the > GCRMA-normalisation, it doesn't make sense to interpret the bi- modality > biologically in that way. > > I have only limited experiences with Affy arrays so far, but at least > in one case the bi-modality also occured (but not so clearly) when using > MAS5 instead of GCRMA, which I took as an indication that in this case, > that GCRMA didn't create the two modes, but just made it easier to > distinguish between them. I would be interested to hear the experiences > of others in this respect. > > Best Wishes > > Claus >

Hi, Wofgang, Yes, mRNA abundances do indeed span the whole range. What I meant was that all the distributions of intensities I have observed seem to be poorly modeled by a single mathematical distribution (that is what I meant by my poor choice of the term "truly unimodal"). Rather, two overlaid (added) distributions seem to model the observed data better, with the first distribution (presumably "absent") spanning the lower part of the range, and the second ("present", presumably modeling the mRNA abundances) spanning the *entire* range, but with a higher mean. The lower distribution would represent the much larger set of probes whose intensities are due only to cross-hyb, NSB, and background, with no true target mRNA signal. Its density peak is therefore much higher than the other. Although there is significant overlap between the two, the two means are separated and distinct, so the sum is a bimodal distribution. Again, this is simply based on my observations of log2-transformed intensity values. In fact, isn't it the main purpose of any of the intensity processing methods (MAS, RMA, GCRMA, etc.) to detect and increase the difference between the two distributions, so as to help distinguish signal from noise? Regards, - Peter > -----Original Message----- > From: Wolfgang Huber [mailto:huber at ebi.ac.uk] > Sent: Tuesday, June 06, 2006 12:15 PM > To: Peter G. Warren > Cc: noel0925 at sbcglobal.net; bioconductor at stat.math.ethz.ch > Subject: Re: [BioC] RMA-bimodality: > > Hi Peter, > > - doesn't the distribution of mRNA abundances (i.e. physical > concentrations measured e.g. in average no. of molecules per > cell) span the whole range from just undetectably above zero > to very large? I am not sure what mechanism would then result > in two distinct peaks of fluorescences, one for "absent" and > and one for "present" mRNAs. > > - I tried find a definition of "truely unimodal > distributions" (and I suppose, "falsely unimodal > distributions"), but couldn't find one, can you advise? > > Cheers > Wolfgang > > Peter G. Warren wrote: > > Hi, Wolfgang, Noel, > > > > It is true that a non-linear transformation can change the > number of > > nodes of the data, and that that transformation can be > sufficient to > > explain the bimodality we see in background-corrected data. > However, > > in my experience, the raw probe-level data is itself bimodal. When > > there is some real signal present, the probe-level intensities are > > actually from two different distributions. The first ("absent") is > > where there is no positive transcript binding, only cross-hyb, > > non-specific binding, and background. The second > > ("present") is all that, plus true target transcript binding. This > > bimodality is more evident with log-transformed values. (In > contrast, > > a log-transformation of a truly unimodal distribution, such as > > density(rnorm(...), is still unimodal.) In every case I've > looked at, > > the "absent" distribution dwarfs the "present" one, so it > often looks > > like one mode, before log transformation. After log > transformation, I > > have been unable to model the data successfully with a single > > distribution; it always takes two. > > > > Regards, > > - Peter Warren > > > >> Hi Noel, > >> > >>> Just so that I am clear- the point is that the bimodality > is not an > >>> artifact of the convolution, but simply the fact that the > number of > >>> modes of a distribution is not conserved under monotonous > >>> transformations. > >> No, I did not say that, and I do not know how to understand this > >> sentence, since "the convolution" is directly related to "the > >> monotonous transformation" that we are talking about > >> > >>> This is why the paper points to the > >>> fact that the histograms of log2 (PMs/MMs) stratified by > log2(PMs) > >>> is bimodal > >> I leave the exegesis of the paper to its authors. > >> > >>> -so bimodality is a more > >>> general property of the probe level data. > >> As you have just said yourself, the number of modes is not > a property > >> of the data, but of the data plus the particular (non-linear) > >> transformation that you choose to apply to them. > >> > >> > >> Best wishes > >> Wolfgang. > > > > > -- > ------------------------------------------------------------------ > Wolfgang Huber EBI/EMBL Cambridge UK http://www.ebi.ac.uk/huber > >

Hi, Wolfgang, Just a short follow-up. The example you provided to Noel has embedded within it two distributions. Your example is as follows: set.seed(123) n = 100000 z = 20 + exp(c(rnorm(n), 3+rnorm(n))) par(mfrow=c(1,2)) plot(density(log2(z))) plot(density(log2(z-20))) Continuing, if we separate the two distributions in z and overlay them, the plot illustrates the point I've been attempting to make: z1=20+exp(rnorm(n)) z2=20+exp(3+rnorm(n)) # Note that z = c(z1+z2) # The left plot shows your bimodal combined density, before the "background correction", as before. plot(density(log2(z))) # The right plot shows the two component distributions separately. plot(density(log2(z1))) lines(density(log2(z2))) Your example in fact models quite well what I see with real, uncorrected data. If the two distributions are not "absent" and "present" intensity distributions, I'm open to other suggestions. Regards, - Peter > -----Original Message----- > From: Wolfgang Huber [mailto:huber at ebi.ac.uk] > Sent: Tuesday, June 06, 2006 12:15 PM > To: Peter G. Warren > Cc: noel0925 at sbcglobal.net; bioconductor at stat.math.ethz.ch > Subject: Re: [BioC] RMA-bimodality: > > Hi Peter, > > - doesn't the distribution of mRNA abundances (i.e. physical > concentrations measured e.g. in average no. of molecules per > cell) span the whole range from just undetectably above zero > to very large? I am not sure what mechanism would then result > in two distinct peaks of fluorescences, one for "absent" and > and one for "present" mRNAs. > > - I tried find a definition of "truely unimodal > distributions" (and I suppose, "falsely unimodal > distributions"), but couldn't find one, can you advise? > > Cheers > Wolfgang > > Peter G. Warren wrote: > > Hi, Wolfgang, Noel, > > > > It is true that a non-linear transformation can change the > number of > > nodes of the data, and that that transformation can be > sufficient to > > explain the bimodality we see in background-corrected data. > However, > > in my experience, the raw probe-level data is itself bimodal. When > > there is some real signal present, the probe-level intensities are > > actually from two different distributions. The first ("absent") is > > where there is no positive transcript binding, only cross-hyb, > > non-specific binding, and background. The second > > ("present") is all that, plus true target transcript binding. This > > bimodality is more evident with log-transformed values. (In > contrast, > > a log-transformation of a truly unimodal distribution, such as > > density(rnorm(...), is still unimodal.) In every case I've > looked at, > > the "absent" distribution dwarfs the "present" one, so it > often looks > > like one mode, before log transformation. After log > transformation, I > > have been unable to model the data successfully with a single > > distribution; it always takes two. > > > > Regards, > > - Peter Warren > > > >> Hi Noel, > >> > >>> Just so that I am clear- the point is that the bimodality > is not an > >>> artifact of the convolution, but simply the fact that the > number of > >>> modes of a distribution is not conserved under monotonous > >>> transformations. > >> No, I did not say that, and I do not know how to understand this > >> sentence, since "the convolution" is directly related to "the > >> monotonous transformation" that we are talking about > >> > >>> This is why the paper points to the > >>> fact that the histograms of log2 (PMs/MMs) stratified by > log2(PMs) > >>> is bimodal > >> I leave the exegesis of the paper to its authors. > >> > >>> -so bimodality is a more > >>> general property of the probe level data. > >> As you have just said yourself, the number of modes is not > a property > >> of the data, but of the data plus the particular (non-linear) > >> transformation that you choose to apply to them. > >> > >> > >> Best wishes > >> Wolfgang. > > > > > -- > ------------------------------------------------------------------ > Wolfgang Huber EBI/EMBL Cambridge UK http://www.ebi.ac.uk/huber > >