Hello,
I have a question about identifying highly variable genes after applying vst from deseq2. I don't think it makes much sense to find those genes after variance stabilization. Should I identify the genes using counts or normalized counts and directly use the gene list on the post-vst data for any downstream analysis?
Thanks in advance
The variance of counts on log2 scale is not equal along the average expression (baseMean), it's higher for low counts than for high counts, that is called overdispersion. The whole point of the vst() is to remove this technical trend and retain only "reliable" variance that is due to sample differences, not due to expression level. Hence, it makes perfect sense to select variable genes after vst(), thereby enriching for reliable between-sample variance, rather than trended variance due to expression level and noise.
Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
version 2.3.6
Thanks so much for the clarification!
My rna-seq dataset contains samples from multiple tissues (heart, kidney, spleen etc). Biologically we expect that samples from different tissue sites to have very different transcriptomic profiles. Should I apply vst separately for each of these tissues, or should I treat the entire dataset across tissues as a whole?
That really depends on the question you want to answer. If you want genes variable between organs than you must run the vst for all in the same analysis. See also the DESeq2 vignette on the question whether to split a dataset into pieces or whether to keep them together.