Dear Tobias,
Multi-level experiments (with comparisons both between and within subjects) are quite complex to analyse.
edgeR can fit arbitrary linear models, but it cannot estimate multiple levels of variation. For an experiment such as yours, edgeR can test for effects within subjects, and it can test for interactions between subject groups for these effects. It cannot however test for marginal differences between the patient groups, because doing so would require estimating the variability of the patients as well as estimating variability of expression values within each patient.
For your experiment, you ask how to test for DE genes between resistant and sensitive cells in the control. The short answer is that you can't using edgeR. (Nor with other negative binomial based packages.)
I would suggest instead that you switch to limma-voom. This will allow you to estimate the intra-patient correlation (using duplicateCorrelation) instead of including subject in the design matrix. You will then be able to test any comparisons you wish between resistance and treatment groups.
The limma-voom approach to experiments such as yours has been discussed at some length on this mailing list, see for example:
https://stat.ethz.ch/pipermail/bioconductor/2014-May/059713.html
and the subsequent links.
Best wishes
Gordon
Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Gordon Smyth Dear Dr. Smyth, in this design, I am interested to extract genes that have different expression profile at any of the time points (2h and 24h after adjusting for baseline) between the two treatment groups, can you please advise how to set up the contrast properly? Thank you very much!