Title: RnBeads - save state between data crunching and report generation checkpoints within the differential methylation module?
I have a 450K dataset that I have been trying to get enrichment results from. I have set up RnBeads on a monitorless cluster, without an X server and have had to forward my X11 (image generation) processing to a personal laptop which has a monitor (and can thus "be drawn on"). Because of this, I suspect, and other unknown variables, RnBeads runs very slowly (>72 hours ), especially in the report generation stage of the differential methylation module.
Oftentimes, the X11 forwarding hack we have set up fails when my VPN connection to the cluster fails (for any network related reason, as 72 hours is a long time to expect no network interruption), and all of the data crunching and progress made in the differential methylation module of the pipeline, is lost forever, because the RnBeads object does not get updated until the entire module completes - which is AFTER the successful completion of the report generation.
I need help in finding a way to save the data crunching part of the differential analysis, before we start the report generation / image generation part of the differential methylation module. Alternatively, can I skip the image generation of the differential methylation images (mean beta vs. mean beta, volcano plots, etc) and jump right to the enrichment analysis image generation?
Have you considered using Xvfb? It's a virtual X11 framebuffer. Essentially it is 'headless' X11.
With Xvfb you should be able to get rid of the need for a monitor and a VPN connection to your laptop.
We tried this, and it works, over all. However, it does not work when I set the differential methylation module method to "limma". I have no idea why LIMMA here is an exception? It works for the "ttest" method, though? Great suggestion, Dan!