MEDIPS
1
0
Entering edit mode
@paolo-kunderfranco-5158
Last seen 7.4 years ago
Dear All, I am facing some problems with MEDIPS package, I Think I'am running out of memory once I run the command MEDIPS.meth ................ Adjusting p.values for multiple testing... Creating results table... Adding differential coverage results... Error: cannot allocate vector of size 207.9 Mb Inoltre: Warning messages: 1: In unlist(vlist, recursive = FALSE, use.names = FALSE) : Reached total allocation of 24501Mb: see help(memory.size) 2: In unlist(vlist, recursive = FALSE, use.names = FALSE) : Reached total allocation of 24501Mb: see help(memory.size) 3: In unlist(vlist, recursive = FALSE, use.names = FALSE) : Reached total allocation of 24501Mb: see help(memory.size) 4: In unlist(vlist, recursive = FALSE, use.names = FALSE) : Reached total allocation of 24501Mb: see help(memory.size) Is there a way to avoid this problem? I have a 24 Gb RAM workstation I don't see why it fails... Thanks, Paolo > sessionInfo() R version 3.0.0 (2013-04-03) Platform: x86_64-w64-mingw32/x64 (64-bit) locale: [1] LC_COLLATE=Italian_Italy.1252 LC_CTYPE=Italian_Italy.1252 LC_MONETARY=Italian_Italy.1252 LC_NUMERIC=C LC_TIME=Italian_Italy.1252 attached base packages: [1] parallel stats graphics grDevices utils datasets methods base other attached packages: [1] MEDIPS_1.10.0 BiocInstaller_1.10.3 gtools_3.1.0 DNAcopy_1.34.0 BSgenome_1.28.0 Biostrings_2.28.0 GenomicRanges_1.12.4 IRanges_1.18.1 BiocGenerics_0.6.0 loaded via a namespace (and not attached): [1] biomaRt_2.16.0 bitops_1.0-6 edgeR_3.2.4 limma_3.16.8 RCurl_1.95-4.1 Rsamtools_1.12.4 stats4_3.0.0 tools_3.0.0 XML_3.98-1.1 zlibbioc_1.6.0
Coverage MEDIPS Coverage MEDIPS • 1.4k views
ADD COMMENT
0
Entering edit mode
Lukas Chavez ▴ 570
@lukas-chavez-5781
Last seen 6.8 years ago
USA/La Jolla/UCSD
Hi Paolo, too bad, you were almost there! When MEDIPS stopped processing due to memory limitations, differential coverage was already calculated. Unfortunately, the last step, i.e. creating the output result table, was too memory intensive for your set up. May I ask how many samples of which species do you process and what is the chosen window size? These are the parameters that influence memory requirements for the differential coverage analysis. While I will investigate, how to further reduce memory requirements in future versions, I see two immediate possible solutions without reducing window and sample sizes: i) migrate your analysis to a cluster with more memory, ii) analyze your data in chunks. While I understand that i) is not always available, this is what you can try for solution ii): Update to MEDIPS version >= 1.11.16. This is currently available at http://www.bioconductor.org/packages/2.13/bioc/html/MEDIPS.html and will be available as version 1.12.0 with the release of Bioconductor 2.13 on October 15th 2013. In MEDIPS version >= 1.11.16, the function MEDIPS.meth() has the parameter "chr" which allows to process only a set of selected chromosomes. In your case, I expect that it will be sufficient to divide the genome into two groups of chromosomes. Therefore, you can try running the MEDIPS.meth() function twice like: res = MEDIPS.meth(MSet1 = MSets_groupA, MSet2 = MSets_groupB, chr = chr_names(MSets_groupA[[1]])[1:x], ...) resB = MEDIPS.meth(MSet1 = MSets_groupA, MSet2 = MSets_groupB, chr = chr_names(MSets_groupA[[1]])[x+1:n], ...) where n is the number of chromosomes included in your MEDIPS Sets and x is an arbitrary number that divides your chromosomes into two groups (e.g. round(n/2)). Please note that some modeled parameters (e.g. for library size normalization) will be slightly different for two subsets of the data compared to processing a genome wide table at once. Finally, you can combine the result tables by res = rbind(res, res_B) rm(res_B) and continue with a normal workflow. All the best, Lukas On Wed, Oct 9, 2013 at 3:33 AM, Paolo Kunderfranco < paolo.kunderfranco@gmail.com> wrote: > Dear All, > I am facing some problems with MEDIPS package, > I Think I'am running out of memory once I run the command MEDIPS.meth > ................ > Adjusting p.values for multiple testing... > Creating results table... > Adding differential coverage results... > Error: cannot allocate vector of size 207.9 Mb > Inoltre: Warning messages: > 1: In unlist(vlist, recursive = FALSE, use.names = FALSE) : > Reached total allocation of 24501Mb: see help(memory.size) > 2: In unlist(vlist, recursive = FALSE, use.names = FALSE) : > Reached total allocation of 24501Mb: see help(memory.size) > 3: In unlist(vlist, recursive = FALSE, use.names = FALSE) : > Reached total allocation of 24501Mb: see help(memory.size) > 4: In unlist(vlist, recursive = FALSE, use.names = FALSE) : > Reached total allocation of 24501Mb: see help(memory.size) > > Is there a way to avoid this problem? > > I have a 24 Gb RAM workstation I don't see why it fails... > > Thanks, > Paolo > > > > sessionInfo() > R version 3.0.0 (2013-04-03) > Platform: x86_64-w64-mingw32/x64 (64-bit) > > locale: > [1] LC_COLLATE=Italian_Italy.1252 LC_CTYPE=Italian_Italy.1252 > LC_MONETARY=Italian_Italy.1252 LC_NUMERIC=C > LC_TIME=Italian_Italy.1252 > > attached base packages: > [1] parallel stats graphics grDevices utils datasets > methods base > > other attached packages: > [1] MEDIPS_1.10.0 BiocInstaller_1.10.3 gtools_3.1.0 > DNAcopy_1.34.0 BSgenome_1.28.0 Biostrings_2.28.0 > GenomicRanges_1.12.4 IRanges_1.18.1 BiocGenerics_0.6.0 > > loaded via a namespace (and not attached): > [1] biomaRt_2.16.0 bitops_1.0-6 edgeR_3.2.4 limma_3.16.8 > RCurl_1.95-4.1 Rsamtools_1.12.4 stats4_3.0.0 tools_3.0.0 > XML_3.98-1.1 zlibbioc_1.6.0 > [[alternative HTML version deleted]]
ADD COMMENT
0
Entering edit mode
Hi Lukas, Specie is mouse, 4 samples plus 2 input, and ws=100. The memory fails when I run MEDIPS.meth for differential analysis between two samples and 1 input. I bypassed the error as you suggested running 3 MEDIPS.meth with chromosomes groups Look forward for the new release! Cheers, Paolo 2013/10/9 Lukas Chavez <lukas.chavez.mailings at="" googlemail.com="">: > > Hi Paolo, > > too bad, you were almost there! When MEDIPS stopped processing due to memory > limitations, differential coverage was already calculated. Unfortunately, > the last step, i.e. creating the output result table, was too memory > intensive for your set up. May I ask how many samples of which species do > you process and what is the chosen window size? These are the parameters > that influence memory requirements for the differential coverage analysis. > > While I will investigate, how to further reduce memory requirements in > future versions, I see two immediate possible solutions without reducing > window and sample sizes: i) migrate your analysis to a cluster with more > memory, ii) analyze your data in chunks. While I understand that i) is not > always available, this is what you can try for solution ii): > > Update to MEDIPS version >= 1.11.16. This is currently available at > http://www.bioconductor.org/packages/2.13/bioc/html/MEDIPS.html and will be > available as version 1.12.0 with the release of Bioconductor 2.13 on October > 15th 2013. In MEDIPS version >= 1.11.16, the function MEDIPS.meth() has the > parameter "chr" which allows to process only a set of selected chromosomes. > In your case, I expect that it will be sufficient to divide the genome into > two groups of chromosomes. Therefore, you can try running the MEDIPS.meth() > function twice like: > > res = MEDIPS.meth(MSet1 = MSets_groupA, MSet2 = MSets_groupB, chr = > chr_names(MSets_groupA[[1]])[1:x], ...) > resB = MEDIPS.meth(MSet1 = MSets_groupA, MSet2 = MSets_groupB, chr = > chr_names(MSets_groupA[[1]])[x+1:n], ...) > > where n is the number of chromosomes included in your MEDIPS Sets and x is > an arbitrary number that divides your chromosomes into two groups (e.g. > round(n/2)). Please note that some modeled parameters (e.g. for library size > normalization) will be slightly different for two subsets of the data > compared to processing a genome wide table at once. > > Finally, you can combine the result tables by > > res = rbind(res, res_B) > rm(res_B) > > and continue with a normal workflow. > > All the best, > Lukas > > > > > > > On Wed, Oct 9, 2013 at 3:33 AM, Paolo Kunderfranco > <paolo.kunderfranco at="" gmail.com=""> wrote: >> >> Dear All, >> I am facing some problems with MEDIPS package, >> I Think I'am running out of memory once I run the command MEDIPS.meth >> ................ >> Adjusting p.values for multiple testing... >> Creating results table... >> Adding differential coverage results... >> Error: cannot allocate vector of size 207.9 Mb >> Inoltre: Warning messages: >> 1: In unlist(vlist, recursive = FALSE, use.names = FALSE) : >> Reached total allocation of 24501Mb: see help(memory.size) >> 2: In unlist(vlist, recursive = FALSE, use.names = FALSE) : >> Reached total allocation of 24501Mb: see help(memory.size) >> 3: In unlist(vlist, recursive = FALSE, use.names = FALSE) : >> Reached total allocation of 24501Mb: see help(memory.size) >> 4: In unlist(vlist, recursive = FALSE, use.names = FALSE) : >> Reached total allocation of 24501Mb: see help(memory.size) >> >> Is there a way to avoid this problem? >> >> I have a 24 Gb RAM workstation I don't see why it fails... >> >> Thanks, >> Paolo >> >> >> > sessionInfo() >> R version 3.0.0 (2013-04-03) >> Platform: x86_64-w64-mingw32/x64 (64-bit) >> >> locale: >> [1] LC_COLLATE=Italian_Italy.1252 LC_CTYPE=Italian_Italy.1252 >> LC_MONETARY=Italian_Italy.1252 LC_NUMERIC=C >> LC_TIME=Italian_Italy.1252 >> >> attached base packages: >> [1] parallel stats graphics grDevices utils datasets >> methods base >> >> other attached packages: >> [1] MEDIPS_1.10.0 BiocInstaller_1.10.3 gtools_3.1.0 >> DNAcopy_1.34.0 BSgenome_1.28.0 Biostrings_2.28.0 >> GenomicRanges_1.12.4 IRanges_1.18.1 BiocGenerics_0.6.0 >> >> loaded via a namespace (and not attached): >> [1] biomaRt_2.16.0 bitops_1.0-6 edgeR_3.2.4 limma_3.16.8 >> RCurl_1.95-4.1 Rsamtools_1.12.4 stats4_3.0.0 tools_3.0.0 >> XML_3.98-1.1 zlibbioc_1.6.0 > >
ADD REPLY

Login before adding your answer.

Traffic: 740 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6