Hi, Martin. SessionInfo() below: R version 2.11.1 (2010-05-31) x86_64-apple-darwin locale: [1] C attached base packages: [1] stats graphics grDevices utils datasets methods other attached packages: [1] Rsamtools_1.0.8 Biostrings_2.16.9 GenomicRanges_1.0.9 [4] IRanges_1.6.17 loaded via a namespace (and not attached): [1] Biobase_2.8.0 For what it's worth I tried just a plain read.table on the Pileup file and had essentially the same error; I don't know if that helps. I'll see about chunking the file into smaller bits in the meantime, thanks... Hollis Wright, PhD Oregon Clinical and Translational Research Institute Oregon Health and Science University On 10/1/10 1:54 PM, "Martin Morgan" <mtmorgan@fhcrc.org> wrote: On 10/01/2010 01:12 PM, Hollis Wright wrote: > Hello, all; I am having a problem with the readPileup() function in > Rsamtools. I'm trying to read in a pileup file generated by SamTools > of about ~4GB in size, on a Mac Pro, OS X 10.5. Top indicates that I > have 18 GB of memory free, and I am using a freshly built R 2.11.1 > with the x86_64 arch specified. However, when I attempt to read in > the file: > > lane1 <- readPileup("test.pup", variant="SNP") > > I eventually get a malloc error and the error "Cannot allocate vector > of size 500 Mb". I have specified ulimit unlimited for the shell that > I'm running R in and have difficulty believing that a 500MB > contiguous space is unavailable in 18 GB of free RAM. Top only ever > indicates that R is using 2-3GB; Samtools has had no problems > processing the files up to this point and a quick inspection seems to > indicate that they are proper Pileup files. Any thoughts? Hi Hollis -- Partly, the message is saying "I've allocated a bunch of memory, and now I'm trying to allocate 500 more MB, and I can't find room for that additional memory". That 2-3 GB use reported by top needs clarification; it could be a mis-representation on the part of top, but it might also be helpful to report sessionInfo() (I'm not a Mac person so can't provide detail on 32 vs. 64 bit memory use...). samtools does stream processing so doesn't run in to memory limits; this is very different from the R programming model where data generally resides in memory. The code you execute ends up more or less directly at Rsamtools:::.readPileup_SNP and Rsamtools:::.readPileup_table. These rely on read.table to input the data, and the ... arguments available in the original call are passed down to read.table. So you can select lines to skip / limit the number of records read with the arguments 'skip' and 'nrows' as documented on ?read.table (samtools does produce multi-line records, so an unfortunate choice of skip / nrows will begin / end in the middle of a record; you could use read.table alone with similar arguments to peak at the file to get the breaks right). Martin > > Hollis Wright, PhD Oregon Clinical and Translational Research > Institute Oregon Health and Science University > _______________________________________________ Bioconductor mailing > list Bioconductor@stat.math.ethz.ch > https://stat.ethz.ch/mailman/listinfo/bioconductor Search the > archives: > http://news.gmane.org/gmane.science.biology.informatics.conductor -- Computational Biology Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109 Location: M1-B861 Telephone: 206 667-2793 [[alternative HTML version deleted]]