R matrix size limits and array data
1
0
Entering edit mode
Rob Dunne ▴ 230
@rob-dunne-292
Last seen 10.2 years ago
Hi, I am hitting the 2^31-1 limit on array entries in loading and normalizing a set of exon arrays. The oligo package uses "ff" if you have it loaded but ff seems to have the same limit. I have modified some routines from an older version of oligo (1.10.4) to use big.matrix from the big.memory package. This necessitated making big.matrix methods for is.matrix, annotatedDataFrameFrom, and possibly some other things. It works but is probably very fragile. How are other people handling the 2^(31)-1 limit on matrix sizes? R
oligo oligo • 2.2k views
ADD COMMENT
0
Entering edit mode
@benilton-carvalho-1375
Last seen 4.7 years ago
Brazil/Campinas/UNICAMP
You can always use NetCDF and HDF5. Even with ff, you can combine different virtual data frames to work around that, as what we do in crlmm. b On Friday, 19 October 2012, Rob Dunne wrote: > Hi, > > I am hitting the 2^31-1 limit on array entries in loading and normalizing > a set of exon arrays. The oligo package uses "ff" if you have it loaded > but ff > seems to have the same limit. > > I have modified some routines from an older version of oligo (1.10.4) to > use > big.matrix from the big.memory package. This necessitated making big.matrix > methods for is.matrix, annotatedDataFrameFrom, and possibly some other > things. > It works but is probably very fragile. > > How are other people handling the 2^(31)-1 limit on matrix sizes? > > R > > _______________________________________________ > Bioconductor mailing list > Bioconductor@r-project.org <javascript:;> > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: > http://news.gmane.org/gmane.science.biology.informatics.conductor > [[alternative HTML version deleted]]
ADD COMMENT

Login before adding your answer.

Traffic: 582 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6