rhdf5 segfault
3
0
Entering edit mode
@brad-friedman-6302
Last seen 10.3 years ago
Dear Bernd and List Members: I'm getting a segmentation fault reading in a character vector (attached) written by rhdf5. If anyone can make a suggestion how to get rhdf5 to read this without throwing the error I would appreciate it. The file appears to be written correctly since h5dump can read it. I got the same error with the same vector on both my mac laptop and our linux server, in both cases using the latest version of rhdf5 (2.7.4). Brad > library(rhdf5) > load("charvec.rda") ## provides "charvec" > str(charvec) chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... > hdf5.file <- tempfile() > h5createFile(hdf5.file) [1] TRUE > h5write(charvec, hdf5.file, "foo") > got <- h5read(hdf5.file, "foo") Error: segfault from C stack overflow > sessionInfo() R version 3.0.1 (2013-05-16) Platform: x86_64-apple-darwin10.8.0 (64-bit) locale: [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] rhdf5_2.7.4 loaded via a namespace (and not attached): [1] zlibbioc_1.6.0
rhdf5 rhdf5 • 2.7k views
ADD COMMENT •
1
Entering edit mode
Ben Bolstad ★ 1.2k
@ben-bolstad-1494
Last seen 7.3 years ago
You might try changing your stack size limits in your shell, before launching R. e.g. ulimit -s unlimited I know I encountered a similar issue with rhdf5 not to long ago reading from a file with a fairly large number of strings, and that was a suitable work around. Note within R Cstack_info() will also tell you stack size. At the time I looked into it, at least in my case I tracked it down to a segfault occuring in the H5Dread_helper_STRING function of file H5D.C at line 244, but it looked like the issue might be with the allocation: char bufSTR[n][size] in the previous line and I can only assume that n*size is too large for the C stack (in my instance, and perhaps yours) at the default settings in your shell. Best, Ben On Dec 23, 2013, at 4:11 PM, Brad Friedman <friedman.brad@gene.com> wrote: > Dear Bernd and List Members: > > I'm getting a segmentation fault reading in a character vector (attached) > written by rhdf5. If anyone can make a suggestion how to get rhdf5 to read > this without throwing the error I would appreciate it. The file appears to > be written correctly since h5dump can read it. > > I got the same error with the same vector on both my mac laptop and our > linux server, in both cases using the latest version of rhdf5 (2.7.4). > > Brad > >> library(rhdf5) >> load("charvec.rda") ## provides "charvec" > > >> str(charvec) > chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... >> hdf5.file <- tempfile() >> h5createFile(hdf5.file) > [1] TRUE >> h5write(charvec, hdf5.file, "foo") >> got <- h5read(hdf5.file, "foo") > Error: segfault from C stack overflow >> sessionInfo() > R version 3.0.1 (2013-05-16) > Platform: x86_64-apple-darwin10.8.0 (64-bit) > > locale: > [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] rhdf5_2.7.4 > > loaded via a namespace (and not attached): > [1] zlibbioc_1.6.0 > _______________________________________________ > Bioconductor mailing list > Bioconductor@r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor [[alternative HTML version deleted]]
ADD COMMENT •
0
Entering edit mode
On Dec 23, 2013, at 10:56 PM, Ben Bolstad <bmb at="" bmbolstad.com=""> wrote: > You might try changing your stack size limits in your shell, before launching R. e.g. > > ulimit -s unlimited I don?t think that is possible, at least as you put it. The soft limit tends to be smaller than the hard limit and is the only thing that can be increased. You may not get unlimited, but this should work and might help: limit -Ss `ulimit -Hs` > I know I encountered a similar issue with rhdf5 not to long ago reading from a file with a fairly large number of strings, and that was a suitable work around. Note within R Cstack_info() will also tell you stack size. > > At the time I looked into it, at least in my case I tracked it down to a segfault occuring in the H5Dread_helper_STRING function of file H5D.C at line 244, but it looked like the issue might be with the allocation: > > char bufSTR[n][size] > > in the previous line and I can only assume that n*size is too large for the C stack (in my instance, and perhaps yours) at the default settings in your shell. > > > Best, > > Ben > > > > On Dec 23, 2013, at 4:11 PM, Brad Friedman <friedman.brad at="" gene.com=""> wrote: > >> Dear Bernd and List Members: >> >> I'm getting a segmentation fault reading in a character vector (attached) >> written by rhdf5. If anyone can make a suggestion how to get rhdf5 to read >> this without throwing the error I would appreciate it. The file appears to >> be written correctly since h5dump can read it. >> >> I got the same error with the same vector on both my mac laptop and our >> linux server, in both cases using the latest version of rhdf5 (2.7.4). >> >> Brad >> >>> library(rhdf5) >>> load("charvec.rda") ## provides "charvec" >> >> >>> str(charvec) >> chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... >>> hdf5.file <- tempfile() >>> h5createFile(hdf5.file) >> [1] TRUE >>> h5write(charvec, hdf5.file, "foo") >>> got <- h5read(hdf5.file, "foo") >> Error: segfault from C stack overflow >>> sessionInfo() >> R version 3.0.1 (2013-05-16) >> Platform: x86_64-apple-darwin10.8.0 (64-bit) >> >> locale: >> [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 >> >> attached base packages: >> [1] stats graphics grDevices utils datasets methods base >> >> other attached packages: >> [1] rhdf5_2.7.4 >> >> loaded via a namespace (and not attached): >> [1] zlibbioc_1.6.0 >> _______________________________________________ >> Bioconductor mailing list >> Bioconductor at r-project.org >> https://stat.ethz.ch/mailman/listinfo/bioconductor >> Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor > > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioconductor mailing list > Bioconductor at r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
ADD REPLY •
0
Entering edit mode
@wolfgang-huber-3550
Last seen 3 months ago
EMBL European Molecular Biology Laborat…
Dear Brad thank you for the feedback, much appreciated. I could not run your code since the file ?charvec.rda? is presumably only on your harddrive, but a similar piece of code (below) worked fine for me. This is on Mac OS 10.9.1, I built R and rhdf5 from source using system libraries from Mac Ports. Since this seems to be a problem specific for certain installations, we?ll need a reproducible example, exact system description, and more detailed debugging information at the C level (afaIu there are people locally at your workplace who can help with that). Kind regards Wolfgang -------------------------- library("rhdf5") ## load("charvec.rda") ## provides "charvec" charvec = paste(1:16630) str(charvec) hdf5.file <- tempfile() h5createFile(hdf5.file) h5write(charvec, hdf5.file, "foo") got <- h5read(hdf5.file, "foo") stopifnot(all(got==charvec), length(got)==length(charvec)) sessionInfo() ---------------------------------------------- > sessionInfo() R Under development (unstable) (2013-12-25 r64540) Platform: x86_64-apple-darwin13.0.0/x86_64 (64-bit) locale: [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] rhdf5_2.7.4 fortunes_1.5-2 loaded via a namespace (and not attached): [1] zlibbioc_1.8.0 On 24 Dec 2013, at 01:11, Brad Friedman <friedman.brad at="" gene.com=""> wrote: > Dear Bernd and List Members: > > I'm getting a segmentation fault reading in a character vector (attached) > written by rhdf5. If anyone can make a suggestion how to get rhdf5 to read > this without throwing the error I would appreciate it. The file appears to > be written correctly since h5dump can read it. > > I got the same error with the same vector on both my mac laptop and our > linux server, in both cases using the latest version of rhdf5 (2.7.4). > > Brad > >> library(rhdf5) >> load("charvec.rda") ## provides "charvec" > > >> str(charvec) > chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... >> hdf5.file <- tempfile() >> h5createFile(hdf5.file) > [1] TRUE >> h5write(charvec, hdf5.file, "foo") >> got <- h5read(hdf5.file, "foo") > Error: segfault from C stack overflow >> sessionInfo() > R version 3.0.1 (2013-05-16) > Platform: x86_64-apple-darwin10.8.0 (64-bit) > > locale: > [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] rhdf5_2.7.4 > > loaded via a namespace (and not attached): > [1] zlibbioc_1.6.0 > _______________________________________________ > Bioconductor mailing list > Bioconductor at r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
ADD COMMENT •
0
Entering edit mode
Hi, Wolfgang Thanks for your response. I attached a file charvec.rda to the original posting....maybe I did this wrong? I'm attaching it here. Do you mind trying one more time if you can reproduce my error? Thanks, Brad On Wed, Dec 25, 2013 at 11:06 AM, Wolfgang Huber <whuber at="" embl.de=""> wrote: > Dear Brad > > thank you for the feedback, much appreciated. I could not run your code > since the file ?charvec.rda? is presumably only on your harddrive, but a > similar piece of code (below) worked fine for me. This is on Mac OS > 10.9.1, I built R and rhdf5 from source using system libraries from Mac > Ports. > > Since this seems to be a problem specific for certain installations, we?ll > need a reproducible example, exact system description, and more detailed > debugging information at the C level (afaIu there are people locally at > your workplace who can help with that). > > Kind regards > Wolfgang > > -------------------------- > library("rhdf5") > > ## load("charvec.rda") ## provides "charvec" > charvec = paste(1:16630) > > str(charvec) > > hdf5.file <- tempfile() > h5createFile(hdf5.file) > > h5write(charvec, hdf5.file, "foo") > got <- h5read(hdf5.file, "foo") > > stopifnot(all(got==charvec), length(got)==length(charvec)) > > sessionInfo() > > ---------------------------------------------- > > sessionInfo() > R Under development (unstable) (2013-12-25 r64540) > Platform: x86_64-apple-darwin13.0.0/x86_64 (64-bit) > > locale: > [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] rhdf5_2.7.4 fortunes_1.5-2 > > loaded via a namespace (and not attached): > [1] zlibbioc_1.8.0 > > > > > On 24 Dec 2013, at 01:11, Brad Friedman <friedman.brad at="" gene.com=""> wrote: > > > Dear Bernd and List Members: > > > > I'm getting a segmentation fault reading in a character vector (attached) > > written by rhdf5. If anyone can make a suggestion how to get rhdf5 to > read > > this without throwing the error I would appreciate it. The file appears > to > > be written correctly since h5dump can read it. > > > > I got the same error with the same vector on both my mac laptop and our > > linux server, in both cases using the latest version of rhdf5 (2.7.4). > > > > Brad > > > >> library(rhdf5) > >> load("charvec.rda") ## provides "charvec" > > > > > >> str(charvec) > > chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... > >> hdf5.file <- tempfile() > >> h5createFile(hdf5.file) > > [1] TRUE > >> h5write(charvec, hdf5.file, "foo") > >> got <- h5read(hdf5.file, "foo") > > Error: segfault from C stack overflow > >> sessionInfo() > > R version 3.0.1 (2013-05-16) > > Platform: x86_64-apple-darwin10.8.0 (64-bit) > > > > locale: > > [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 > > > > attached base packages: > > [1] stats graphics grDevices utils datasets methods base > > > > other attached packages: > > [1] rhdf5_2.7.4 > > > > loaded via a namespace (and not attached): > > [1] zlibbioc_1.6.0 > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor at r-project.org > > https://stat.ethz.ch/mailman/listinfo/bioconductor > > Search the archives: > http://news.gmane.org/gmane.science.biology.informatics.conductor > >
ADD REPLY •
0
Entering edit mode
Dear Brad thank you. When I use the instance of ?charvec? loaded from your charvec.rda file, then I can reproduce the error you observe: "Error: segfault from C stack overflow? and return to R prompt after running the below code once, "Segmentation fault: 11? and termination of the R process after the second time. So this might be related to the content or size of the character vector. I leave the rest to Bernd (package maintainer). This should be enough information to get going. Btw, your original post as received in my inbox did not contain an attachment, and it appears that attachments are also stripped from here: https://stat.ethz.ch/pipermail/bioconductor/2013-December/056760.html ?? library("rhdf5") if (TRUE) load("charvec.rda") else charvec = paste(1:16630) hdf5.file <- tempfile() h5createFile(hdf5.file) h5write(charvec, hdf5.file, "foo") got <- h5read(hdf5.file, "foo") stopifnot(all(got==charvec), length(got)==length(charvec)) sessionInfo() ?? Kind regards Wolfgang On 26 Dec 2013, at 17:49, Brad Friedman <friedman.brad at="" gene.com=""> wrote: > Hi, Wolfgang > > Thanks for your response. I attached a file charvec.rda to the original posting....maybe I did this wrong? I'm attaching it here. Do you mind trying one more time if you can reproduce my error? > > Thanks, > > Brad > > > On Wed, Dec 25, 2013 at 11:06 AM, Wolfgang Huber <whuber at="" embl.de=""> wrote: > Dear Brad > > thank you for the feedback, much appreciated. I could not run your code since the file ?charvec.rda? is presumably only on your harddrive, but a similar piece of code (below) worked fine for me. This is on Mac OS 10.9.1, I built R and rhdf5 from source using system libraries from Mac Ports. > > Since this seems to be a problem specific for certain installations, we?ll need a reproducible example, exact system description, and more detailed debugging information at the C level (afaIu there are people locally at your workplace who can help with that). > > Kind regards > Wolfgang > > -------------------------- > library("rhdf5") > > ## load("charvec.rda") ## provides "charvec" > charvec = paste(1:16630) > > str(charvec) > > hdf5.file <- tempfile() > h5createFile(hdf5.file) > > h5write(charvec, hdf5.file, "foo") > got <- h5read(hdf5.file, "foo") > > stopifnot(all(got==charvec), length(got)==length(charvec)) > > sessionInfo() > > ---------------------------------------------- > > sessionInfo() > R Under development (unstable) (2013-12-25 r64540) > Platform: x86_64-apple-darwin13.0.0/x86_64 (64-bit) > > locale: > [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8 > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] rhdf5_2.7.4 fortunes_1.5-2 > > loaded via a namespace (and not attached): > [1] zlibbioc_1.8.0 > > > > > On 24 Dec 2013, at 01:11, Brad Friedman <friedman.brad at="" gene.com=""> wrote: > > > Dear Bernd and List Members: > > > > I'm getting a segmentation fault reading in a character vector (attached) > > written by rhdf5. If anyone can make a suggestion how to get rhdf5 to read > > this without throwing the error I would appreciate it. The file appears to > > be written correctly since h5dump can read it. > > > > I got the same error with the same vector on both my mac laptop and our > > linux server, in both cases using the latest version of rhdf5 (2.7.4). > > > > Brad > > > >> library(rhdf5) > >> load("charvec.rda") ## provides "charvec" > > > > > >> str(charvec) > > chr [1:16630] "lincRNA:chr8:119735533-119736887 forward strand" ... > >> hdf5.file <- tempfile() > >> h5createFile(hdf5.file) > > [1] TRUE > >> h5write(charvec, hdf5.file, "foo") > >> got <- h5read(hdf5.file, "foo") > > Error: segfault from C stack overflow > >> sessionInfo() > > R version 3.0.1 (2013-05-16) > > Platform: x86_64-apple-darwin10.8.0 (64-bit) > > > > locale: > > [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 > > > > attached base packages: > > [1] stats graphics grDevices utils datasets methods base > > > > other attached packages: > > [1] rhdf5_2.7.4 > > > > loaded via a namespace (and not attached): > > [1] zlibbioc_1.6.0 > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor at r-project.org > > https://stat.ethz.ch/mailman/listinfo/bioconductor > > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor > > > <charvec.rda>
ADD REPLY •
0
Entering edit mode
imadcat • 0
@imadcat-7268
Last seen 9.9 years ago
Shanghai, China

The solution (ulimit -s unlimited) given by Ben Bolstad works on Linux machine, but not on Mac OS. ulimit -s won's accept unlimited as a valid parameter, instead you have to specify the size in number such as  ulimit -s 20000.

It appears that in Mac OS 10.10 ulimit no longer works, you should instead use launchctl limit stack to set the stack size, but the size can not exceed the system stack hard limit.

So first run a  

launchctl limit stack  

to check the 2nd number (in my case 67104768).  Then 

sudo launchctl limit stack 67104768

to set the current stack limit to the hard limit.

Restart your terminal or rstudio, now h5read should work without segment fault.
ADD COMMENT •

Login before adding your answer.

Similar Posts
Loading Similar Posts
Traffic: 538 users visited in the last hour
Content Search
Users
Tags
Badges
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6