oligo: crash/segfault when fitting probelevel model
1
0
Entering edit mode
Guido Hooiveld ★ 4.1k
@guido-hooiveld-2020
Last seen 27 minutes ago
Wageningen University, Wageningen, the …

A fresh Linux install of R crashes/segfaults when fitting a probelevel model using oligo on a dataset. Is this a bug? If not, any idea how to fix this? I don't know whether it is important, but please note that R is configured to use multi threaded OpenBLAS (rather than the default single-threaded BLAS).

 

library(oligo)
library(pd.huex.1.0.st.v2)
library(oligoData)
data(affyExonFS)

res <- fitProbeLevelModel(affyExonFS)

Background correcting... OK
Normalizing... OK
Summarizing...
 *** caught segfault ***
address 0x7f93eacd3028, cause 'memory not mapped'

Traceback:
 1: .Call("R_sub_rcModelSummarize_plm", y, rowIndexList, PsiCode,     PsiK, input.scale, PACKAGE = "preprocessCore")
 2: theFun(y = transfo(mat), pnVec)
 3: runSummarize(tmpMat, probeInfo$man_fsetid, method = method, verbose = verbose)
 4: fitProbeLevelModel(affyExonFS)

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection:

<< I selected '2'>>

> sessionInfo()
R version 3.4.1 Patched (2017-08-27 r73146)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux

Matrix products: default
BLAS/LAPACK: /opt/OpenBLAS/lib/libopenblas_haswellp-r0.2.20.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets
[8] methods   base     

other attached packages:
 [1] oligoData_1.8.0          pd.huex.1.0.st.v2_3.14.1 DBI_0.7                 
 [4] RSQLite_2.0              oligo_1.40.2             Biostrings_2.44.2       
 [7] XVector_0.16.0           IRanges_2.10.2           S4Vectors_0.14.3        
[10] Biobase_2.36.2           oligoClasses_1.38.0      BiocGenerics_0.22.0     

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.12               compiler_3.4.1            
 [3] BiocInstaller_1.26.0       GenomeInfoDb_1.12.2       
 [5] bitops_1.0-6               iterators_1.0.8           
 [7] tools_3.4.1                zlibbioc_1.22.0           
 [9] digest_0.6.12              bit_1.1-12                
[11] memoise_1.1.0              tibble_1.3.4              
[13] preprocessCore_1.38.1      lattice_0.20-35           
[15] ff_2.2-13                  rlang_0.1.2               
[17] Matrix_1.2-11              foreach_1.4.3             
[19] DelayedArray_0.2.7         GenomeInfoDbData_0.99.0   
[21] affxparser_1.48.0          bit64_0.9-7               
[23] grid_3.4.1                 blob_1.1.0                
[25] splines_3.4.1              codetools_0.2-15          
[27] matrixStats_0.52.2         GenomicRanges_1.28.4      
[29] SummarizedExperiment_1.6.3 RCurl_1.95-4.8            
[31] affyio_1.46.0             
> 
oligo openblas preprocessCore • 2.0k views
ADD COMMENT
0
Entering edit mode
@james-w-macdonald-5106
Last seen 1 day ago
United States

Do you have a corrupted oligoData package?

> data(affyExonFS)
res <- fitProbeLevelModel(affyExonFS)
>
Loading required package: pd.huex.1.0.st.v2
Loading required package: RSQLite
Loading required package: DBI
Background correcting... OK
Normalizing... OK
Summarizing... OK
Extracting...
  Estimates... OK
  StdErrors... OK
  Weights..... OK
  Residuals... OK
  Scale....... OK
> sessionInfo()
R version 3.4.0 (2017-04-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 8 (jessie)

Matrix products: default
BLAS: /data/oldR/R-3.4.0/lib64/R/lib/libRblas.so
LAPACK: /data/oldR/R-3.4.0/lib64/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets
[8] methods   base     

other attached packages:
 [1] pd.huex.1.0.st.v2_3.14.1 DBI_0.7                  RSQLite_2.0             
 [4] oligoData_1.8.0          oligo_1.41.0             Biostrings_2.44.2       
 [7] XVector_0.16.0           IRanges_2.10.2           S4Vectors_0.14.3        
[10] Biobase_2.36.2           oligoClasses_1.38.0      BiocGenerics_0.22.0     

 

ADD COMMENT
0
Entering edit mode

Thanks James for your suggestion! But... no, reinstalling oligoData unfortunately did not solve it.

RMA normalization just works fine?!?. See below.

As said before, could it be that preprocessCore somehow is not working with a multi-threaded BLAS? However, I am not knowledgeable enough to show this is the case....

 

> data(affyExonFS)
> res <- rma(affyExonFS)
Background correcting
Normalizing
Calculating Expression
> res
ExpressionSet (storageMode: lockedEnvironment)
assayData: 22011 features, 20 samples
  element names: exprs
protocolData: none
phenoData
  rowNames: 10_5N.CEL 11_6T.CEL ... 9_5T.CEL (20 total)
  varLabels: exprs
  varMetadata: labelDescription channel
featureData: none
experimentData: use 'experimentData(object)'
Annotation: pd.huex.1.0.st.v2
>
ADD REPLY
0
Entering edit mode

Ah, I missed that part. There is C code intended to work with parallel threads, but that's all beyond my ken. It's all Ben Bolstad's code, and he is long since gone, so I am afraid you are probably on your own.

The easiest fix would be to compile a version of R that uses the stock BLAS libraries.

ADD REPLY
1
Entering edit mode

It is debatable as to how far gone over the edge I am :) But it is true that my activity level has been minimal for some years now.

In any case preprocessCore does have and has long had it's own pthreads based multithreading implementation. You can control the number of threads it is using with a shell set R_THREADS variable  (eg export R_THREADS=4) before starting R. I suspect that there is some sort of interaction between this and multi-threaded BLAS that might be solvable by putting a mutex or two in the right place. But that is just a wild guess.

 

ADD REPLY
0
Entering edit mode

Also it might be possible to turn threading off in OpenBLAS rather than just recompiling R. See

https://github.com/xianyi/OpenBLAS/wiki/faq#multi-threaded

 

 

ADD REPLY
0
Entering edit mode

Thanks Ben for replying so promptly! As per your suggestion (and for completeness I also post this here): I have opened an issue on GitHub: https://github.com/bmbolstad/preprocessCore/issues/1 to which I also added some more information on my attempts to get it working.

ADD REPLY

Login before adding your answer.

Traffic: 700 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6