Dear all,

I am trying to run a bash file calling an R script on a cluster running CentOS 7. My R sessionInfo looks like this:

> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS: /cm/shared/apps/R/64/3.4.1/lib64/R/lib/libRblas.so
LAPACK: /cm/shared/apps/R/64/3.4.1/lib64/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] forcats_0.3.0   stringr_1.3.1   dplyr_0.7.8     purrr_0.3.0    
 [5] readr_1.3.1     tidyr_0.8.2     tibble_2.0.1    ggplot2_3.1.0  
 [9] tidyverse_1.2.1 EBImage_4.20.1 

loaded via a namespace (and not attached):
 [1] tidyselect_0.2.5    locfit_1.5-9.1      haven_2.0.0        
 [4] lattice_0.20-35     colorspace_1.4-0    generics_0.0.2     
 [7] htmltools_0.3.6     rlang_0.3.1         pillar_1.3.1       
[10] withr_2.1.2         glue_1.3.0          BiocGenerics_0.24.0
[13] modelr_0.1.2        readxl_1.2.0        bindrcpp_0.2.2     
[16] jpeg_0.1-8          bindr_0.1.1         plyr_1.8.4         
[19] munsell_0.5.0       gtable_0.2.0        cellranger_1.1.0   
[22] rvest_0.3.2         htmlwidgets_1.3     parallel_3.4.1     
[25] broom_0.5.1         Rcpp_1.0.0          scales_1.0.0       
[28] backports_1.1.3     jsonlite_1.6        abind_1.4-5        
[31] hms_0.4.2           png_0.1-7           digest_0.6.18      
[34] stringi_1.2.4       tiff_0.1-5          grid_3.4.1         
[37] cli_1.0.1           bitops_1.0-6        tools_3.4.1        
[40] magrittr_1.5        RCurl_1.95-4.10     lazyeval_0.2.1     
[43] crayon_1.3.4        pkgconfig_2.0.2     xml2_1.2.0         
[46] lubridate_1.7.4     assertthat_0.2.0    httr_1.4.0         
[49] rstudioapi_0.9.0    R6_2.3.0            fftwtools_0.9-8    
[52] nlme_3.1-131        compiler_3.4.1

My problem is that if I try to run my script on the main node (this has no purpose, it was just for testing after getting into troubles) everything runs fine. However when I am trying to run the same script on a computation node than I am facing this issue:

Error: package or namespace load failed for ‘fftwtools’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/home/gendarme/apps/R/fftwtools/libs/fftwtools.so':
libfftw3.so.3: cannot open shared object file: No such file or directory
Execution halted

This is how my bash file looks like.

module load R
module load slurm
#!/bin/sh
#SBATCH --nodes=1
#SBATCH --cpus-per-task=20
#SBATCH --job-name=sample1
#BATCH --mem=64000
export LD_LIBRARY_PATH=/cm/shared/apps/fftw:$LD_LIBRARY_PATH
export R_LIBS_USER=$HOME/apps/R:$R_LIBS_USER
Rscript ~/data/IfTestCluster/IFscript_CustomComputeFeatures_Cluster.R

I specified on purpose with LD_LIBRARY where the fftw is located on the main ode. However it doesn't seem to be enough.

I also checked the fftw installation with

yum install fftw3

and it returns that the latest version is already installed.

Has anyone an idea why this happening and how fix it?

Thanks a lot for you help in advance.

Cheers,

Mathieu

I think your problem can be reduced to properly installing the fftwtools package for R from CRAN (https://cran.r-project.org/web/packages/fftwtools/index.html) on your cluster. EBImage imports this package, but your problem is at the level of fftwtools, not of EBImage.

Installation of the system library fftw3 is probably necessary, but not sufficient. You may have to involve a unix system administrator to make sure that on your cluster, all nodes all coherently set up, and able to run fftwtools.

Suggestion: do you maybe run .bashrc or similar scripts when you interactively login to what you call "main node", but these are not run when one of your batch jobs starts on a compute node? This could lead to all sorts of weird difference between runtime environments.