Hey,

since we upgraded our slurm to the version 15.08.3 the Biocparallel package does not work any more.

When running the example code `xx <- bplapply(1:100, FUN)` from the vignette it fails with the following error:

Error in (function (reg, ids, resources = list(), wait, max.retries = 10L,  :
  Fatal error occured: 101. sbatch produced exit code 1; output sbatch: error: Invalid numeric value "" for number of tasks.

This is the param object

> param
class: BatchJobsParam
  bpjobname:BPJOB; bpworkers:1; bpisup:TRUE
  bpstopOnError:FALSE; bpprogressbar:TRUE
  cleanup:TRUE
> str(param)
Reference class 'BatchJobsParam' [package "BiocParallel"] with 10 fields
 $ workers      : int 1
 $ tasks        : int 0
 $ jobname      : chr "BPJOB"
 $ catch.errors : logi TRUE
 $ stop.on.error: logi FALSE
 $ progressbar  : logi TRUE
 $ reg.pars     :List of 1
  ..$ work.dir: chr "<somepath>/testslurmout"
 $ submit.pars  :List of 1
  ..$ resources:List of 1
  .. ..$ ncpus: num 1
 $ conf.pars    :List of 12
  ..$ staged.queries     : logi TRUE
  ..$ max.concurrent.jobs: num Inf
  ..$ default.resources  : list()
  ..$ cluster.functions  :List of 5
  .. ..$ name             : chr "SLURM"
  .. ..$ submitJob        :function (conf, reg, job.name, rscript, log.file, job.dir, resources, arrayjobs)  
  .. ..$ killJob          :function (conf, reg, batch.job.id)  
  .. ..$ listJobs         :function (conf, reg)  
  .. ..$ getArrayEnvirName:function ()  
  .. ..- attr(*, "class")= chr "ClusterFunctions"
  ..$ raise.warnings     : logi FALSE
  ..$ db.options         : list()
  ..$ mail.start         : chr "none"
  ..$ fs.timeout         : num NA
  ..$ db.driver          : chr "SQLite"
  ..$ mail.error         : chr "none"
  ..$ mail.done          : chr "none"
  ..$ debug              : logi FALSE
 $ cleanup      : logi TRUE
 and 18 methods, of which 4 are  possibly relevant:
   initialize, initialize#BiocParallelParam, show#BiocParallelParam,
   show#envRefClass

This is my SessionInfo output:

> sessionInfo()
R version 3.2.2 (2015-08-14)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Scientific Linux release 6.7 (Carbon)

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] checkmate_1.6.3    BatchJobs_1.6      BBmisc_1.9         BiocParallel_1.4.0

loaded via a namespace (and not attached):
 [1] digest_0.6.8         futile.options_1.0.0 DBI_0.3.1           
 [4] magrittr_1.5         RSQLite_1.0.0        stringi_1.0-1       
 [7] futile.logger_1.4.1  brew_1.0-6           lambda.r_1.1.7      
[10] tools_3.2.2          stringr_1.0.0        parallel_3.2.2      
[13] sendmailR_1.2-1      base64enc_0.1-3      fail_1.3            

My slurm.tmpl looks like the example one from the vignette.

I hope, that those information can help debugging the problem. If some more details are needed please let me know. I try to provide all information.

Best

Christian Mertes

Hi,

The number of tasks is defined by the 'resources' passed to BatchJobsParam(). You can see how this is defined in the slurm template in the BiocParallel vignette:

#SBATCH --ntasks=<%= resources$ntasks %>

In the code above it looks like you're passing an empty 'resources' list when instead it should contain the names of machines/nodes. More details about how to specify the 'resources' list is here, specifically see the 'Resource Allocation' section.

Valerie

The vignette does show how to specify it. See the code just after "The R script run interactively or from the command line ..." which shows how to create a BatchJobsParam with resources:

## register SLURM cluster instructions from the template file
funs <- makeClusterFunctionsSLURM("slurm.tmpl")
param <- BatchJobsParam(4, resources=list(ncpus=1), cluster.functions=funs)

Valerie