I want to use "BatchtoolsParam" to parametrize schedule options on slurm backend. When using slurm backend, the template slurm-simpl.tmpl provided by batchtools is used to parametrize sbatch variables. My problem is that I cannot find a way to pass the resources variable to the template;

Here is my code:

library(BiocParallel)

piApprox <- function(n) {
    nums <- matrix(runif(2 * n), ncol = 2)
    d <- sqrt(nums[, 1]^2 + nums[, 2]^2)
    4 * mean(d <= 1)
}

param <- BatchtoolsParam(worker=4, cluster="slurm",
                         jobname="test",
                         stop.on.error=TRUE)
bpstart(param)
result <- bplapply(rep(10e5, 100), piApprox, BPPARAM=param)
bpstop(param)

I got the error message because sbatch variable cpus-per-task is invalid.

Adding 100 jobs ...
Submitting 100 jobs in 4 chunks using cluster functions 'Slurm' ...
Error in batchtools::submitJobs(ids = ids, resources = list(), reg = registry) :
  Fatal error occurred: 101. Command 'sbatch' produced exit code 1. Output: 'sbatch: error: Invalid numeric value "" for cpus-per-task.'

I use the The "slurm" template from batchtools:

a <- batchtoolsTemplate("slurm")
readLines(a)

The slurm template looks like this:

#!/bin/bash

#SBATCH --job-name=<%= job.name %>
#SBATCH --output=<%= log.file %>
#SBATCH --error=<%= log.file %>
#SBATCH --time=<%= ceiling(resources$walltime / 60) %>
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=<%= resources$ncpus %>
#SBATCH --mem-per-cpu=<%= resources$memory %>

I want to set resources$ncpus and resources$partition, but don't know how. If I just modify the template manually, that will do. But I am curious what is the best practice to modify the resources using BatchtoolsParam.

Thank you for your support.

> sessionInfo()
R version 3.5.1 (2018-07-02)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.5 LTS

Matrix products: default
BLAS/LAPACK: /app/easybuild/software/OpenBLAS/0.2.18-GCC-5.4.0-2.26-LAPACK-3.6.1/lib/libopenblas_prescottp-r0.2.18.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] BatchJobs_1.7        BBmisc_1.11          BiocParallel_1.14.2
[4] BiocInstaller_1.30.0

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.18      magrittr_1.5      rappdirs_0.3.1    hms_0.4.2
 [5] progress_1.2.0    bit_1.1-14        debugme_1.1.0     R6_2.2.2
 [9] rlang_0.2.1       brew_1.0-6        sendmailR_1.2-1   blob_1.1.1
[13] tools_3.5.1       parallel_3.5.1    checkmate_1.8.5   data.table_1.11.4
[17] DBI_1.0.0         withr_2.1.2       assertthat_0.2.0  base64url_1.4
[21] bit64_0.9-7       digest_0.6.15     crayon_1.3.4      base64enc_0.1-3
[25] fs_1.2.5          batchtools_0.9.10 memoise_1.1.0     RSQLite_2.1.1
[29] stringi_1.2.4     compiler_3.5.1    prettyunits_1.0.2 backports_1.1.2
[33] pkgconfig_2.0.1

I think the appropriate way is to use the resources argument to BatchToolsParam, which takes a named list e.g.

BatchtoolsParam(resources = list(ncpus=4))

You can find a list of the resources you can set via ?batchtools::submitJobs

This is using:

BiocParallel_1.15.12
batchtools_0.9.11