BiocParallel: zombie processes on FORK cluster
1
1
Entering edit mode
alserg ▴ 280
@assaron
Last seen 3 months ago
St Louis, MO

Hello,

Is it a knwon issue that bplapply function leaves zombie processes on FORK cluster?

library(BiocParallel)
b <- MulticoreParam(workers=4)
bplapply(1:10, sqrt, BPPARAM=b)
# here 4 zombie processes appear
bplapply(1:10, sqrt, BPPARAM=b)
# here another 4 zombie processes appear

It could be realted to this issue: https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=15471 As far as I understand that bug was not yet fixed in R Core but it is more mild:

library(parallel)
mclapply(1:10, sqrt, mc.cores=4)
# here 3 zombie processes appear
mclapply(1:10, sqrt, mc.cores=4)
# no additional zombies

sessionInfo:

> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 9 (stretch)

Matrix products: default
BLAS: /usr/lib/openblas-base/libblas.so.3
LAPACK: /usr/lib/libopenblasp-r0.2.19.so

locale:
 [1] LC_CTYPE=ru_RU.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=ru_RU.UTF-8        LC_COLLATE=ru_RU.UTF-8    
 [5] LC_MONETARY=ru_RU.UTF-8    LC_MESSAGES=C             
 [7] LC_PAPER=ru_RU.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=ru_RU.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
[1] BiocParallel_1.12.0

loaded via a namespace (and not attached):
[1] compiler_3.4.1 tools_3.4.1    RCurl_1.95-4.8 bitops_1.0-6
biocparallel parallel • 1.8k views
ADD COMMENT
2
Entering edit mode
@martin-morgan-1513
Last seen 3 months ago
United States

I believe that this has been addressed in R-devel

------------------------------------------------------------------------
r73775 | kalibera | 2017-11-22 08:37:18 -0500 (Wed, 22 Nov 2017) | 3 lines

Do not leave zombie processes behind in parallel. Improvements to signal
handling and termination of forked processes in parallel.

and in NEWS.Rd

      \item (Package \pkg{parallel}.)  \code{mclapply()}, \code{pvec()}
      and \code{mcparallel()} (when \code{mccollect()} is used to
      collect results) no longer leave zombie processes behind.

so that zombies will be vanquished in the next release of R.

 

ADD COMMENT
0
Entering edit mode

Thanks for the info! I'll try to check R-devel version

ADD REPLY
0
Entering edit mode

Indeed, it is fixed in R-devel.

ADD REPLY

Login before adding your answer.

Traffic: 620 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6