Disable messages for bplapply when in parallel
1
1
Entering edit mode
@ioannisvardaxis-11763
Last seen 18 months ago
Norway/Oslo

Hey,

I have a simple question.

I am using  the BiocParallel::bplapply in a package I am building, and the function runs in parallel. However every time the function will start running I get the following messages:

Setting options('download.file.method.GEOquery'='auto')
Setting options('GEOquery.inmemory.gpl'=FALSE)

If I use 4 workers I will get those messages 4 times. The thing is that BiocParallel::bplapply is run in a for loop and every time it is called it prints those messages which is very annoying because I print some messages in my code and those extra messages break the flow.

Is there any way to disable those mesages somehow? I have tried  suppressMessages and suppressPackageStartupMessages with no success.

if I dont run the program in parallel then no message is printed. So I guess the problem is that GEOquery package is loaded in each worker each time a worker is called and therefore the messages are produced.

 

Thanks

biocparallel bplapply • 1.9k views
ADD COMMENT
0
Entering edit mode
@martin-morgan-1513
Last seen 4 months ago
United States

I guess you're on Widows and / or using 'SnowParam()`, which starts new processes and loads packages. If you are using a for loop then one thing you might do start the cluster outside the loop, and stop it at the end

register(SnowParam())    # default workers
bpstart()                # start default workers
...                      # use workers
bpstop()                 # top workers

The message comes from GEOquery, and can indeed be supressed

> suppressPackageStartupMessages(library(GEOquery))
> 

So I guess the problem is that you are not suppressing the message at the right place in the code -- do it inside the function that you are using in bplapply().

ADD COMMENT
0
Entering edit mode

I am on MAC and I register the snow workers like the following:

snow <- BiocParallel::SnowParam(workers = 4, type = 'SOCK', progressbar=FALSE)
BiocParallel::register(snow, default=TRUE)

I tried bpstart/bpstop but the messages are still printed every time the loop is run.

I tested the following: library(GEOquery) just to see if I get any messages, but no messages are printed.

I then tested to load my package (say pkg) and I get the messages.... which is very strange.

So the problem is when my package is loaded, and it is also loaded every time the workers start..

ADD REPLY
0
Entering edit mode

remember to bpstart() once, before all your bplapply(). You'll get a message, but only once.

It sounds like your package depends or imports GEOquery, directly or indirectly, and that your test library(GEOquery) was done in a session where the package was already attached to the search path. In the worker function, use

suppressPackageStartupMessages(library(YourPackage))

Provide further details, e.g., the code for a minimal reproducible example or a link to github repository where your package is being developed, if you need further help.

ADD REPLY
0
Entering edit mode

I tried

suppressPackageStartupMessages(library(YourPackage))

inside the function I call in parallel but I still get the messages.

Link to my package:

https://bioconductor.org/packages/devel/bioc/html/MACPET.html

Now this version also runs bplapply() but not in a loop (the loop is prepared for an update of the pakage)

If you run the function MACPETUlt with the example provided at the documentation, but initiate a parallel backhead, then you will get the messages.

 

 

ADD REPLY
0
Entering edit mode

I looked into this more thoroughly, and have updated BiocParallel (version 1.13.3) to suppress package startup messages. This version is available now under R and Bioc 'devel' via biocLite("Bioconductor/BiocParallel") or after tomorrow's builds for R and bioc 'devel' via biocLite("BiocParallel").

Your own code can be made more efficient with a change such as

diff --git a/R/MACPETUlt.R b/R/MACPETUlt.R
index 00881d2..12247a5 100644
--- a/R/MACPETUlt.R
+++ b/R/MACPETUlt.R
@@ -377,6 +377,11 @@ MACPETUlt = function(SA_AnalysisDir = "", SA_stages = c(0:3), SA_prefix = "MACPE
         S1_RbowtieIndexPrefix = S1_RbowtieIndexPrefix, S1_RbowtieRefDir = S1_RbowtieRefDir,
         S2_PairedEndBAMpath = S2_PairedEndBAMpath, S2_image = S2_image, S2_BlackList = S2_BlackList,
         S3_fileSelfDir = S3_fileSelfDir, S3_image = S3_image, S3_method = S3_method)
+    if (!bpisup()) {
+        bpstart()
+        on.exit(bpstop())
+    }
+                        
     #--------------------------------------------
     #---------------Check input is correct:
     #--------------------------------------------

which starts the workers once in your overall workflow, rather than in each call to bplapply().

Questions about package development should be addressed to the Bioconductor devel mailing list.

ADD REPLY
0
Entering edit mode

Perfect!

Thanks for the help!

ADD REPLY

Login before adding your answer.

Traffic: 742 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6