remoter

remoter copied to clipboard

Hi again,

each time I use a parallel implementation of lapply or mapply using the Bioconductor package BiocParallel (bplapply, bpmapply) remoter at the server side is killed, producing the message "R_zmq_msg_recv errno: 4 strerror: Interrupted system call" It seems a problem with the BiocParallel package though. Same command directly executed at the server manually does not produce any problem. Regards, Carlos

I just did a quick test with bplapply(1:10, sqrt) and things seemed ok. Do you have some example code you could share?

Hi,

bpparam <- MulticoreParam(workers = 10)
system.time ( lista_views <- bpmapply(coverageabove,genome_file,cds.df$seqnames,as.numeric(cds.df$start),as.numeric(cds.df$end),cds.df$strand, USE.NAMES = FALSE, BPPARAM=bpparam) )

Best regards,

Carlos

Interesting. So I've found a machine where this fails. However, it doesn't fail when I use a different backend, like SnowParam(). So could you try using a different backend from the client, like:

BiocParallel::bplapply(1:5, sqrt, BPPARAM=SnowParam())

I'd also be curious to know if parallel::mclapply(1:5, sqrt) crashes things, because it doesn't for me (on the machine where bplapply() things break). This sort of makes me think it's more a problem with BiocParallel than remoter, but I'm not sure.

Maybe someone from the BiocParallel project has some ideas. Pinging @mtmorgan.

HI, it works well with SnowParam(workers=5,type="SOCK") but not with type="FORK" As you said, It seems that all open sockets are closed when "FORK" is used, including those of remoter. Best regards, Carlos

A non-BiocParallel example that kills the server for me is

remoter> library(parallel)
remoter> cl = makeForkCluster(2)
remoter> stopCluster(cl)

I think it is from parallel:::mcexit(), which calls _exit(), which in turn generates a SIGCHLD signal, but I'm not really sure.

Thanks for the quick response! Unfortunately(/fortunately?) that doesn't kill the server for me; does it for you?

The only way so far I've been able to recreate the server crashing is on one machine with bplapply() with the fork backend.

Hi, this works well: parallel::mclapply(1:5, sqrt)

but last non-BiocParallel example from @mtmorgan kills the server:

[2016-09-15 08:05:00]: *** Launching UNSECURE server ***
[2016-09-15 08:05:18]: client connected
[2016-09-15 08:05:38]: RMSG: library(parallel)
[2016-09-15 08:06:04]: RMSG: cl = makeForkCluster(2)
[2016-09-15 08:06:14]: RMSG: stopCluster(cl)
R_zmq_msg_recv errno: 4 strerror: Interrupted system call
Error in unserialize(rmsg) : read error
>

Regards, Carlos

This is so strange; I can't replicate it. I really have no idea what's going on.

Just curious how this issue is going on. Since I am going to implement a centralize R server based on remoter in the lab, and a lot of scripts from bioconductor will be run on it.