WISH: Shutting down clusters upon session exit

[HenrikBengtsson]

If the parent session of a PSOCK cluster terminates without properly shutting down the cluster first, the workers will (i) keep running, and (ii) eventually produce an error due to a broken connection:

$ R --vanilla
> cl <- parallelly::makeClusterPSOCK(1L, outfile='')
> quit('no')
starting worker pid=27586 on localhost:11470 at 20:15:04.478
Error in unserialize(node$con) : error reading from connection
Calls: workRSOCK ... doTryCatch -> recvData -> recvData.SOCKnode -> unserialize
$

Idea

We could set up an "on-session-exit" function that will be called when R terminates. Here's a proof of concept:

$ R --quiet
> cl <- parallelly::makeClusterPSOCK(1L, outfile='')
> suppressMessages(library(R.utils)); onSessionExit(function() parallel::stopCluster(cl))
> quit('no')

The onSessionExit() function of R.utils works by injecting/updating a .Last() function, which will be called when calling quit(..., runLast = TRUE) which is the default. BTW, it looks like there's a bug in R.utils causing this to only work if R.utils is attached - it's not sufficient to just loading, e.g. R.utils::onSessionExit() won't do.

Here is the same example when running in non-interactive mode;

$ Rscript -e "cl <- parallelly::makeClusterPSOCK(1L, outfile='')"
starting worker pid=29883 on localhost:11511 at 20:28:48.438
Error in unserialize(node$con) : error reading from connection
Calls: workRSOCK ... doTryCatch -> recvData -> recvData.SOCKnode -> unserialize

and

$ Rscript -e "cl <- parallelly::makeClusterPSOCK(1L, outfile=''); suppressMessages(library(R.utils)); onSessionExit(function() parallel::stopCluster(cl))"
starting worker pid=29738 on localhost:11208 at 20:28:40.744

[HenrikBengtsson]

There is a major downside with this idea; depending how busy the workers are, it might take s very long time before the main R session can terminate. It might require a user interrupt or a timeout

[renkun-ken]

Does reg.finalizer(globalenv(), f, onexit = TRUE) behave differently from R.utils::onSessionExit()?

[HenrikBengtsson]

That's a neat trick. Should probably investigate and update the quite-old R.utils::onSessionExit() to do that instead and avoid the whole .Last() hack. Thxs