Option mc.preschedule=F from mclapply has no effect on pblapply

[Alex-Cremers]

Because the input is first split into chunks of length cl, the option mc.preschedule=F has no effect (we need to wait for the chunk to finish before sending the next chunk, so workers remain idle until the longest job is done). I don't think it can easily be fixed without introducing some communication with the child processes, but it would be useful to explain this in the documentation. I'm not too familiar with the other parallel functions called by pblapply, but if they have similar options, the same would apply to them.

Minimum reprex where setting prescheduling to FALSE should cut the runtime by almost 2. It works well with mclapply but not pblapply.

library(pbapply)
library(parallel)


random_sleep <- function(x){
  duration <- sample(c(0, 2), 1)
  Sys.sleep(duration)
  return(duration)
}

microbenchmark::microbenchmark(
  mclapply(1:20, random_sleep, mc.cores = 10L, mc.preschedule = T),
  mclapply(1:20, random_sleep, mc.cores = 10L, mc.preschedule = F),
  pblapply(1:20, random_sleep, cl = 10L, mc.preschedule = T),
  pblapply(1:20, random_sleep, cl = 10L, mc.preschedule = F),
  times = 1L
)