HLA icon indicating copy to clipboard operation
HLA copied to clipboard

change from detectCores() to future::availableCores() in testing.r?

Open wresch opened this issue 6 years ago • 0 comments

Hi,

in testing.r, you use detectCores() to decide how many worker processes to use for parallel execution.

https://github.com/humanlongevity/HLA/blob/34221ea2bb3e09177e237b33ba946404214cc270/bin/typing.r#L15

However, detectCores() returns all available cores on the machine, even if the process is not allowed to use them as is the case for, for example, cluster jobs that do per-core allocations (e.g. in our case a slurm cluster that uses cgroups for resource limitations).

the availableCores() function from the future package however takes resource allocations into consideration (see https://cran.r-project.org/web/packages/future/vignettes/future-1-overview.html). Would you consider switching to availableCores()? Or, alternatively, allow the size of the parallel cluster to be determined.

Wolfgang

wresch avatar Apr 04 '18 18:04 wresch