parallelly
parallelly copied to clipboard
HenrikBengtsson
R package: parallelly - Enhancing the 'parallel' Package
Hi there, Great package! Really cool. I am using the plan function along with as.cluster where docker images are being pulled on the VMs. The issue is that this processes...
The remote server I'm working with (the head node of a cluster) requires me to call newgrp then load a module for R before I open R. e.g. ``` newgrp...
(Adopted from an old 2017-09-05 note of mine) When loaded, have the package set `parallel:::defaultClusterOptions$type` from (new) environment `R_PARALLELLY_CLUSTER_TYPE`, if available. This will then allow users to switch the default...
Mention security issues with PSOCK/FORK clusters, i.e. there is no authentication taking place when setting up the connection between parent and worker. There's room for man-in-the-middle attacks in shared environments.
Additional functions that might also be migrated from the **future** package: * uuid, uuid_of_connection * get_random_seed, set_random_seed, next_random_seed, make_rng_seeds, is_lecyer_cmrg_seed, is_valid_random_seed * supports_omp_threads * session_info, add_cluster_session_info, sessionDetails, print.sessionDetails * mandelbrot,...
# Wish Add `availableThreads()`, which works similarly to `availableCores()`, but instead reports on settings that control multi_threading_, in contrast to multi_processing_. It should work something like: ```r > parallelly::availableThreads(which =...
# Issue On Linux systems(*), `parallel::detectCores()` ignores the `logical` argument. For example, on my notebook with four physical CPU cores with two "processors" each, I get: ```r > parallel::detectCores(logical =...
behavior of `availableWorkers` when Slurm `--nodes`, `--ntasks` and `--cpus-per-task` are provided
I noticed that in this case, available workers is based on `-c` (`--cpus-per-task`) and not `-n` (`--ntasks`). ```bash paciorek@smeagol:~> srun --ntasks=4 --cpus-per-task=2 --pty bash paciorek@scf-sm11:~> Rscript -e "parallelly::availableWorkers()" [1] "scf-sm11"...
Setting up parallel workers: * From a Linux and macOS machine, - on remote machines running Linux - on remote machines running macOS - on remote machines running MS Windows...
Metadata
Owner
Metadata
R package: parallelly - Enhancing the 'parallel' Package