pyDVL icon indicating copy to clipboard operation
pyDVL copied to clipboard

More fine-grained assignment of resources to tasks in MapReduceJob

Open AnesBenmerzoug opened this issue 1 year ago • 1 comments

In order to have proper interaction with within-node parallelization of utility computations and avoid cpu contention, we need to be able to configure remote tasks to run e.g. one per node, or to limit the number of cores (and ram and gpus) that they see.

AnesBenmerzoug avatar Mar 09 '23 10:03 AnesBenmerzoug

We might want to drop MapReduce altogether (think of issues with convergence criteria)

mdbenito avatar Oct 26 '23 09:10 mdbenito

Usage of MapReduce is deprecated, fine grain config is achieved in #558 via joblib.

schroedk avatar Sep 09 '24 09:09 schroedk