pydra icon indicating copy to clipboard operation
pydra copied to clipboard

Improve SLURM support to address individual tasks

Open satra opened this issue 5 years ago • 1 comments

SLURM support for options is rather limited. so switching partitions or requesting hardware resources (e.g., GPUs) at a task specific basis is not implemented.

this would relate to #34 in terms of requirements. a task should be able to say things about its resource requirements: storage, network, gpu, cpu, memory (using the CWL specs). and then the worker should have a way of mapping those requirements to potential slurm configurations of a given cluster.

satra avatar Apr 02 '20 16:04 satra

@tclose - this nipype issue (https://github.com/nipy/nipype/issues/3726) reminded me of this. it would be nice to have some tutorials touch on this issue of different requirements for different tasks, and how that interacts with the schedulers and environments.

satra avatar May 06 '25 10:05 satra