dask-jobqueue
dask-jobqueue copied to clipboard
Unable to use `cpus-per-task`
Describe the issue:
Hello,
Currently, I am unable to launch a job due to not being able to use cpus-per-task on our SLURM cluster. Is there any way to use a different configuration, or do I need to rewrite the SLURM job launcher in order to get around this? Currently, it looks like the command is hard-coded into the SLURM job class.
Minimal Complete Verifiable Example:
cluster = SLURMCluster(
account="my-account",
queue="my-gpu-cluster",
cores=4,
memory="4GB",
processes=1,
walltime="4:00:00",
)
cluster.scale(1)
client = Client(cluster)
Anything else we need to know?:
Environment:
"dask[complete]>=2025.3.0"
"dask-jobqueue>=0.9.0"
"distributed>=2025.3.0",
Thank you,
Enrico
Hi @conceptofmind,
Slurm clusters often have some specific configuration like that.
In this case, you should be able to use job_directives_skip kwarg.
See https://jobqueue.dask.org/en/latest/clusters-advanced-tips-and-tricks.html#skipping-unrecognised-line-in-submission-script-with-job-directives-skip
Hi @conceptofmind,
Slurm clusters often have some specific configuration like that. In this case, you should be able to use
job_directives_skipkwarg.See https://jobqueue.dask.org/en/latest/clusters-advanced-tips-and-tricks.html#skipping-unrecognised-line-in-submission-script-with-job-directives-skip
Hi @guillaumeeb,
Thank you for the response!
I will test that out now.
Enrico
Hi @conceptofmind,
Slurm clusters often have some specific configuration like that. In this case, you should be able to use
job_directives_skipkwarg.See https://jobqueue.dask.org/en/latest/clusters-advanced-tips-and-tricks.html#skipping-unrecognised-line-in-submission-script-with-job-directives-skip
I was able to resolve this issue with the job directives skip.
Thank you,
Enrico