dask-jobqueue icon indicating copy to clipboard operation
dask-jobqueue copied to clipboard

Unable to use `cpus-per-task`

Open conceptofmind opened this issue 7 months ago • 2 comments

Describe the issue:

Hello,

Currently, I am unable to launch a job due to not being able to use cpus-per-task on our SLURM cluster. Is there any way to use a different configuration, or do I need to rewrite the SLURM job launcher in order to get around this? Currently, it looks like the command is hard-coded into the SLURM job class.

Minimal Complete Verifiable Example:

cluster = SLURMCluster(
    account="my-account",
    queue="my-gpu-cluster",
    cores=4,
    memory="4GB",
    processes=1,
    walltime="4:00:00",
)
cluster.scale(1)
client = Client(cluster)

Anything else we need to know?:

Environment:

    "dask[complete]>=2025.3.0"
    "dask-jobqueue>=0.9.0"
    "distributed>=2025.3.0",

Thank you,

Enrico

conceptofmind avatar Apr 12 '25 17:04 conceptofmind

Hi @conceptofmind,

Slurm clusters often have some specific configuration like that. In this case, you should be able to use job_directives_skip kwarg.

See https://jobqueue.dask.org/en/latest/clusters-advanced-tips-and-tricks.html#skipping-unrecognised-line-in-submission-script-with-job-directives-skip

guillaumeeb avatar Apr 18 '25 09:04 guillaumeeb

Hi @conceptofmind,

Slurm clusters often have some specific configuration like that. In this case, you should be able to use job_directives_skip kwarg.

See https://jobqueue.dask.org/en/latest/clusters-advanced-tips-and-tricks.html#skipping-unrecognised-line-in-submission-script-with-job-directives-skip

Hi @guillaumeeb,

Thank you for the response!

I will test that out now.

Enrico

conceptofmind avatar Apr 21 '25 01:04 conceptofmind

Hi @conceptofmind,

Slurm clusters often have some specific configuration like that. In this case, you should be able to use job_directives_skip kwarg.

See https://jobqueue.dask.org/en/latest/clusters-advanced-tips-and-tricks.html#skipping-unrecognised-line-in-submission-script-with-job-directives-skip

I was able to resolve this issue with the job directives skip.

Thank you,

Enrico

conceptofmind avatar Aug 11 '25 17:08 conceptofmind