dask-jobqueue icon indicating copy to clipboard operation
dask-jobqueue copied to clipboard

Documentation about `memory` vs 'job_mem` could be improved

Open Andrew-S-Rosen opened this issue 1 year ago • 1 comments

The SLRUMCluster documentation's description for the memory keyword argument is a bit ambiguous when comparing against job_mem.

We have:

  • memory: Total amount of memory per job
  • job_mem: Amount of memory to request in SLURM. If None, defaults to worker processes * memory

The documentation states that a "job" is a Slurm allocation. So, this leads us to some potentially confusing docs.

Andrew-S-Rosen avatar Mar 06 '24 22:03 Andrew-S-Rosen

You are right, this is the memory documentation that is correct, but not precise enough, this should be something like:

  • memory: Total amount of memory to be used by all workers inside a job. Used by job queuing system by default as amount of memory per job.
  • job_mem: Amount of memory to request in SLURM for each job, If None, defaults to memory.

guillaumeeb avatar Mar 08 '24 09:03 guillaumeeb