aws-eda-slurm-cluster
aws-eda-slurm-cluster copied to clipboard
[BUG] Slurm doesn't preempt a job because of licenses
Describe the bug
If a job in a high priority queue is pending because of licenses then Slurm will not preempt a lower priority job to free up a license. Since AWS allows us to have virtually unlimited compute resources, this is the main reason that jobs should pend.
I've opened an issue with SchedMD. When it gets resolved validate the new feature.
License preemption is expected to come as a new feature in Slurm 23.02: see slide 25 in https://slurm.schedmd.com/SC22/Roadmap.pdf
I don't know if I've done something wrong, but jobs pending for licenses are not preempting lower priority jobs with 23.2.0.