flux-core
flux-core copied to clipboard
add queue specific resource constraint/property for partitioned resources
Problem: to implement simple "partitions", we can use the jobspec-default plugin to set queue-specific properties that match properties set in R that designate a partition. So for example, if the user requests a queue named "debug", and we could configure
[queues.debug.policy.jobspec.defaults]
system.constraints.properties = [ "debug" ]
However,
- The jobspec
propertiesconstraint operator value is defined as an array, butjobspec-defaultwould not set the array if it already exists in the jobspec. - a user could set a queue specific property and then get access to a partition wtihout picking up the queue specific limits
See also RFC 31
We could define that queue constraints are combined with any user specified constraints via the and operator. Fluxion does not currently support and, but if we only use properties in the constraint, then if we append to any existing constraints.properties array, then it would effectively be the same.
That would solve both points above, because:
- jobspec-default (or a different plugin) would be able to modify the user's constraint, so it could set new values in the array even if it already exists
- If a user sets a queue specific property then submits a job to a different queue, then the job will be marked as infeasible if the queue constraints were not overlapping. Even if they are overlapping, the user would always be constrained to resources in the queue to which their job was submitted.
Ideally, we'd be able to use the and operator so I may spend some time seeing how much work it would be to add support to Fluxion for and or and not
Closed by #4587