allow selected users to temporarily bypass limits

[garlick]

Problem: sometimes a user needs to be temporarily given the ability to bypass system limits.

[garlick]

One simple idea would be to add a way to add a temporary, named exception "capability" to the KVS under some name that the user would reference from their job submission. The capability would contain the userid, the expiration date, and a "policy" object like in RFC 33 that overrides the configuration.

jobtap plugins that implement limits could check the jobspec for the capability, then look it up by name. If found, and if unexpired, the policy object would override the configured ones.

There might need to be some housekeeping operation to remove expired capabilities from the KVS. Maybe there could be an automated way to load them from a system directory too so eventually this could be integrated into the hotline's identity management system.