WIP: Add scheduler overprovision for new actions before namespace throttling

[bdoyle0182]

Description

This is a wip and trying to get feedback if this is an option we could support. This would add two new configs for the scheduler:

  ```allow-over-provision-before-throttle = true
  namespace-over-provision-before-throttle-ratio = 1.5``` (would it be preferred by people that this is a fixed value rather than ratio)

If the first config is false, the ratio is not used anywhere. But the purpose of this is to prevent deadlocking on the new scheduler for actions within a namespace due to the new concept of namespace throttling. Since the scheduler can be aggressive with initially over-provisioning, you can get into a scenario where a single action uses up all of the container concurrency of the namespace before another action gets a change to run. What the code change below attempts to do is up the container concurrency limit to the over provision threshold to allow new actions to have a chance to run. This is really needed if one action depends on another. i.e. action a attempts to execute action b, but action a has all of the action concurrency for the namespace. The workload now will never make progress and action a will only ever fail all of its executions. Ultimately the right thing to do is scale out the namespace, but that's manual by a human and can't be done until impact has already occurred and this helps mitigate by slowing throughput rather than total failures.

Example of how it works:

• Namespace A has a container concurrency limit of 30
• Action A uses all 30 containers
• With an over provision before throttle ratio of 1.5, the namespace can actually have a max total of 45 containers.
• Action B comes in and since the namespace has hit its initial limit, the scheduler gives it max of 1 container until the number of containers for the namespace goes below it's normal limit of 30. Since it's not likely provisioned enough it will get action throttled which is still better than not letting any traffic at all.
• The namespace can get a total of 15 additional actions with a max of 1 container before it is now completely over-provisioned and gets namespace throttled so no additional containers can be created.

I'll admit this is a little hacky, but I do think we need to account for this throttling case until we are able to support action level container concurrency limits. So any better ideas are welcome.

Related issue and scope

• [ ] I opened an issue to propose and discuss this change (#????)

My changes affect the following components

• [ ] API
• [ ] Controller
• [ ] Message Bus (e.g., Kafka)
• [ ] Loadbalancer
• [X] Scheduler
• [ ] Invoker
• [ ] Intrinsic actions (e.g., sequences, conductors)
• [ ] Data stores (e.g., CouchDB)
• [ ] Tests
• [ ] Deployment
• [ ] CLI
• [ ] General tooling
• [ ] Documentation

Types of changes

• [ ] Bug fix (generally a non-breaking change which closes an issue).
• [X] Enhancement or new feature (adds new functionality).
• [ ] Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

• [X] I signed an Apache CLA.
• [X] I reviewed the style guides and followed the recommendations (Travis CI will check :).
• [ ] I added tests to cover my changes.
• [ ] My changes require further changes to the documentation.
• [ ] I updated the documentation where necessary.