ExpediaGroup
Limit warmup for a faster startup
Describe the feature
We're using mittens as a side-container to warm up our main app container. Currently, there exists a single parameter to set the duration of warming up, max-duration-seconds, and that includes the time of doing readiness checks - the remaining time is the actual time to warmup. In our Kubernetes setup, the time to start our main app varies, so we have to set max-duration-seconds to a high value (just in case), because if we miss it and don't warm up the container, the initial requests present a huge latency, affecting our SLO. So we really want mittens to get a chance to do its thing. The undesired side-effect of this just-in-case increase is that our pods are very slow to start, which is not a nice thing to have in Kubernetes - as a result, we have to lower the horizontal pod autoscaler's targets to over-provision resources, so that the existing pods can support the increasing load while new (slow-starting) pods come slowly into circulation.
I understand that there was a "cleanup" of the codebase done earlier, and the current state of things is due to this change.
I'm proposing to add a new warmup-duration-seconds parameter, optional, to set the time the warmup runs for, after the readiness check succeeds, to be used to cut short the warmup when it's enough. This parameter has to be within the total
max-duration-seconds, if it is greater, then it can be discarded with a warning logged. When the warmup runs (if it runs), it'll be running for the warmup-duration-seconds or the remainder of max-duration-seconds, whichever is less.
Rational
I believe that this should be in mittens because:
- The proposal is easier to understand than what was there before (
target-readiness-timeout-seconds) - It is backwards-compatible
- The need to warmup an app for X seconds is a more common need than keep running mittens container for X seconds so many people will start using this parameter as well, knowingly
- I believe that using mittens as a side-car in Kubernetes is common, and other people may be having the same problems of slow-starting pods, and that problem would be addressed by this feature, saving costs
Example Scenario
Example 1 (backwards-compatibility)
max-duration-seconds is 120
warmup-duration-seconds is not set or set to 125
app becomes ready in 10 sec
the warmup goes on for 110 sec
total time to start - 120 sec
Example 2 (warmup time requested exceeds the total max)
max-duration-seconds is 120
warmup-duration-seconds 30
app becomes ready in 100 sec
the warmup goes on for 20 sec
total time to start - 120 sec
Example 3 (startup time saved)
max-duration-seconds is 120
warmup-duration-seconds 30
app becomes ready in 10 sec
the warmup goes on for 30 sec
total time to start - 40 sec
Thank for opening this issue @ssoukhanov
Makes sense.
I'll reach out to some folks who are using Mittens today and get their feedback on this change.
