kyle-v6x
kyle-v6x
Ah, one more comment on (1). Increasing look_back_period_s also makes tuning the target_num_ongoing_requests very difficult. Usually you can calculate a good value based on the processing time of one request,...
@zcin Checking in again. This PR would greatly improve our production deployments.
Bumping this again. Trying to avoid building and managing our own fork, but this might be useful enough to us to do so for the time being. (cc. @zcin @edoakes)
@zcin No worries! Some good points. > Hmm, if increasing look_back_period_s will make the autoscaler less sensitive to changes in metrics, wouldn't using a second-layer average window also affect the...
> Ah, one more comment on (1). Increasing look_back_period_s also makes tuning the target_num_ongoing_requests very difficult. Usually you can calculate a good value based on the processing time of one...
Bumping this again in hopes we can get it merged this month. I believe the tests failed due to un-related reasons, so I'll push a commit to have them re-run.
Bumping again. Happy to complete ane work needed to get this merged ASAP, but I'm helpless without approval from a code owner.
@zcin Can you give an example of future hardware metric processing? Since `look_back_period` can be used to approximate the no-average case (when set close to 0), I think it would...
@zcin I have some core work that will take me to the end of the month, so I plan to work on this in early February. Thanks for the draft...
Still planning to work on this, but haven't had time.