contour icon indicating copy to clipboard operation
contour copied to clipboard

Gradual traffic increase for cold starting endpoints

Open Zsolt-LazarZsolt opened this issue 1 year ago • 1 comments

Some of our backends require a slow start mechanism for Java based new cold pods as they have a special compile on demand approach, where the required logic is compiled on first use.

For such backends increased connect timeouts and retry policies did not help. It would be good to optionally use Envoy-s slow start mode for such nodes more info about the feature: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/slow_start.html

This could let new endpoints get a smaller percentage of the traffic at first, then over a set time increase their share to the original percentage. This could further minimise traffic loss caused by newly appearing slow starting pods.

The feature could be governed on a backend cluster basis in HTTPProxy.

Zsolt-LazarZsolt avatar Aug 25 '22 14:08 Zsolt-LazarZsolt

xref #2296

tsaarni avatar Sep 12 '22 15:09 tsaarni