gobetween icon indicating copy to clipboard operation
gobetween copied to clipboard

Published 20 hours ago • verified publisher icon yyyar

K8s shutdown

Open SVyatoslavG opened this issue 4 years ago • 2 comments

Added environment variable to support k8s graceful shutdown. It is not a graceful shutdown by definition. It would be useful for running gobetween behind Google Cloud Loadbalancer. Kubernetes Engine sends syscall.SIGTERM and at the same time stop sending requests to the pod before shutdown. Because those actions are done at the same time race condition occurs and the process needs to sleep before exiting. Please accept simple pull request.

SVyatoslavG avatar Aug 17 '20 17:08 SVyatoslavG

Kubernetes Engine sends syscall.SIGTERM and at the same time stop sending requests to the pod before shutdown. Because those actions are done at the same time race condition occurs and the process needs to sleep before exiting.

Could you please clarify why the process needs to sleep before exiting? So that we make sure gobetween won't shutdown while Kubernetes still sending requests to it? And also what value should GOBETWEEN_SHUTDOWN_TIME have - are there some recommendations for those who use Google Cloud Loadbalancer? Thanks!

yyyar avatar Aug 18 '20 05:08 yyyar

Sorry for the delay in answer. Yes, Kubernetes still sending requests to pods that are about to get deleting. SIGTERM & routing of requests happen at the same time and cause race conditions. I am using GOBETWEEN_SHUTDOWN_TIME 20s in my production environment. It is extremely hight for safety (theoretically GKE doesn't route any requests after 1s but for safety, I set 20s in case there are long-running requests).

SVyatoslavG avatar Sep 03 '20 21:09 SVyatoslavG