sigterm not handled ?

[fvigotti]

Maybe It's my fault but seems that the process doesn't handle graceful process termination... SIGTERM does nothing.... only sigkill works :(

[andpol]

SIGTERM is handled https://github.com/andreas-schroeder/kafka-health-check/blob/master/main.go#L31, but in our setup with k8s, the container doesn't shutdown within a 60 second grace period. The logs indicate the check is trying to delete the broker health check topic, but it just waits there until getting a SIGKILL 60 seconds later. This is with Kafka 2.0.1

[andpol]

In my case, it turned out the reason the health check was just sitting there was because the Kafka broker running in the same k8s pod was the topic leader, and it was responding to the pod sigterm quicker than the topic could be deleted. The topic leader needs to be alive to respond to the deletion request (via ZK). I fixed my setup by adding a 15 second "preStop" sleep to the Kafka broker in the pod.

While investigating, I did however find a bug during the shutdown process, preventing proper shutdown of the health heck, should be fixed in this PR: https://github.com/andreas-schroeder/kafka-health-check/pull/32