Store stopped and exited before terminationGradePeriod is done

[rgarcia89]

Hi, I am currently working on a project where a "deny all" network policy is the default for our Kubernetes cluster. This works fine once communication between the store, query, etc. is allowed. However, I am a bit confused about the termination process of the store.

From what I understand the terminationGracePeriodSeconds is set to 120 by default. Which should be sufficient for a graceful shutdown where all persistent connections, iptables, coredns entries, etc. are removed. I am logging every detected denial, and here arises an issue due to the face that the pod terminates directly once a kubectl delete is triggered. Thus it seems to me like the application is stopped and exited before the terminationGradePeriod is done.

I have seen some arguments like grpc-grace-period and http-grace-period, but I am uncertain if they can effectively address the problem, or whether they might introduce other issues. Does someone have experience with that situation?