k8s-node-termination-handler
k8s-node-termination-handler copied to clipboard
[Question] Handles simulated termination events?
Hi there,
Does this handle simulated terminations via Stopping a preemptible instance manually? Initial test seems to suggest not, node was not being drained
If not, how can we test terminations?
Thanks, Jonathan
Seeing similar behaviour; a manual STOP seemed to trigger no action at all by the node-termination-handler (neither in the logs, or nothing cordoned).
Should we send the signal manually to test the behaviour ?
@vishh
What base image (COS vs Ubuntu) was used for the test? Was the test performed on GKE?
Yes, GKE COS.
If you follow the code from here, I'm pretty sure this does not handle manual termination.
Any pointers as to how would one add support for manual termination too?
@larsha answered this for me in the gke slack, you can "simulate" (manually invoke) this with: gcloud compute instances simulate-maintenance-event <instance>
I validated with a standalone ubuntu pod running this curl:
curl "http://metadata.google.internal/computeMetadata/v1/instance/preempted?wait_for_change=true" -H "Metadata-Flavor: Google"
And then running the simulate-maintenance-event command above. The curl returned TRUE right before the pod died.