k8s-node-termination-handler icon indicating copy to clipboard operation
k8s-node-termination-handler copied to clipboard

[Question] Handles simulated termination events?

Open ernsheong opened this issue 6 years ago • 7 comments

Hi there,

Does this handle simulated terminations via Stopping a preemptible instance manually? Initial test seems to suggest not, node was not being drained

If not, how can we test terminations?

Thanks, Jonathan

ernsheong avatar Jan 01 '19 06:01 ernsheong

Seeing similar behaviour; a manual STOP seemed to trigger no action at all by the node-termination-handler (neither in the logs, or nothing cordoned).

Should we send the signal manually to test the behaviour ?

petervandenabeele avatar Jan 07 '19 16:01 petervandenabeele

@vishh

meysammeisam avatar Mar 12 '19 17:03 meysammeisam

What base image (COS vs Ubuntu) was used for the test? Was the test performed on GKE?

vishh avatar Mar 12 '19 18:03 vishh

Yes, GKE COS.

meysammeisam avatar Mar 12 '19 18:03 meysammeisam

If you follow the code from here, I'm pretty sure this does not handle manual termination.

wbyoung avatar Apr 02 '19 18:04 wbyoung

Any pointers as to how would one add support for manual termination too?

Monnoroch avatar Aug 11 '19 16:08 Monnoroch

@larsha answered this for me in the gke slack, you can "simulate" (manually invoke) this with: gcloud compute instances simulate-maintenance-event <instance>

I validated with a standalone ubuntu pod running this curl:

curl "http://metadata.google.internal/computeMetadata/v1/instance/preempted?wait_for_change=true" -H "Metadata-Flavor: Google"

And then running the simulate-maintenance-event command above. The curl returned TRUE right before the pod died.

jbartus avatar Oct 24 '19 21:10 jbartus