k-rail
k-rail copied to clipboard
cruise-automation
[FR] add tainted annotation to Pods that have been execed into
When a Pod has been execed into, a annotation should be added to the Pod to indicate that.
Maybe k-rail.cruise-automation.github.com/taint/exec: <timestamp>
This is a good idea. Can you elaborate on the OPs process you have in mind that works with the label?
I assume #18 should cleanup the the tainted pod. I missed the issue when reading this.
For the grace periodic of the reconciliation task would need to know another timestamp to compare the elapsed time. It can either be the first or last exec into the pod for example. First probably makes more sense. This timestamp could be stored in a custom annotation so that it is easily available, when iterating though the tainted pods. 🤔 there is probably a good reason to exec into a pod. Can you imagine any reason you still want to keep the pod running to preserve state? Maybe forensic analysis? In this case another annotation may make sense to prevent the reconciliation task to evict the pod. This annotation must be set manually though.
Yes, #18 is related. I was thinking the same thing during the WIP mentioned in the other issue. There is an exec taint annotation and the value is a timestamp.
Eventually it would be pretty nice to have a DaemonSet that could automatically capture forensic information on a Pod prior to it getting evicted. Until then, I like your idea of an annotation that could prevent the post-exec eviction.
Thinking about this a bit more for the spike, I would rather use a label than an annotation. A label can be queried easily and also shows up in (prometheus) monitoring. The timestamp and "no-evict-overwrite" should be annotations though.
👋 The k-rail project has been deprecated and is no longer under active development. We recommend taking a look at OPA Gatekeeper to see if it might meet your needs going forward.
Thanks for your contribution(s) to the project!
