spegel
spegel copied to clipboard
Remove predefined node taint after spegel pod finished initializing on the node
Describe the problem to be solved
On EKS, with Karpenter for nodes provisioning, in 99%, the workload pod starts before the spegel daemonset configures the node. This makes spegel useless in dynamic EKS + Karpenter environments. The suggested in the FAQ solution nidhogg is very unstable and not working as expected or not working at all. Additionally, with Karpenter, adding a taint that Karpenter is unaware of on the node can lead to a situation when Karpenter will create new nodes, because the pod doesn't have the required toleration.
Proposed solution to the problem
Use the "cilium" approach. With Karpenter (or some other methods) add startupTaints with some predefined name. When spegel initialized, it should remove the taint from the node.