Allow Updating startupTaint without Drifting Nodes

[huiser]

Description

Observed Behavior:

Adding an extra startupTaint to provisioner (Karpenter 0.31.x) caused drift to replace all nodes.

Expected Behavior:

Adding an extra startupTaint should not be a reason to replace nodes because of drift. These taints are already removed from nodes.

Reproduction Steps (Please include YAML):

Change provisioner:

kubectl edit provisioners.karpenter.sh default

Change:

  startupTaints:
  - effect: NoExecute
    key: node.cilium.io/agent-not-ready
    value: "true"

to this:

  startupTaints:
  - effect: NoExecute
    key: node.cilium.io/agent-not-ready
    value: "true"
  - effect: NoExecute
    key: efs.csi.aws.com/agent-not-ready
    value: "true"

Observed in Karpenter logs:

{"level":"INFO","time":"2024-02-08T12:48:51.504Z","logger":"controller.deprovisioning","message":"deprovisioning via drift replace, terminating 1 machines ip-10-12-96-47.eu-central-1.compute.internal/m6g.xlarge/spot and replacing with machine from types r6gd.xlarge, is4gen.4xlarge, r6gd.2xlarge, c6gd.metal, is4gen.xlarge and 48 other(s)","commit":"709030d"}

Versions:

• Chart Version: 0.31.2
• Kubernetes Version (kubectl version): v1.28.5-eks-5e0fdde

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment