eksctl icon indicating copy to clipboard operation
eksctl copied to clipboard

`eksctl delete cluster` should not drain nodes

Open cPu1 opened this issue 10 months ago • 4 comments

When deleting a cluster, eksctl drains all nodes before proceeding with nodegroup deletion. This behaviour is of little use and further increases the duration of the command as the workloads will remain unschedulable after pod termination.

Starting with CoreDNS v1.9.3-eksbuild.5 and v1.10.1-eksbuild.2, an addon that is installed by default, a PodDisruptionBudget with maxUnavailable: 1 is also created, preventing the nodes from being drained as attempting to evict more than one CoreDNS pod will violate the PodDisruptionBudget. While this can be worked around by passing --disable-nodegroup-eviction, eksctl delete cluster should work without needing any additional arguments in most cases.

We may optionally want to still gracefully terminate pods to give pods a chance to clean up any external resources.

Related: https://github.com/eksctl-io/eksctl/issues/6287

cPu1 avatar Apr 22 '24 16:04 cPu1

+1 we have short-lived EKS clusters used for testing k8s stuff. We create and destroy them. At least give an option to in eksctl cluster destroy to skip draining, we don't care about that, just want the cluster gone. eksctl cluster delete -n NAME --no-drain maybe?

rglonek avatar Apr 23 '24 19:04 rglonek

+1 we have short-lived EKS clusters used for testing k8s stuff. We create and destroy them. At least give an option to in eksctl cluster destroy to skip draining, we don't care about that, just want the cluster gone. eksctl cluster delete -n NAME --no-drain maybe?

@rglonek there is already an option for skipping pod eviction by passing --disable-nodegroup-eviction to eksctl delete cluster. This issue for having that as the default behaviour.

cPu1 avatar Apr 23 '24 21:04 cPu1

It skips node eviction, but still runs through the draining process, no? The logs seem to indicate so, unlike when I do eksctl delete nodegroup ... --drain=false. Would be nice if we had --drain-nodegroup=false option in eksctl delete cluster :)

rglonek avatar Apr 24 '24 02:04 rglonek

This is not a bug. Changing this default behavior is a bad idea because "incident" do happens! Explicit use of --disable-nodegroup-eviction --force is a fully conscious approach and will delete the cluster successfully despite the disruption budget.

tanvp112 avatar Apr 25 '24 04:04 tanvp112

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar May 26 '24 01:05 github-actions[bot]

This issue was closed because it has been stalled for 5 days with no activity.

github-actions[bot] avatar Jun 01 '24 01:06 github-actions[bot]