cdk-eks-blueprints
cdk-eks-blueprints copied to clipboard
aws-quickstart
[BUG][ADDON]Removal of VPC CNI addon results in a cluster with no CNI installed
Steps to reproduce:
- Create cluster with VPC CNI addon and apply configuration
- Remove VPC CNI addon and apply configuration
Result: Cluster with no CNI installed, and pods cannot be created
Expected behaviour: Addon should be removed, but cluster should still have latest CNI DaemonSet deployed Should use the "preserve" flag for the DeleteAddon API
I can see how this behavior can affect some customers.
We are leveraging CFN support and built-in CFN mechanism for removal. Current configuration options for EKS Core add-on configuration is documented here.
The approach to delete VPC CNI is requested by customers who would like deploy an alternate CNI instead of VPC CNI. This is a common requirement for customers to deal with IP exhaustion, support for multicast, eBPF features and better DNS filtering.
So the approach when VPC CNI should be removed is valid. I will look if there is a way to make it configurable, so that customers could opt-in to retain the running version.
Can you articulate why customers may need to keep aws node ds running and opt out of the managed add-on?
If you're exploring or start using it with its default configuration (especially with the VPC CNI add-on), but then later on you realise you need to customise arguments for the add-on itself (prefix assignments, warm eni/ip numbers, external SNAT, and others). Although Amazon EKS Add-ons support server-side apply, it sometime more convenient to manage the add-on entirely from IaC/CM/GitOps technique, than to add part of it (mostly version) with EKS Add-on, and other part with other tool.
@tsahiduek We have revamped the VPC CNI Addon totally now with todays release to suppoer Advanced configurations, Custom networking, prefix assignment etc etc. Please try this out and reachout if you still see this issue.
@tsahiduek We are closing issue. Please reachback if issue still persist.