containers-roadmap
containers-roadmap copied to clipboard
aws
Issues with eks add-ons coredns that is overwritten
Dears, How are you? We needed to change the corefile to add some specific records in eks cluster 1.20. Any time that we changed the configuration in the core file config maps, the configuration is overwriteen by the add-ons eks.
Could you help us?
This is the same issues that it was not resolved. https://github.com/aws/containers-roadmap/issues/1159
We are working on functionality to address this. Unfortunately, I can't provide an ECD at this time.
It's always possible to run the add-on in self-managed mode by removing it using the --preserve flag as described here. This option removes Amazon EKS management of any settings and the ability for Amazon EKS to notify you of updates and automatically update the Amazon EKS add-on after you initiate an update, but preserves the add-on's software on your cluster. There is no downtime for the add-on.
Another cloud provider uses a coredns-custom configmap to modify the settings. This might be one option. (Compatibility with that would be awesome)
The other option is to allow users to configure things from the EKS UI and incorporate that into the generated configmap.
(we mainly use the hosts plugin to override DNS answers for pods in the cluster) (but things like the log plugin is useful for troubleshooting as well)
I wanted to edit the Core DNS configmap to enable logging. Whenever I am patching the configmap, it is getting overwritten by the Addon. I have multiple EKS clusters and I wanted to use the EKS addon, so that I don't have to manage Core DNS.
An early solution to this will help or I will have to switch back to using the self managed core dns.
We recently released a behavior change that will NOT overwrite configuration changes made to EKS managed add-ons through the Kubernetes API. Previously, a reconciliation process ran every 15 minutes that overwrote configuration changes made to EKS managed add-ons through the Kubernetes API. Example – changes you make to the CoreDNS Config Map through the Kubernetes API will no longer be overwritten during steady state. However, if a managed add-on is upgraded, then any configuration changes made will not be retained at this time.
This change is a first step in ensuring configurations made to EKS add-ons are preserved. We are also working on additional changes to support advanced configuration of EKS add-ons directly through the EKS API, and the ability to preserve the configuration changes during add-on upgrades.
In which version is it going to be available? When to expect this for use in production?
In which version is it going to be available? When to expect this for use in production?
The change that will NOT overwrite configuration changes made to EKS managed add-ons through the Kubernetes API is already available. Other changes to preserve configuration during add-on upgrades is still being worked upon.
this is now available with addon configuration support - you can now pass the corefile to CoreDNS
this is now available with addon configuration support - you can now pass the corefile to CoreDNS
That does mean that changes in the default config needs to be maintained manually as well though? (It still does not seem possible to add parts to the config, just to supply the entire thing?) (But this is progress)
Amazon EKS team recently announced the general availability of advanced configuration feature for managed add-ons. You can now pass in advanced configuration for cluster add-ons, enabling you to customize add-on properties not handled by default settings. Configuration can be applied to add-ons either during cluster creation or at any time after the cluster is created.
Using advanced configuration feature, you can now modify the corefile of the CoreDNS add-on.
To learn more about this feature, check out this blogpost - https://aws.amazon.com/blogs/containers/amazon-eks-add-ons-advanced-configuration/
Check out the Amazon EKS documentation - https://docs.aws.amazon.com/eks/latest/userguide/managing-add-ons.html