Managed node group upgrade using AWS customer AMI

[obiordu]

Attempts to implementing a continuos process for the creation and update of eks cluster api and managed node groups using eksctl are not possible for nodegroups using customer/user AMIs.

This process uses our own AMIs that are produced from AWS's default amazon2 images that are patched with minimal addons.

The workflow for using eksctl does not allow for upgrading managed node groups (e.g 1.20) that have been created for an older controller api version (e.g 1.20) when the controller has been updated to 1.21.

Updating the ami used in the eksctl config.yml file does not upgrade the managed nodegroup even though updating the cluster version in the same config.yml file updates the controller api version.

We are therefore going to be forced to implement the creation of new managed node groups when upgrades to controller api is done , and then removal of older node groups later.

This command below shows an example upgrade path from the eksctl docs which makes use of the "--release-version" flag: eksctl upgrade nodegroup --name=managed-ng-1 --cluster=managed-cluster --release-version=1.19.6-20210310

Request and suggestion:

1. What would it take to implement our AMIs similar to eks optimized AMIs, to be able to handle upgrades using the --release-version flag ?

2. Will it be possible to tag our AMIs with a --release-version version and use same ?

3. Is there any known solution for upgrading node groups that do not use an optimized release-version ?

Please let me know if you have questions or need additional information. I am also available to chat if necessary. Thanks.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[Himangini]

First we will spike on this to understand whats required to deliver this Feature. We'll add the findings here on the ticket

Timebox: 1-2 days

[TiberiuGC]

Hi @obiordu - the official AWS docs suggest a way for upgrading nodegroups with custom AMIs. In short, you would need to initially deploy the managed nodegroups using a launch template, pointing to your custom AMI. When a new version of your custom AMI is created, you'd need to modify your launch template (i.e. create a new version of it, using the console) and make it point to the new AMI version. Afterwards you can run e.g.

eksctl upgrade nodegroup --name nodegroup-name --cluster cluster-name --launch-template-version new-template-version

I will add a note to our docs explaining this workflow.