eksctl icon indicating copy to clipboard operation
eksctl copied to clipboard

Support IP address prefixes from vpc-cni 1.9.0

Open aidan-mundy opened this issue 4 years ago • 15 comments

What feature/behavior/change do you want? As of vpc-cni version 1.9.0, EKS now supports much larger pod capacities per node with some additional configuration. The documentation for this new functionality is here. It would be extremely helpful to have support for this functionality natively in eksctl. Some entries will be needed in the schema, a change to cluster creation will be required (and a potentially a new command for enabling this feature in existing clusters), and some additional logic will be needed to detect which node groups can support the higher pod counts as this only works on nitro instances running AL2. I would suggest that enabling the feature should also automatically determine the correct number of pods per node using the new logic present in that documentation.

Why do you want this feature? The additional pods per node afforded by this update enable more dense packing on each node. Nodes like t3.micro that could previously only support 4 pods can now support 34. (an 8 fold increase!) This is a massive improvement to the capabilities of EKS and makes the price/performance much more competitive with the other cloud providers offering managed Kubernetes. It is inconvenient (albeit not really a big deal) to take the extra steps of running the additional commands needed to enable the feature, and it would be nice to have it all wrapped into eksctl.

I am extremely busy with work at the moment, but I may take a look at implementing this if no one else is able to pick it up.

aidan-mundy avatar Aug 03 '21 19:08 aidan-mundy

Thanks for this request @aidan-mundy 👍

Definitely looks cool, the team will discuss how best to implement. Likely the first step would be a proposal, and if you do find time please feel free to get started on that 😄

Callisto13 avatar Aug 05 '21 11:08 Callisto13

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar Sep 05 '21 01:09 github-actions[bot]

Haven't had time for a proposal yet, but this definitely should not be closed.

aidan-mundy avatar Sep 05 '21 02:09 aidan-mundy

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar Oct 07 '21 01:10 github-actions[bot]

😒 Is there any way to keep the stale bot from popping up? This is a legit feature that shouldn't be hidden whether or not there is constant discussion about it.

aidan-mundy avatar Oct 07 '21 02:10 aidan-mundy

There are labels I can add to make it ignore this issue 👍

It may be good to hold off for a bit on this feature, as some of the work being done on IPv6 may make it redundant.

Edit: actually I am no longer a maintainer, so I can't add anything 😄 but if someone adds priority/important-longterm or something that will skip the bot

Callisto13 avatar Oct 12 '21 09:10 Callisto13

Added a spike to determine the work needed.

Himangini avatar Oct 20 '21 12:10 Himangini

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar Dec 16 '21 01:12 github-actions[bot]

@aidan-mundy Just to clarify. We can't do anything until a Go SDK issue is solved. This one: https://github.com/aws/aws-sdk/issues/286

Until that is done, eksctl can't pass anything to vpc-cni. :/

Skarlso avatar Dec 16 '21 07:12 Skarlso

The AWS docs you linked show that the ENV vars are set with kubectl, not an AWS API. Is it an SDK issue if the API doesn't exist?

dougbyrne avatar Dec 16 '21 13:12 dougbyrne

To clarify, we can't pass in any kind of environment property from our side. It is an SDK issue as in if the ability would ever exist, then the SDK needs to expose it and THEN we can do something from eksctl's side.

Skarlso avatar Dec 16 '21 13:12 Skarlso

Oh my Gods. Actually, I think we can do this.

Skarlso avatar Dec 16 '21 14:12 Skarlso

Okay, sorry folks, I think we might be able to pull this off. We do have some code we use to patch aws-node already. We can update that patch to add the Environment property here: https://github.com/weaveworks/eksctl/blob/main/pkg/actions/addon/create.go#L156

Skarlso avatar Dec 16 '21 14:12 Skarlso

Ahh no :( That just gets reverted anyways once the addon is installed. :((( Sorry. :((

Skarlso avatar Dec 16 '21 14:12 Skarlso

Yeah, if you do a forced install/update of the plugin, the config will be reset. If you were to apply the patch after the plugin action is done, it should stick. The plugin actions are async, so you'd need some kind of waiter. I'm doing these actions manually currently.

The other part of this is the increased pod counts. Even if the config isn't managed by eksctl, it would be nice to have the config detected and have the appropriate pod count limit set.

dougbyrne avatar Dec 16 '21 14:12 dougbyrne

@aidan-mundy Just to clarify. We can't do anything until a Go SDK issue is solved. This one: aws/aws-sdk#286

Until that is done, eksctl can't pass anything to vpc-cni. :/

AWS made a decision here not supporting this. Closing this issue since we rely upon upstream support to deliver this.

Himangini avatar Mar 29 '23 13:03 Himangini