containers-roadmap icon indicating copy to clipboard operation
containers-roadmap copied to clipboard

[EKS] Karpenter inside control plane

Open James-Quigley opened this issue 3 years ago • 14 comments

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request It would be really useful if EKS came pre-installed with Karpenter running in the control plane. Currently with Karpenter you have to first setup at least one nodegroup for Karpenter, then Karpenter can manage the rest.

If the control plane ran Karpenter for you, then there wouldn't be a need for any management of nodegroups for customers.

Which service(s) is this request for? EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? As an EKS cluster manager, when creating a cluster I have to bootstrap it with various tools as well as enough resources to run workloads. Also as a user of Karpenter, this requires setting up a nodegroup (managed), installing Karpenter and limiting it to run on that nodegroup before deploying other workloads.

This also means that as a cluster manager I need to be aware of the two different node setups: managed nodegroups and Karpenter nodes.

Are you currently working around this issue? Currently we deploy managed nodegroups, then karpenter.

Additional context Sort of similar to https://github.com/aws/containers-roadmap/issues/47

James-Quigley avatar Jul 19 '22 21:07 James-Quigley

Karpenter doesnt use ASG's and hence its availability is of paramount importance (equal to the availability requirement of ASG itself). Is that the main reason you use managed node group to run Karpenter ? You could technically run Karpenter in a node group managed by ASG as well. Would help to understand why that would not work ? Overall I support it being available as pre-installed and managed by EKS, but looking for how other teams are currently using it ? Also is Karpenter production ready ?

krmayankk avatar Oct 16 '22 19:10 krmayankk

@krmayankk Karpenter has been ready for production for almost a year at this point.

I'd love to see Karpenter running as part of the control plane, otherwise I have to managed separate node groups which don't have the best ergonomics for upgrades.

stevehipwell avatar Oct 17 '22 09:10 stevehipwell

I run karpenter using eks fargate which is a good solution until karpenter moves to the control plane itself.

runningman84 avatar Nov 03 '22 15:11 runningman84

Same here, running Karpenter in fargate. Feel it's a perfect fit for EKS control plane.

HenryYanTR avatar Nov 16 '22 15:11 HenryYanTR

I'd feel happier if I could use fargate spot instances but it's a compromise.

danielloader avatar Feb 23 '23 17:02 danielloader

@mikestef9 Any thoughts on when this is going to become available? Also happy to work on some testing when it's ready for that.

abebars avatar Apr 24 '23 04:04 abebars

@mikestef9 Any thoughts on when this is going to become available? Also happy to work on some testing when it's ready for that.

I believe it will be available after coming re:invent..!!

KuldeepSharmaTR avatar Aug 15 '23 16:08 KuldeepSharmaTR

I was also looking for this, we were setting up a ng only for karpenter, but if it can run on control plane...the problem is gone

kasvith avatar Oct 05 '23 06:10 kasvith

would love to EKS control plan enhancement on supporting dedicated hardware for karpenter control plane without us having to manage fargate or EKS node group.

maturity improvements like this would benefit the adoption for both EKS and karpenter.

duxing avatar Jan 31 '24 22:01 duxing

Hi,

Could we have more information about the implementation? How will it work? What will be the required setup compared to current EKS etc...

Thanks

deimosfr avatar Feb 13 '24 10:02 deimosfr

Hello,

Do you have any update this?.

I need this feature.

zendz avatar Mar 04 '24 07:03 zendz

If we're already managing karpenter ourselves (using Fargate/MNGs/whatever), I'm curious how cutting over to the karpenter-inside-control-plane would look/work.

kr3cj avatar Mar 07 '24 16:03 kr3cj

If we're already managing karpenter ourselves (using Fargate/MNGs/whatever), I'm curious how cutting over to the karpenter-inside-control-plane would look/work.

i would expect us to provide our own provisioners, let the managed karpenter create new nodes, and set the existing ones as unscheduled, so workloads move

FernandoMiguel avatar Mar 12 '24 12:03 FernandoMiguel

Any updates regarding this feature?

Aym3nTN avatar May 15 '24 10:05 Aym3nTN