karpenter
karpenter copied to clipboard
kubernetes-sigs
Discussion: Extending Cloud Providers
Tell us about your request
There's a growing number of requests to extend the cloud providers that Karpenter supports from AWS to other cloud providers (Azure, GCP, Orcale, etc.). This issue is intended to cover the discussion of extending the number of cloud providers broadly.
Additional Context
Related Issues
aws/karpenter#2507 aws/karpenter#936
Tracking Tasks
- [ ] #754
Attachments
No response
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
I'd like to explore two models for this: 1: Karpenter as a Library (as currently implemented here: https://github.com/bwagner5/karpenter-k3d) 2: Karpenter as an API: https://github.com/kubernetes-csi
A few table stakes in my mind:
- Vendors should be able to define their own API surface (e.g. AWSNodeTemplate) that's separately versioned
- Vendor code should live in a separate repository from Karpenter core (k8s is doing this across the board w/ in-tree / out-of-tree)
@debugger24 Are you interested in helping/building a particular provider? GCP? Oracle? Digital Ocean?
I'd like to help with the Oracle portions of this in whatever capacity I can. If I read @ellistarn correctly, I'm okay to create a separate repo like @bwagner5 did. I haven't been able to find an equivalent open-source project on the 'net, so hopefully I'm not re-inventing a wheel. If I don't see anything in this thread in a day or so, I'll follow Brandon's example and put what little I have in a repo and share it here.
Nope! @tonymarkel. Feel free to take a stab at the Oracle cloudprovider and share the repo in this issue!
The best example you'll find is to look at aws/karpenter as the aws implementation. You'll need your provider to rely on aws/karpenter-core.
seems that if we want to make a cloud provider, we cannot reuse the existing repo as it is tied to AWS implementation, we need to use karpenter-core to create our own karpenter ....
many implementation is not interfaced like
- https://github.com/aws/karpenter/tree/main/pkg/providers/instance
- https://github.com/aws/karpenter/tree/main/pkg/providers/instancetype
For an example of another provider implementation, check out: https://github.com/Azure/karpenter
For an example of another provider implementation, check out: https://github.com/Azure/karpenter
Note the source attribution section:
https://github.com/Azure/karpenter#source-attribution
Which is to say: for folks looking to rapidly prototype a new provider the AKS provider above is proof that you can just take the existing AWS bits and replace the relevant AWS-specific stuff with your own stuff.
cc @Anhui-tqhuang
There are valid points about how we can improve the portability of the various provider interfaces, but it shouldn't be a blocker per se for folks who are eager to build now.
Looking over this issue: I'm realizing that what this issue really is is a discussion issue around a bunch of things, providing a common space for any cloud providers that want to implement a Karpenter cloud provider to come to have an active discussion about how to do it.
There's no true exit criteria for this issue, so I'm changing this to be prefixed with "Discussion:"
We can get this converted to a GitHub Discussion if you like, @jonathan-innis.
We can get this converted to a GitHub Discussion
@sftim My one reservation about converting this to a discussion is routing. Do we have a good feeling which issues will get routed as discussions and which ones will just be bare-bones issues. I worry if we start having separate discussions for some one-off issues over in the Discussions tab in Github that we are going to lose people who are used to going to the issues section.
I also don't want this to be the only issue that ends up converting into a Discussion. Realistically, I see a lot of "discussion-y" issues that probably still make sense classified as issues so I wonder if we just keep this as-is for now.
Happy to hear any additional insight that you might have here.
Another Datapoint this thread might be interested in: https://github.com/kubernetes/autoscaler/issues/5394#issuecomment-1485331666
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
- After 90d of inactivity,
lifecycle/staleis applied - After 30d of inactivity since
lifecycle/stalewas applied,lifecycle/rottenis applied - After 30d of inactivity since
lifecycle/rottenwas applied, the issue is closed
You can:
- Mark this issue as fresh with
/remove-lifecycle stale - Close this issue with
/close - Offer to help out with Issue Triage
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
I would argue that the outcome of this ticket is documentation that implements a dummy provider with docker containers or something simple. 😆
There is a simple cloud provider at https://github.com/kubernetes-sigs/karpenter/tree/main/kwok that uses KWOK
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
- After 90d of inactivity,
lifecycle/staleis applied - After 30d of inactivity since
lifecycle/stalewas applied,lifecycle/rottenis applied - After 30d of inactivity since
lifecycle/rottenwas applied, the issue is closed
You can:
- Mark this issue as fresh with
/remove-lifecycle stale - Close this issue with
/close - Offer to help out with Issue Triage
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
/remove-lifecycle stale
On Thu, Aug 22, 2024, 5:56 PM Kubernetes Triage Robot < @.***> wrote:
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
- After 90d of inactivity, lifecycle/stale is applied
- After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
- After 30d of inactivity since lifecycle/rotten was applied, the issue is closed
You can:
- Mark this issue as fresh with /remove-lifecycle stale
- Close this issue with /close
- Offer to help out with Issue Triage https://www.kubernetes.dev/docs/guide/issue-triage/
Please send feedback to sig-contributor-experience at kubernetes/community https://github.com/kubernetes/community.
/lifecycle stale
— Reply to this email directly, view it on GitHub https://github.com/kubernetes-sigs/karpenter/issues/741#issuecomment-2305113515, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIUWZSHVNSEAJJO26I2ENTZSYCY7AVCNFSM6AAAAAA62KU3H2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBVGEYTGNJRGU . You are receiving this because you are subscribed to this thread.Message ID: @.***>
