Documenting how to implement another cloud provider

[achetronic]

Is an existing page relevant? No pages for anything

What karpenter features are relevant?

How should the docs be improved? Generic documentation about interfaces and what should be implemented in a generic way would help, instead of just giving an implementation for AWS with a lot of specific cases. Done in this way it is impossible to collaborate crafting the providers for other clouds. With this, I mean, with no documentation pages and no documentation comments in the code, how is it supposed to guess what is for Karpenter and what is just for AWS?

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[ellistarn]

Hi @achetronic, this isn't something we've prioritized yet. Implementing another cloud provider is a pretty big undertaking. While Karpenter is designed to support this (see: https://github.com/aws/karpenter/blob/main/pkg/cloudprovider/types.go), I expect there will be some changes as we work towards support for another provider.

Off the top of my head:

1. changes to helm release processes and getting started guides
2. separation of the docs site into a separate repository, and a multi-vendor docs strategy
3. separation of the aws cloud provider code (/pkg/cloudprovider/aws) into a separate repository from karpenter core

It sounds like you might be interested in contributing support for another cloud provider. If you'd like to chat about this further, I recommend joining us at https://github.com/aws/karpenter/blob/main/WORKING_GROUP.md, or reaching out on slack.

[achetronic]

Hi @achetronic, this isn't something we've prioritized yet. Implementing another cloud provider is a pretty big undertaking. While Karpenter is designed to support this (see: https://github.com/aws/karpenter/blob/main/pkg/cloudprovider/types.go), I expect there will be some changes as we work towards support for another provider.

Off the top of my head:

1. changes to helm release processes and getting started guides

2. separation of the docs site into a separate repository, and a multi-vendor docs strategy

3. separation of the aws cloud provider code (/pkg/cloudprovider/aws) into a separate repository from karpenter core

It sounds like you might be interested in contributing support for another cloud provider. If you'd like to chat about this further, I recommend joining us at https://github.com/aws/karpenter/blob/main/WORKING_GROUP.md, or reaching out on slack.

Hello @ellistarn, I have seen that interfaces but there are some specific things for AWS and I should have to implement them as a null or a dirty other thing because they are inside the generic interface (about AWS Neuron and so)

I could implement the cloudprovider for Gcloud and for DO, but I would need to deattach the AWS related things to take the classes and only implement them for gcloud.

Would you consider to add a section into the API spec to reference a secret on a namespace to get, for example, the credentials for other providers?

[ellistarn]

Agreed that we need to factor out AWSNeuron and NvidiaGPU into a generic instancetype.Resources(). We've talked about it, and it should be relatively straightforward -- I think @bwagner5 might've made some progress on it.

In the short term, you can make progress on this by returning 0 for resource types not supported by your cloud provider.

For the API, each cloud provider implements the Provider runtime.RawExtensions (https://github.com/aws/karpenter/blob/3a9fa808b6eb2a42e17603135838ec5a75817399/pkg/apis/provisioning/v1alpha5/constraints.go#L42). This struct is just raw bytes that gets serialized according to the provider that the binary was built with. For example, AWS's provider API is here: https://github.com/aws/karpenter/blob/3a9fa808b6eb2a42e17603135838ec5a75817399/pkg/cloudprovider/aws/apis/v1alpha1/provider.go#L35.

[ellistarn]

I'd love to chat about all of this on slack if you are able to.

[achetronic]

I'd love to chat about all of this on slack if you are able to.

Hello there! Of course, I would like to talk about details on Slack to implement, at least, the basic support for another provider. Are you available on the Kubernetes Slack?

[ellistarn]

You can find me here: https://kubernetes.slack.com/archives/C02SFFZSA2K

[github-actions[bot]]

Labeled for closure due to inactivity in 10 days.

[till]

Hey 👋. I am interested in this as well. Did anything materialize from the conversation on Slack?

[ellistarn]

@tallaxes, would you be interested in helping out on this one?

[tallaxes]

@tallaxes, would you be interested in helping out on this one?

Yes, absolutely - assuming you are referring to ironing out (and maybe ultimately documenting) what it takes to implement a provider. I may not have a ton of bandwidth for this, but would be happy to contribute, provide feedback, point out (and maybe help pin down) expectations or requirements that are not clear, etc. (This would also be a good opportunity to revisit/sharpen some of the assumptions and decisions in Azure provider ...)

[ellistarn]

I love to review an azure provider design so we can make sure everything is lining up nicely.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[jonathan-innis]

/reopen

[k8s-ci-robot]

@jonathan-innis: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[jonathan-innis]

/remove-lifecycle rotten

[jonathan-innis]

/triage accepted