controller-runtime icon indicating copy to clipboard operation
controller-runtime copied to clipboard

:book: Add designs/multi-cluster.md

Open sttts opened this issue 1 year ago • 17 comments

Controller-runtime today allows to write controllers against one cluster only. Multi-cluster use-cases require the creation of multiple managers and/or cluster objects. This proposal is about adding native support for multi-cluster use-cases to controller-runtime.

The proposed changes are prototyped in https://github.com/kubernetes-sigs/controller-runtime/pull/3019.

sttts avatar Mar 31 '24 13:03 sttts

For those reading, this is currently a little outdated. https://github.com/kubernetes-sigs/controller-runtime/pull/2726 has a changed design proposed by @alvaroaleman. Will come back soon to both PRs.

sttts avatar May 28 '24 13:05 sttts

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Aug 26 '24 13:08 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle rotten
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Sep 25 '24 14:09 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Reopen this PR with /reopen
  • Mark this PR as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-triage-robot avatar Oct 25 '24 15:10 k8s-triage-robot

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Reopen this PR with /reopen
  • Mark this PR as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Oct 25 '24 15:10 k8s-ci-robot

/reopen

We'd like to continue working on this, time is simply a bit scarce at the moment.

embik avatar Oct 28 '24 11:10 embik

@embik: Reopened this PR.

In response to this:

/reopen

We'd like to continue working on this, time is simply a bit scarce at the moment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Oct 28 '24 11:10 k8s-ci-robot

Hi, could you please share the future plans for this feature? Thank you! @embik

Gomaya avatar Nov 13 '24 06:11 Gomaya

@Gomaya I'm working on a prototype that attempts to address the review comments in #2726. Once everyone is back from KubeCon, I plan to run this by everyone involved and try to move the feature forward.

embik avatar Nov 13 '24 08:11 embik

/remove-lifecycle rotten

embik avatar Dec 03 '24 11:12 embik

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: sttts Once this PR has been reviewed and has the lgtm label, please assign sbueringer for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Jan 07 '25 14:01 k8s-ci-robot

Hi @alvaroaleman @sbueringer, we (@sttts and I) finally got around to refreshing this document. Could you please take a look and let us know what you think? The (new) implementation PR at #3019 is functional if you want to look at implementation details, but it's of course not finalised yet (e.g. missing tests).

embik avatar Jan 07 '25 14:01 embik

Linked the updated implementation https://github.com/kubernetes-sigs/controller-runtime/pull/3019.

sttts avatar Feb 03 '25 09:02 sttts

+1 highly appreciated in the name of @SAP & is needed in context of our open source efforts with our european partners.

maximilianbraun avatar Feb 03 '25 09:02 maximilianbraun

+1 We are very excited about this and are looking forward us it in our open source efforts with https://apeirora.eu/ @SAP

mirzakopic avatar Feb 04 '25 11:02 mirzakopic

I already discussed this with @sbueringer and others:

I went ahead earlier this week to implement the design outside of controller-runtime: https://github.com/multicluster-runtime/multicluster-runtime.

As written in the upper comment, everything but the builder is easy to implement by wrapping controller-runtime concepts. Thanks to the extensive work to get Go generics into CR, those wrappers are easy and natural.

The builder consists of roughly 500 lines of code with mostly mechanical changes, which should be feasible to regularly rebase onto the latest state in controller-runtime.

Our plan for the moment is to use https://github.com/multicluster-runtime/multicluster-runtime to prove the design with non-trivial projects. Last but not least, it will help to iterate fast and learn on the way. It is not ruled out that parts of it should flow back into controller-runtime at some point. Definitely it can and should influence the design.

sttts avatar Feb 16 '25 17:02 sttts