karmada
karmada copied to clipboard
karmada-io
Proposal: Cluster Authorization
What type of PR is this?
/kind design
What this PR does / why we need it:
This PR adds a simple design proposal of Cluster Authorization which can enforce that karmada-agents runs with least privilege in control plane as much as possible
Which issue(s) this PR fixes:
Ref #https://github.com/karmada-io/karmada/issues/4485#issuecomment-1905815180
Special notes for your reviewer:
This is a rough draft. I would be glad to have your active feedbacks.
Please also refer to my working POC: #4948
Does this PR introduce a user-facing change?:
[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign garrybest after the PR has been reviewed.
You can assign the PR to them by writing /assign @garrybest in a comment when ready.
The full list of commands accepted by this bot can be found here.
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Comparison is base (
fa4d6d3) 51.93% compared to head (8ac9628) 51.92%.
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@ Coverage Diff @@
## master #4593 +/- ##
==========================================
- Coverage 51.93% 51.92% -0.01%
==========================================
Files 248 248
Lines 24634 24634
==========================================
- Hits 12793 12791 -2
- Misses 11144 11145 +1
- Partials 697 698 +1
| Flag | Coverage Δ | |
|---|---|---|
| unittests | 51.92% <ø> (-0.01%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
It seems that it needs to rely on the integration and release of https://github.com/kubernetes-sigs/controller-runtime/pull/2664 before it can continue to advance the current proposal.
Yes, my POC branch relies on the unmerged/unreleased code of controller-runtime. So, if we can't wait for controller-runtime, could we use pkg/util/lifted temporary??
@everpeace thanks for your quick response!
Yes, my POC branch relies on the unmerged/unreleased code of controller-runtime. So, if we can't wait for controller-runtime, could we use
pkg/util/liftedtemporary??
Now we don't need to rush to decide how to do it specifically. We can let everyone take a look at this proposal first.
In addition, may I ask if you are currently using or planning to use Karmada?
@XiShanYongYe-Chang
Now we don't need to rush to decide how to do it specifically. We can let everyone take a look at this proposal first.
Sure.
In addition, may I ask if you are currently using or planning to use Karmada?
Yeah, it's still pre-POC phase actually, though. I'm looking into Karmada to sync/distribute our custom resources (for ML training jobs) into multiple kubernetes clusters across multiple clouds, multiple regions, and on-prems. Particularly, on-prem clusters are managed independently by multiple project teams (not platform teams). So, that's(on-prem part) the reason why I want make karmada-agent with the least privilege in control plane.
Moreover, from the view point of ML training platform, I would like to introduce "Bring your own cluster" model into our platform. Thus, I will open an issue to make karmada-agent works only for selected namespaces in control plane as my next step.
my POC branch depends on the controller-runtime PR which supports authorization webhook: https://github.com/kubernetes-sigs/controller-runtime/pull/2664
Hi @everpeace Is there any progress on this mission?
Is there any progress on this mission?
Unfortunately no... Nobody gave a review on my PR(https://github.com/kubernetes-sigs/controller-runtime/pull/2664) in a month.
I'm wondering how to promote my PR in the controller-runtime community.
should we consider using the alternative first?
Do you mean Generating fine-grained RBAC entries for karmada-agent? Controlling agent authorization via RBAC is not clean and easy to be buggy (because Role/ClusterRole are resource name dependent). So, in my personal view, I would not recommend this alternative, honestly.
So, if we can't wait for controller-runtime, could we use pkg/util/lifted temporary??
could we use lifted temporarily?? Authroization webhooks are already supported in kube-apiserver. The problem is only in library (controller-runtime).
https://github.com/kubernetes-sigs/controller-runtime/pull/2664#issuecomment-2112618194 gave dismissive feedback, unfortunately. This means, my PR would not be merged into container-runtime. So, let me to put authorization webhook implementation inside of karmada.
So, let me to put authorization webhook implementation inside of karmada.
I'm not sure if this implementation is appropriate in Karmada. Originally I wanted to use the features provided by upstream directly. Let me understand. Can we consider alternatives in parallel?
Originally I wanted to use the features provided by upstream directly.
Yeah, I wanted it, too. But upstream would reject hosting aurhotization webhook implementation code in controller-runtime repo😿.
Can we consider alternatives in parallel?
Sure, absolutely. Let me update my POC branch first then. But, I would like to insist to authroization webhook solution instead of generating RBACs because RBAC solution is not clean and easy to regress when agent code was updated.
@XiShanYongYe-Chang I updated my poc branch and opened a WIP PR just to make our discussion easier: #4948. I will close #4948 once this proposal agreed. Please take a look.
kubernetes authorization webhook handler which depends on controller-runtime is implemented in pkg/util/authorization_webhook package. see this commit: https://github.com/karmada-io/karmada/pull/4948/commits/a009de69f2181cbd8568bb053aaaa6502b1aec03.