cluster-api-provider-azure icon indicating copy to clipboard operation
cluster-api-provider-azure copied to clipboard

Use user kubeconfig instead of admin kubeconfig

Open austintackaberry opened this issue 3 years ago • 20 comments

What type of PR is this?

/kind feature

What this PR does / why we need it: Uses user kubeconfig instead of admin kubeconfig to access the target cluster. It is needed because the admin kubeconfig contains a 2year cert that cannot be revoked.

Related to https://github.com/kubernetes-sigs/cluster-api-provider-azure/issues/2129

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): Fixes #

Special notes for your reviewer:

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

One issue with this PR is that CAPI controller logs into the API server of the cluster to get details of the nodes (reconcileNodeRefs). The change to user kubeconfig breaks this flow.

TODOs:

  • [ ] squashed commits
  • [ ] includes documentation
  • [ ] adds unit tests

Release note:


austintackaberry avatar Apr 20 '22 21:04 austintackaberry

@austintackaberry: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Apr 20 '22 21:04 k8s-ci-robot

CLA Signed

The committers listed above are authorized under a signed CLA.

  • :white_check_mark: login: austintackaberry / name: Austin Tackaberry (0599839415d66c6dc345ddc9073f7e2bd011ffc2, 3315480677d861881e8145a3c0fc6cd4734ce9f0, 43b79ec1821103d5b5cd53b6ec006e25338feb2d)

Welcome @austintackaberry!

It looks like this is your first PR to kubernetes-sigs/cluster-api-provider-azure 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/cluster-api-provider-azure has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. :smiley:

k8s-ci-robot avatar Apr 20 '22 21:04 k8s-ci-robot

Hi @austintackaberry. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Apr 20 '22 21:04 k8s-ci-robot

@austintackaberry thanks for the PR! Please sign the CLA so we can move forward

CecileRobertMichon avatar Apr 20 '22 21:04 CecileRobertMichon

/assign @alexeldeib /area managedclusters

CecileRobertMichon avatar Apr 20 '22 21:04 CecileRobertMichon

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: To complete the pull request process, please ask for approval from alexeldeib after the PR has been reviewed.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Apr 20 '22 21:04 k8s-ci-robot

@CecileRobertMichon I should now be in a group that has signed the CLA. How can I re-trigger the bot to check?

austintackaberry avatar Apr 20 '22 22:04 austintackaberry

/check-cla

CecileRobertMichon avatar Apr 20 '22 22:04 CecileRobertMichon

/easycla

CecileRobertMichon avatar Apr 20 '22 22:04 CecileRobertMichon

@austintackaberry can you please follow the instructions in https://github.com/kubernetes/community/blob/master/CLA.md? Commenting "/easycla" should retrigger the check, in theory, but not seeing it go green.

CecileRobertMichon avatar Apr 20 '22 22:04 CecileRobertMichon

also please squash commits

/ok-to-test

CecileRobertMichon avatar Apr 20 '22 22:04 CecileRobertMichon

/retest

austintackaberry avatar Apr 22 '22 20:04 austintackaberry

@alexeldeib thoughts on this one?

CecileRobertMichon avatar Apr 25 '22 14:04 CecileRobertMichon

@alexeldeib thoughts on this one?

CecileRobertMichon avatar Apr 25 '22 14:04 CecileRobertMichon

how does this work for a non-AAD enabled cluster? I admittedly am not super familiar with the func you switched to, but I thought it fetched credentials for the AAD user and requires RBAC role for access?

I see e2e passed which is good, but I am wondering what rights the e2e SP has over the subscription and if it's simply a consequence of that providing sufficient access, which I don't think we can rely on?

alexeldeib avatar May 03 '22 18:05 alexeldeib

how does this work for a non-AAD enabled cluster? Yeah, I don't think using user kubeconfig would work for non-aad clusters

I am wondering what rights the e2e SP has over the subscription and if it's simply a consequence of that providing sufficient access, which I don't think we can rely on? As long as the SP is included in the AADProfile.AdminGroupObjectIDs, then it will have admin access to all target clusters

austintackaberry avatar May 03 '22 23:05 austintackaberry

I think this change probably needs to account for differentiating AAD and non AAD clusters. Non AAD clusters will still require the admin kubeconfig.

could you also doc the requirement for the management cluster SP to use user credentials if the workload cluster is aad-enabled? That seems like a natural foot gun. @CecileRobertMichon curious on your thoughts about the UX here.

alexeldeib avatar May 05 '22 18:05 alexeldeib

@austintackaberry: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar May 24 '22 23:05 k8s-ci-robot

@austintackaberry I think we can probably introduce this functionality with the addition of DisableUserAccounts as a first-class capz + AKS configuration. I've begun work on this here:

https://github.com/kubernetes-sigs/cluster-api-provider-azure/pull/2512

jackfrancis avatar Jul 27 '22 14:07 jackfrancis

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Oct 25 '22 14:10 k8s-triage-robot

/close

Doing some PR queue cleanup 🧹, please feel free to reopen when this is ready for review

CecileRobertMichon avatar Nov 08 '22 21:11 CecileRobertMichon

@CecileRobertMichon: Closed this PR.

In response to this:

/close

Doing some PR queue cleanup 🧹, please feel free to reopen when this is ready for review

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Nov 08 '22 22:11 k8s-ci-robot