cluster-api-provider-nested ✨ Projected ServiceAccount Support

User Story

As a user I would like to use ServiceAccount Projection from a VirtualCluster so that I can expose my cluster as an Identity provider and issue certs to tools like vault.

Detailed Description

Upstream Kubernetes added support for ServiceAccount projection but it requires that the kubelet make requests on behalf of a workload to grant it tokens with specific audiences and expirations this is difficult to do with shared kubelets in VC. It would be nice if we could figure a way to support this.

Docs: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-token-volume-projection

Anything else you would like to add:

You can make these with a TokenRequest definition and using curl against the token subresource for ServiceAccounts like so:

curl -X "POST" {API}/api/v1/namespaces/{namespace}/serviceaccounts/{serviceaccount-name}/token -H 'Authorization: Bearer {token}' -H 'Content-Type: application/json; charset=utf-8' -d $'{}'

the Body of this usually accepts this object - https://github.com/kubernetes/api/blob/v0.20.1/authentication/v1/types.go#L131

For example:

{
  "apiVersion": "authentication.k8s.io/v1",
  "kind": "TokenRequest",
  "metadata": {
    "name": "{name}"
  },
  "spec": {
    "audiences": [
      "vault"
    ],
    "expirationSeconds": 7200,
    "boundObjectRef": {
      "kind": "Pod",
      "apiVersion": "v1",
      "name": "{name}",
      "uid": "{uid}"
    }
  }
}

Ideas

My initial thought process is adding a step in the pod syncer, where it currently checks for ServiceAccount tokens being created before creating the pod that it would call out to a "projected serviceaccount syncer" to make this request against the tenant control plane then stores the contents in a secret only in the super cluster then we mutate the pod spec to change from projected to a secret source and mount like a normal secret. We'd then need something that was checking validity of these tokens, the "projected serviceaccount syncer" for example, and when it got close to expiring we redo the call and update the token causing the secret mount to update as well.

Alternatively, we could modify the Kubelet to be "tenant aware" but I imagine this would become a massive effort.

/kind feature

Jul 07 '22 00:07 christopherhein

I agree. From modeling perspective, the syncer can be viewed as the "Kubelet" for all tenants since it honors the Pod lifecycle events for all tenants and provision the actual workload. It can "fork" the kubelet behavior to support SA token auto refresh.

During Pod creation, syncer can request a valid token, add the projection spec in a special annotation, and mutate the project volume to secret mount. Syncer can implement a patroller to scan all super pods with SA project enabled by checking the annotation and refreshing their secrets on behalf the pods. It is a nontrivial change but should work.

Jul 15 '22 19:07 Fei-Guo

Hi @christopherhein , do you have any plan to support this feature? Just want to make sure this issue is still on track. :smile:

Aug 24 '22 03:08 wondywang

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Nov 22 '22 04:11 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

Dec 22 '22 05:12 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen
Mark this issue as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Jan 21 '23 06:01 k8s-triage-robot

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen

Mark this issue as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Jan 21 '23 06:01 k8s-ci-robot

/reopen

Jan 22 '23 05:01 christopherhein

@christopherhein: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Jan 22 '23 05:01 k8s-ci-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen
Mark this issue as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Feb 21 '23 06:02 k8s-triage-robot

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen

Mark this issue as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Feb 21 '23 06:02 k8s-ci-robot

/reopen

Apr 05 '23 22:04 christopherhein

@christopherhein: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Apr 05 '23 22:04 k8s-ci-robot

Hi @christopherhein , do you have any plan to support this feature? Just want to make sure this issue is still on track. 😄

Hey @wondywang I have not been able to prioritize this feature on our end, is this something you are doing? It would be a great addition.

Apr 05 '23 22:04 christopherhein

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen
Mark this issue as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

May 05 '23 23:05 k8s-triage-robot

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen

Mark this issue as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

May 05 '23 23:05 k8s-ci-robot

/reopen

May 06 '23 04:05 christopherhein

@christopherhein: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

May 06 '23 04:05 k8s-ci-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen
Mark this issue as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Jun 05 '23 05:06 k8s-triage-robot

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen

Mark this issue as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Jun 05 '23 05:06 k8s-ci-robot

/reopen /remove-lifecycle rotten

Jun 06 '23 22:06 christopherhein

@christopherhein: Reopened this issue.

In response to this:

/reopen /remove-lifecycle rotten

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Jun 06 '23 22:06 k8s-ci-robot

Hey @wondywang I have not been able to prioritize this feature on our end, is this something you are doing? It would be a great addition.

Hi @christopherhein. Sorry for taking so long to reply to you. I have developed a feature based on Kubernetes 1.22 to be compatible with the kube-root-ca feature-gate. And kubernetes 1.22 has not completely discarded the ServiceAccount token, which will not be discarded until 1.24.

At the same time, I have developed a feature to be compatible with Kubernetes 1.24, which the ServiceAccount token no longer exists. Currently, this feature is still being verified (due to the current internal cluster, the 1.24 version cluster is lacking).

cc @Fei-Guo

Jul 25 '23 02:07 wondywang

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Jan 25 '24 04:01 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle rotten
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

Feb 24 '24 05:02 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen
Mark this issue as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Mar 25 '24 05:03 k8s-triage-robot

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen

Mark this issue as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Mar 25 '24 05:03 k8s-ci-robot

cluster-api-provider-nested cluster-api-provider-nested copied to clipboard

✨ Projected ServiceAccount Support

cluster-api-provider-nested
cluster-api-provider-nested copied to clipboard