consul-k8s
consul-k8s copied to clipboard
hashicorp
Kubernetes 1.21+: Token-handling code assumes auto-created service account token secrets
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
- Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
- If you are interested in working on this issue or have submitted a pull request, please leave a comment.
Overview of the Issue
Sweeping token-scraping of auto-generated Kubernetes token secrets in preparation for Kubernetes 1.24 showed the following code locations assume auto-generated tokens will exist:
https://github.com/hashicorp/consul-k8s/blob/7ae81732981bb971243b52e4b25f08e80f5ccfd7/control-plane/connect-inject/handler.go#L279-L283
https://github.com/hashicorp/consul-k8s/blob/7ae81732981bb971243b52e4b25f08e80f5ccfd7/acceptance/framework/vault/vault_cluster.go#L173-L180
That assumption is not universally correct.
In 1.21+, secret-based tokens are no longer used for mounting into pods (ephemeral time-limited tokens are), and the token controller can be turned off.
In 1.24+, secret-based tokens are no longer auto-created by default for new service accounts.
Reproduction Steps
Create a 1.24 Kubernetes cluster or a 1.21+ Kubernetes cluster with the token controller disabled.
Additional Context
Using ephemeral time-bound tokens is preferred in 1.21+ (see the TokenRequest API) if possible.
If a secret-based token is still desired, one can be created manually, but will not be referenced from the service account's .secrets list.
Thank you @liggitt for pointing out the API changes that will be landing in 1.24 shortly. We'll review the KEP as well to see how we remediate the API compatibility issue that will surface in 1.24: https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/2799-reduction-of-secret-based-service-account-token
Out of curiosity, how did you find out that we were using auto-generated Kubernetes token secrets?
https://grep.app/search?q=.Secrets%5B0%5D.Name&filter[lang][0]=Go&filter[repo.pattern][0]=hashicorp
Its been 4months since it got reported, when are we planning to close this?
Hi @vaibhavgulati we are actively working on closing this issue out. Out of curiosity what distro and version of K8s are you using?
i am experiancing the same issue when trying to do a fresh install from helm chart with manageSystemACLs: true- deleted pvc and secrets multiple time. i am on 1.24.3 now. not shure what distro it is. but this is a scaleway managed cluster
Thanks for the heads up @dagtveit. We are targeting early next month at the latest to support K8s 1.24.
Early next what. ? 1.24 was just default on the cluster i setup so i can try lower. what is the highest supported version then ?
We will be planning on releasing support for 1.24 early next month, you should use 1.23.x and lower until then.
@david-yu we are way too behind in the release of this bug 1.21+ the way kubernetes version is progressing people will loose hope over it. Can we have a slack channel to actively track this?
We have a PR up to address this but it still needs to reviewed, merged and released: https://github.com/hashicorp/consul-k8s/pull/1431. Stay tuned.