vault-csi-provider
vault-csi-provider copied to clipboard
hashicorp
Vault token (and credentials) active renew of lease during lifetime of Pod to avoid TTL expiration?
The vault-agent-in-sidecar does active renewal of token (and/ credentials) lease for the lifetime of the Pod. cf https://www.vaultproject.io/docs/platform/k8s/injector#renewals-and-updating-secrets and https://www.vaultproject.io/docs/agent/template#renewals-and-updating-secrets
That's especially useful as it allows using workload without any modification: it can read the secrets as env var at startup as usual and never be bothered to dynamically rotate them.
I could not find anything like that in this secret csi provider. (the closest was https://github.com/hashicorp/vault-csi-provider/pull/64 talking about token bound to the pod lifetime, but as I understand it it's about kubernetes token, not vault token)
Am I missing something? Is it something that is planned to be added in the future?
Thanks!
Hi @thomas-riccardi, thanks for the feedback! We are indeed planning something in this area. The likely route is that we will do #82 first, and then start deploying Vault Agent as a sidecar to the vault-csi-provider daemonset and route all requests though the Agent. Then we can use the same exact mechanism as the sidecar injector.
To add, as alluded to by @thomas-riccardi , this would be really useful with regards to vault provided dynamic credentials, which can be renewable but would need to be handled by an agent. Thanks much for the work!
@tomhjp checking in on this - now that #138 closed #82 , is this work able to go forward? This feature would be really useful for our use case.
@jonpjenkins yep I've been thinking about this quite a bit lately. Adding lease renewal should now be relatively straightforward, and I think it probably won't even require any further changes in this repo - it should all be deployment changes which we'll help automate in hashicorp/vault-helm.
One slight wrinkle as far as caching is concerned though: the cache entry key depends on the Vault token, but because we generate a fresh k8s token on every mount request, the provider will never get a cache hit from Agent. That means when auto rotation is used (via the flag on the driver), a fresh secret will be fetched on every re-mount. I'm wondering about a new Agent feature to help fix this, which could perhaps rely on JWT validation because even if the token isn't stable, the identity is, so that could be a verifiably correct way to allow cache hits from different tokens. But I haven't explored that in depth yet.
For anyone following this issue, #163 and https://github.com/hashicorp/vault-helm/pull/749 are relevant. It's not quite the whole picture, but they should be good steps forward when merged.
Hello @tomhjp, hope everything is fine. I am also interested on the issue raised by @thomas-riccardi; if possibile, is there a time estimation for the realization of that feature? Thanks so much, Angelo.
@tomhjp I wanted to bump the above ask - the question of time estimation. I am on the cusp of writing a work-around for now until this is resolved.
@tomhjp Thanks for getting #163 taken care of :tada: :1st_place_medal:
Is that #749 WIP close to being complete?
Thanks for the ping, it is! I've just pushed some updates to that PR and requested review.
@tomhjp I see the referenced PR is merged, and the agent is included by default now -- thanks! :tada: :raised_hands:
Would you be wiling to offer your thoughts on what would be left yet to do on this front?
I think #202 probably wraps up this issue. Alongside the Agent sidecar deployed by vault-helm, it will mean that tokens and secret leases are always renewed by Vault Agent for as long as possible, and each pod will only have one dynamic lease generated at a time per requested secret. There are some other TTL-related issues like #149 and #151 that won't be fixed, but as this issue is primarily about TTL renewal, I think I'll close this with #202.