AKS
AKS copied to clipboard
Azure
[BUG] AKS does not query MS Graph for group memberships when using Azure AD authentication with Kubernetes RBAC
Not sure if its a bug or wrong documentation.
Describe the bug When updating one of our clusters to use Azure AD authentication with Kubernetes RBAC, the observed behaviour is that the JWT is verified, but no call is done to the MS Graph for group memberships.
According to the following docs; https://learn.microsoft.com/en-us/azure/aks/concepts-identity#azure-ad-integration The JWT should be verified, and then Webhook Token Authentication should be performed to assert current group memberships.
Our specific scenario is using PIM for Groups to elevate access temporarily.
This worked when we were using Azure AD authentication with Azure RBAC, albeit with up to 5 minute delay due to cache TTL and PIM for Azure Role. After the switch however, we have to acquire a new access token with an updated groups claim for the change to be reflected when we call the AKS cluster.
This also means that any group membership is retained for the duration of the token, so a revoked group membership is delayed by 90 minutes in the worst case.
Checking the kubelet config on the nodes also indicates that this should work.
The flags --authentication-token-webhook=true --authorization-mode=Webhook --kubeconfig=/var/lib/kubelet/kubeconfig are set in /etc/default/kubelet, and the kubeconfig follows the spec as documented here:
https://kubernetes.io/docs/reference/access-authn-authz/webhook/
To Reproduce
- Get credentials for AKS:
az aks get-credentials -g <resourcegroup> -n <clustername> - Run
kubectl get podsto verify access - Run
kubectl get secret supersecretto verify that access is denied - Add user to a group granting access to
supersecretvia (Cluster)RoleBinding - After 5 minutes, run
kubectl get secret supersecret. Access is still not granted - Run
kubelogin remove-tokens - Run
kubectl get secret supersecretagain, perform any authentication necessary, and observe that access is granted
Expected behavior Access should be granted already in step 5, just like when using Azure AD authentication with Azure RBAC.
Environment (please complete the following information):
- az CLI Version: 2.49.0
- Kubernetes version: v1.25.6
- kubectl version: v1.27.2
![microsoft-github-policy-service[bot] avatar](https://avatars.githubusercontent.com/in/95686?v=4 "Published by a pub.dev verified publisher"){kind=small}
@oddsund Thanks for reporting this.
AAD supports 200 groups claim in JWT, refer to doc.
If current user has <= 200 groups, the client token will already contain groups claim. Server side WILL NOT perform additional call to fetch the groups. This helps reducing unnecessary external calls. If current user has > 200 groups, the client token doesn't contain group claim but a claim source URL. Server side will perform additional call to this URL to fetch groups.
For your case, it matches the behavior when user has <= 200 groups, such that server side will use the built-in group token. It is expected behavior to update kubeconfig credential to get the JIT applied.
Thanks for the response @karataliu.
That's fine, but then this doc should be updated to clarify that; https://learn.microsoft.com/en-us/azure/aks/concepts-identity#azure-ad-integration
The docs currently states The server application uses user-provided credentials to query group memberships of the logged-in user from the MS Graph API., which is only true for user with > 200 groups.
This issue has been automatically marked as stale because it has not had any activity for 30 days. It will be closed if no further activity occurs within 7 days of this comment. @karataliu
This issue will now be closed because it hasn't had any activity for 7 days after stale. oddsund feel free to comment again on the next 7 days to reopen or open a new issue after that time if you still have a question/issue or suggestion.