oauth2-proxy
Azure provider does not work when "resource" is used - fails to retrieve email
on AKS 1.20.2, I'm using oauth2-proxy as a ext-authz extension for istio. I want to protect Kiali. Kiali has multiple authorization strategies, one of them being the header strategy, which would work perfectly with oauth2-proxy. My goal is to forward the kubernetes user token, retrieved using oauth2-proxy from Azure Kubernetes AAD, to kiali, which would then use it to query the kubernetes resources.
I basically want to get an access token to kubernetes through oauth2-proxy, through Azure Kubernetes AD, forward it to Kiali, then kiali would use that to query the Kubernetes API on behalf of the user.
On AKS, there's an integration with Active Directory for RBAC permissions, which is enabled in our clusters. The Kubernetes API is configured with an OpenID provider, being the public Azure Kubernetes AD. So users have to do az aks get-credentials, which guides them through an oidc flow, then configures their kube.config.
I registered an Azure App, which has the Azure Graph user.read permission, as well as the user.read permission on the Azure Kubernetes AD service. The id of this public service is 6dae42f8-4368-4678-94ff-3960e28e3630
I can successfully login trough the istio ext-authz set up on our ingress gateway, then I'm redirected to kiali. But this does not forward the actual kubernetes token to kiali yet.
If I set the oauth2-proxy resource parameter to 6dae42f8-4368-4678-94ff-3960e28e3630 (the Azure Kubernetes AD), then there's an error. The error is that oauth2-proxy cannot retrieve the user email from the graph api (InvalidAudience)
Since the id_token and access_token returned by azure do not include the email, oauth2-proxy tries to retrieve the email from the graph api. This works when the resource parameter is not set, but if fails with an InvalidAudience error if the resource parameter is set.
I tried setting the oauth2-proxy scope arguments to openid email profile or openid,email,profile, but it seems that it does not include the email in the response.
Back to the root of the error, I guess that the reason behind this is that the access token used to query the graph api is the token for the "protected resource" (in this case Azure Kubernetes AD) which does not have permission to query the Graph API.
Expected Behavior
- Should not fail to retrieve the email when using a protected resource
Current Behavior
- The query fails to retrieve the email from the id_token, access_token and graph api
Possible Solution
- Perhaps the Graph API request could be made with a token that was retrieved "not" on the protected resource
- Find another way of including the email in the id_token or access_token from Azure
Your Environment
AKS 1.20.2
- Version used: latest
oauth2-proxy [2021/03/31 10:47:07] [azure.go:165] unable to get email claim from id_token: <nil>
oauth2-proxy [2021/03/31 10:47:07] [azure.go:173] unable to get email claim from access token: <nil>
oauth2-proxy [2021/03/31 10:47:07] [oauthproxy.go:809] Error creating session during OAuth2 callback: unable to get email address: unexpected status "401": {
oauth2-proxy "error": {
oauth2-proxy "code": "InvalidAuthenticationToken",
oauth2-proxy "message": "Access token validation failure. Invalid audience.",
oauth2-proxy "innerError": {
oauth2-proxy "date": "2021-03-31T10:47:07",
oauth2-proxy "request-id": "XXXXX",
oauth2-proxy "client-request-id": "XXXXXX"
oauth2-proxy }
oauth2-proxy }
oauth2-proxy }
token received with the "resource" param:
{
"aud": "UUID",
"iss": "https://sts.windows.net/TENANT_ID/",
"iat": 2983129083,
"nbf": 2983129083,
"exp": 2983129083,
"aio": "REDACTED",
"appid": "REDACTED",
"appidacr": "1",
"idp": "https://sts.windows.net/TENANT_ID/",
"oid": "OBJECT_ID",
"rh": "REDACTED",
"sub": "REDACTED",
"tid": "REDACTED",
"uti": "REDACTED",
"ver": "1.0"
}
Not sure if it helps for your use case, but in mine, with the app registration I was using for testing at least, I was able to use Azure AAD OAuth with both v1 and v2 tokens using absolutely no special logic whatsoever from the azure provider - instead I used the generic OIDC provider. Note that my tokens had a lot of default scopes enabled (all of them, for testing purposes). I haven't yet tried using generic Azure app registration tokens with limited scopes. Also note that in the azure app registration Manifest I set the accessTokenAcceptedVersion to "2" for it to actually issue v2 tokens from the v2 endpoint, otherwise it only issues v1 tokens even from the v2 endpoint.
For v1 tokens I used the following (in addition to cookie secrets and other config):
--skip-provider-button
--provider=oidc
--client-id={client-id}
--client-secret={client-secret}
--oidc-issuer-url=https://sts.windows.net/{azure-tenant}/
For v2 tokens I used the following (in addition to cookie secrets and other config):
--skip-provider-button
--provider=oidc
--client-id={client-id}
--client-secret={client-secret}
--oidc-issuer-url=https://login.microsoftonline.com/{azure-tenant}/v2.0
It's not yet clear to me what advantages, if any, there are in using the azure provider directly, especially since it apparently does not validate issuer. #1135 This is fine but not great, as if you're running a multi-tenant install, a better check would be to validate the issuer format matches what is expected, or maybe there's a generic issuer (perhaps v2?) that works in all cases. I don't have a multi-tenant use case currently, and I also don't know if I've set up my token with special scopes or not, though I did basically request all scopes in my token to test out how much data it can return for development purposes.
The advantage of using v1 tokens is that if you use --pass-authorization-header=true (and maybe --pass-basic-auth=false) you can get access to fields that aren't available in v2, such as the amr claim type field which can tell you what form of authentication was used (such as pwd,mfa - rsa is also an option) Upstream you can pull the token from the Authorization header and validate the audience matches your client ID, the issuer matches the issuer noted above, and the JWKS to validate with can be from https://login.microsoftonline.com/{azure-tenant}/discovery/keys for v1 or https://login.microsoftonline.com/{azure-tenant}/discovery/v2.0/keys for v2. (Note that Microsoft doesn't include the "alg" key so PyJWT 2.0.1 has problems, but PyJWT from git upstream, or potentially a newer release, should work fine.)
Hello,
@ludydoo, I'm currently running into similar issues as I'm trying to set up oauth2-proxy to grab an access token to use with the Kubernetes dashboard. @weinong wrote a guide related to this, and implemented the PR (thanks to him, by the way). However, I can't get it to work either.
I'm not familiar with go but by inspecting the code, recompiling and deploying locally, I actually discovered that your current issue is not that the returned id_token does not have an e-mail claim (you can actually configure that in the AAD App registration, in the "Token configuration" section). Based on your logs, you face the same issue I initially had, which is that, in order to extract this claim, the tokens need to be verified, which isn't done unless you configure the issuer with --oidc-issuer-url=https://sts.windows.net/<your tenant id>/. (Trailing slash needed - see #1135 as mentioned by @LouisStAmour).
Once you do that, the tokens will be verified against the AAD OIDC endpoints, and the claims extracted, but now I'm facing the following:
- The logic is that the id_token is verified first and checked for an email claim. However, for some reason, the id_token delivered by AAD - in my case, at least - is not signed (the
algheader claim is set tonone). This causes the validation to fail because of an unsupported algorithm :unable to verify token: oidc: id token signed with unsupported algorithm, expected ["RS256"] got "none". @weinong was aware of this, as documented in the code. - Next, we fall back to the access_token which is also verified. However, since we use the
--resourceflag, theaudclaim of this token is not oauth2-proxy's id but rather the requested app id, in my case : the AKS Service (6dae42f8-4368-4678-94ff-3960e28e3630). oauth2-proxy sends the validation with its own client_id, though, so, AAD fails to validate it withunable to verify token: oidc: expected audience "<my oauth2-proxy app id>" got ["6dae42f8-4368-4678-94ff-3960e28e3630"] - As a last resort, a request is made to the Graph API with the access token, but, once again, it is rejected because the audience is not the one expected by the API :
oauth2-proxy "error": {
oauth2-proxy "code": "InvalidAuthenticationToken",
oauth2-proxy "message": "Access token validation failure. Invalid audience.",
oauth2-proxy "innerError": {
oauth2-proxy "date": "XXXX",
oauth2-proxy "request-id": "XXXXX",
oauth2-proxy "client-request-id": "XXXXXX"
oauth2-proxy }
oauth2-proxy }
So, despite this interesting fallback mechanism, all 3 methods fail - in my case, at least.
Would anyone have some advice in order to get this to work ?
My args:
- --provider=azure
- --email-domain=* (will be enforced once the token validation works)
- --http-address=0.0.0.0:4180
- --azure-tenant=REDACTED
- --client-id=<my app id - see weinong's guide>
- --client-secret=REDACTED
- --oidc-issuer-url=https://sts.windows.net/<my tenant id>/
- --scope=user.read
- --cookie-secret=REDACTED
- --pass-access-token=true
- --resource=6dae42f8-4368-4678-94ff-3960e28e3630
- --set-xauthrequest=true
@ludydoo check out my instruction at https://github.com/weinong/k8s-dashboard-with-aks-aad, I verified that it works with my AKS AAD cluster @LouisStAmour are you using AKD AAD cluster? I believe generic oidc provider will not work with AKS AAD as oidc only establishes who you are, but it doesn't provide the authorization model to access AAD resource (in this case, AKS AAD app)
@weinong , from which token are you able to extract the email claim with your setup ?
@nodranael id_token, I understand your scenario, that basically renders the fallback to access token useless in AKS AAD scenario. I'm still trying to think how it can be worked around
well, after removing some token customizations I had made while trying to make this work, I changed --scope=user.read back into --scope=openid and it eventually worked... Not sure why my id_tokens weren't signed before, and now they are signed. Go figure...
I had the same issue and I solved it using --oidc-email-claim=sub.
However, it fails to refresh the cookie with the following error:
[2021/04/13 14:18:33] [stored_session.go:122] Refreshing 1h4m44.989096937s old session cookie for Session{email:p72F5JLVKndLBeNDPZMyn1bUGKuRp_21OAti04fflWY user: PreferredUsername: token:true id_token:true created:2021-04-13 13:13:48.010903063 +0000 UTC expires:2021-04-13 14:13:46 +0000 UTC refresh_token:true} (refresh after 1m0s)
[2021/04/13 14:18:34] [azure.go:303] unable to get email claim from id_token: <nil>
[2021/04/13 14:18:34] [azure.go:256] refreshed id token Session{email:p72F5JLVKndLBeNDPZMyn1bUGKuRp_21OAti04fflWY user: PreferredUsername: token:true created:2021-04-13 14:18:34.128283903 +0000 UTC m=+4106.916933411 expires:2021-04-13 15:18:33 +0000 UTC refresh_token:true} (expired on 2021-04-13 14:13:46 +0000 UTC)
I started re-exploring the current Azure AD support through the generic oidc provider yesterday and I can confirm what @LouisStAmour said, as it seems to offer everything that is needed (and more, e.g. groups verification). I only managed to make it work with v2 tokens however.
I gave up on v1 pretty early, so can't say much about chance of success, the only thing I know for sure is that v1 would need the --resource argument to be passed to the authorization endpoint, which is currently only implemented for the Azure provider. The v2 endpoints don't support resource, but instead expect it to be passed as a scope (I only tried it with <app-id>/user_impersonation.
If you miss this part (scope in v2 and resource in v1), access tokens are not validating anymore (as they are only meant for Azure Graph API). If you pass it properly, the access token is not usable anymore to get the profile info (which you have to pass the url first, via --profile-url).
There are currently 2 options to this:
- Configure your App Registration to include the email and (optionally) group claims in the ID tokens, this avoids the need for the profile API call
- Or oauth2-proxy is modified to support using a different access token for the profile api than passing to the proxied application. Azure AD supports the "On-behalf-of" flow which could be used for this. Not sure if this can be generalised in the oidc provider or if this should be part of the Azure provider.
General question to @weinong @JoelSpeed does it actually make sense to continue supporting the Azure provider? Or does it maybe make sense to drop it and then document how to use the oidc provider for Azure?
In general, where a provider is OIDC compatible, we are looking to deprecate the existing providers and either encourage people to use the OIDC provider directly or convert the existing providers to be extensions of the existing OIDC providers.
So I think documenting using Azure on the OIDC provider would be great
This issue has been inactive for 60 days. If the issue is still relevant please comment to re-activate the issue. If no action is taken within 7 days, the issue will be marked closed.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
@JoelSpeed I agree with you but to date, Azure is not providing an OIDC compliant provider. Please take a look here especially to my comment here
This issue has been inactive for 60 days. If the issue is still relevant please comment to re-activate the issue. If no action is taken within 7 days, the issue will be marked closed.
@pierluigilenoci What would your proposed route forward be here? We are in the process of refactoring a bunch of providers to rely more heavily on the OIDC provider, reducing our maintenance burden in the long term. We also just added a new keycloakOIDC provider and deprecated the old keycloak provider.
Do you think it's worth looking into building a new OIDC based Azure provider that works with the V2 APIs but can also talk to the graph API to grab the groups?
@JoelSpeed It is absolutely worth a try. Whereas hoping that Azure fixes the behavior is quite naive.
This issue has been inactive for 60 days. If the issue is still relevant please comment to re-activate the issue. If no action is taken within 7 days, the issue will be marked closed.
This issue has been inactive for 60 days. If the issue is still relevant please comment to re-activate the issue. If no action is taken within 7 days, the issue will be marked closed.
This issue has been inactive for 60 days. If the issue is still relevant please comment to re-activate the issue. If no action is taken within 7 days, the issue will be marked closed.
This issue has been inactive for 60 days. If the issue is still relevant please comment to re-activate the issue. If no action is taken within 7 days, the issue will be marked closed.
This issue has been inactive for 60 days. If the issue is still relevant please comment to re-activate the issue. If no action is taken within 7 days, the issue will be marked closed.
This issue has been inactive for 60 days. If the issue is still relevant please comment to re-activate the issue. If no action is taken within 7 days, the issue will be marked closed.
This issue has been inactive for 60 days. If the issue is still relevant please comment to re-activate the issue. If no action is taken within 7 days, the issue will be marked closed.