authentik icon indicating copy to clipboard operation
authentik copied to clipboard

providers/proxy: fix handling of AUTHENTIK_HOST_BROWSER

Open chrootlogin opened this issue 1 year ago β€’ 3 comments

Details

This PR fixes the incorrect handling of the AUTHENTIK_HOST_BROWSER environment variable. When this variable is set, the OIDC issuer is now also adjusted to prevent the error: β€œoidc: id token issued by a different provider”.

This change is necessary because Authentik, as the OIDC issuer, uses the public browser URL for token issuance, but within environments like Kubernetes, the internal cluster URL is typically used. Therefore, the token must be validated against the public URL, not the internal one.

Rationale

The current behavior leads to mismatched issuer validation when tokens are issued by the public URL but verified using the internal URL. This PR addresses this by ensuring the issuer is correctly set based on the AUTHENTIK_HOST_BROWSER value.

Linked Issues

Closes #9622, #4688, #6476.


Checklist

  • [x] Local tests pass (ak test authentik/)
  • [x] The code has been formatted (make lint-fix)

If an API change has been made

  • [ ] The API schema has been updated (make gen-build)

If changes to the frontend have been made

  • [ ] The code has been formatted (make web)

If applicable

  • [ ] The documentation has been updated
  • [ ] The documentation has been formatted (make website)

chrootlogin avatar Oct 18 '24 10:10 chrootlogin

Deploy Preview for authentik-docs ready!

Name Link
Latest commit 1c474a22ef0561419acd4ec269c8d96a74df08de
Latest deploy log https://app.netlify.com/sites/authentik-docs/deploys/67192bf8dd4b3c0008740e67
Deploy Preview https://deploy-preview-11722--authentik-docs.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

netlify[bot] avatar Oct 18 '24 10:10 netlify[bot]

Deploy Preview for authentik-storybook ready!

Name Link
Latest commit 1c474a22ef0561419acd4ec269c8d96a74df08de
Latest deploy log https://app.netlify.com/sites/authentik-storybook/deploys/67192bf7f21c410008aedcf9
Deploy Preview https://deploy-preview-11722--authentik-storybook.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

netlify[bot] avatar Oct 18 '24 10:10 netlify[bot]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 92.56%. Comparing base (b57df12) to head (1c474a2). Report is 55 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #11722      +/-   ##
==========================================
- Coverage   92.70%   92.56%   -0.15%     
==========================================
  Files         739      760      +21     
  Lines       36747    37714     +967     
==========================================
+ Hits        34067    34909     +842     
- Misses       2680     2805     +125     
Flag Coverage Ξ”
e2e 49.16% <ΓΈ> (-0.16%) :arrow_down:
integration 24.93% <ΓΈ> (-0.03%) :arrow_down:
unit 90.13% <ΓΈ> (-0.10%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Oct 18 '24 14:10 codecov[bot]

thanks for the PR @chrootlogin! could you update the tests with this new logic?

BeryJu avatar Oct 23 '24 16:10 BeryJu

Yes, i will take a look into it!

chrootlogin avatar Oct 23 '24 16:10 chrootlogin

I now get the same error but then the other way around.

authentik_host is set to http://authentik-server authentik_host_browser is set to https://auth.domain.com

oidc: id token issued by a different provider, expected \"https://auth.domain.com/application/o/app/\" got \"http://authentik-server/application/o/app/\"","event":"failed to redeem code","level":"warning","logger":"authentik.outpost.proxyv2.application","name":"app","timestamp":"2024-11-02T17:58:54Z"}

Manually switching the outpost back to version 2024.8 makes it work again even though the Authentik version is 2024.10

luukrijnbende avatar Nov 02 '24 18:11 luukrijnbende

@luukrijnbende Hmm, that actually makes no sense to me. Because your token should be created by the external URL and not with the internal one. For me it looks like that Authentik issues the token with his internal URL which is wrong.

Do you have a proxy in front of Authentik? And does this proxy forward the Host header? See: Reverse Proxy Configuration Because from what I know authentik issues the token with the hostname taken from Host header. You may also should check the X-Forwarded-Host header: https://docs.goauthentik.io/docs/add-secure-apps/providers/proxy/#x-forwarded-host Also there may is a related issue: https://github.com/goauthentik/authentik/issues/2284

From my side I can say that Authentik deployed in a Kubernetes cluster with nginx-ingress now does the right thing with 2024.10.0

chrootlogin avatar Nov 03 '24 13:11 chrootlogin

Yes, I do have a proxy in front of it. I'm running it in Kubernetes as well and I'm using https://haproxy-ingress.github.io/ as my ingress controller. This is sending X-Forwarded-Host, X-Forwarded-Proto, X-Forwarded-For, Host and X-Original-Url.

My ingress resources are configured as follows to use forward auth;

haproxy-ingress.github.io/auth-url: http://ak-outpost-proxy.authentik.svc.cluster.local:9000/outpost.goauthentik.io/auth/nginx
haproxy-ingress.github.io/auth-signin: https://%[hdr(host)]/outpost.goauthentik.io/start?rd=%[url]

Some more info:

  • I deployed Authentik using the Helm chart.
  • I'm using a separate proxy deployment, not the embedded one.
  • The separate proxy deployment automatically creates an ingress resource for all apps assigned to it for the /outpost.goauthentik.io path.
  • If I point the proxy at https://auth.domain.com instead of http://authentik-server and leave the authentik_host_browser empty, it works and the issuer is the external domain, but then of course all traffic goes back through the ingress again instead of staying inside the cluster.

luukrijnbende avatar Nov 03 '24 18:11 luukrijnbende

This really sounds like header issues. Because my setup is mostly the same like yours. The only difference I see is that I use nginx-ingress. Which then is made available to the interwebs.

I also have separate authentik-outpost deployments with the following env:

AUTHENTIK_HOST: "http://authentik-server.security.svc.cluster.local"
AUTHENTIK_HOST_BROWSER: "https://authentik.my-public.domain"
AUTHENTIK_INSECURE: "false"
AUTHENTIK_TOKEN: XXXXXXXXXXXXXXX

For Authentik I just use the official helm chart. And on my outpost ingresses I use the nginx.ingress.kubernetes.io/auth-snippet: proxy_set_header X-Forwarded-Host $http_host; annotation which resets the X-Forwarded-Host to the actual hostname entered in the browser. I think this setting is very important, as it is what Authentik uses as issuing host.

chrootlogin avatar Nov 03 '24 21:11 chrootlogin

This also broke my configuration on the upgrade to 2024.10.0 with the same error as @chrootlogin. I am not running in k8s, but instead in Podman. I ended up "fixing" this by setting AUTHENTIC_HOST to the external URL for Authentik, eating the cost of the traffic exiting the virtual network.

yodaldevoid avatar Nov 03 '24 22:11 yodaldevoid

@yodaldevoid Just to make it sure. I don't have any errors anymore after the release of this pull-request.

With the patch I did Authentik now works as excepted. This pull-request which seems to break your setup fixes #9622, #4688, #6476.

So my best guess is that you don't have set the headers Host and X-Forwarded-Host correctly on the Authentik Server (not the Proxy).

Because the issuing works like this:

  1. You don't have any token. You get redirected to Authentik Server.
  2. Authentik Server receives the Request and does the login flow.
  3. When logged in, Authentik issues a token with the Issuer beeing the Hostname in the Host or X-Forwarded-Host header.
  4. You get redirected to the Outpost.
  5. Outpost checks if the Issuer is either the AUTHENTIK_HOST or if set the AUTHENTIK_HOST_BROWSER value.

TL;DR Make sure that the Authentik Server knows it's external Hostname either by setting the Host or X-Forwarded-Host header. You can also check this in the logs. My findings are documented in #9622.

Otherwise if configured wrong. Authentik Server issues tokens for it's internal URL not the external one. And the outpost tries to check with the external one and if they don't match your authentication will be blocked.

And just for clarification. This pull-request only fixes the issue that the AUTHENTIK_HOST_BROWSER was ignored when checking the issuer URL. This broke setups where Authentik issued tokens against the external URL.

So may you could also check which URL Authentik uses to issue your tokens (iss in your openid claims). When there is the internal url or ip then your setup is definitely wrong.

chrootlogin avatar Nov 03 '24 23:11 chrootlogin

I did some more digging through the logs and it seems that Authentik uses the URL that the outpost uses to talk to Authentik as the issuer. As far as I can see all requests that are coming through the proxy have the right headers and Authentik can see that the host is auth.domain.com. As soon as the proxy does a call it uses the internal URL (bypassing the proxy) and that is then seen as the host.

{"auth_via": "oauth_client_secret", "domain_url": "authentik-server", "event": "/application/o/token/", "host": "authentik-server", "level": "info", "logger": "authentik.asgi", "method": "POST", "pid": 78278, "remote": "10.42.1.31", "request_id": "a2df579b2f2c49af8c390e52e78e73ec", "runtime": 43, "schema_name": "public", "scheme": "http", "status": 200, "timestamp": "2024-11-04T15:09:52.328994", "user": "", "user_agent": "goauthentik.io/outpost/2024.8.4 (provider=App)"}

Yields a token with issuer: http://authentik-server/application/o/app/

If I then change the authentik_host to be http://authentik-server.authentik.svc.cluster.local it will use that as the host and issue a token with it.

{"auth_via": "oauth_client_secret", "domain_url": "authentik-server.authentik.svc.cluster.local", "event": "/application/o/token/", "host": "authentik-server.authentik.svc.cluster.local", "level": "info", "logger": "authentik.asgi", "method": "POST", "pid": 41, "remote": "10.42.1.38", "request_id": "54aefa2e4d6148caa0d43986b13eb233", "runtime": 67, "schema_name": "public", "scheme": "http", "status": 200, "timestamp": "2024-11-04T15:37:41.881911", "user": "", "user_agent": "goauthentik.io/outpost/2024.8.4 (provider=App)"}

Yields a token with issuer: http://authentik-server.authentik.svc.cluster.local/application/o/app/

If I update the outpost to version 2024.10.0 instead of 2024.8.4 it still issues the token in the same way but then the id token issued by a different provider error returns.

I'll see if I can reproduce it using the nginx ingress controller, maybe that will give us some more clues πŸ˜„

luukrijnbende avatar Nov 04 '24 15:11 luukrijnbende

@yodaldevoid Just to make it sure. I don't have any errors anymore after the release of this pull-request.

To be clear, I fully expect the problem to be on my end. I am not suggesting that this PR is reverted or that it was implemented wrong. I wanted to leave a comment for anyone else who runs into the same error pointing them to a possibly easy, if not quite perfect, solution.

Thank you for the suggestions, I will look into them.

yodaldevoid avatar Nov 04 '24 18:11 yodaldevoid

That all makes no sense to me. πŸ˜…

Because actually I also use http://authentik-server.security.cluster.local as AUTHENTIK_HOST and this means that Authentik Outpost directly communicates with the Authentik Server theough the service (not using the ingress) which should not modify or include any headers.

chrootlogin avatar Nov 04 '24 23:11 chrootlogin

@yodaldevoid @luukrijnbende Could you please check whats written in the yellow field below the outpost name when opening Authentik Settings -> Applications -> Outposts. There should be the login URL. Do you see there the internal or external URL?

Also you could check in your Outpost configuration whats mentioned at extended settings "authentik_host". There should be the external URL.

chrootlogin avatar Nov 04 '24 23:11 chrootlogin

The proxy says, Logging in via http://authentik-server.authentik.svc.cluster.local. and that changes whenever I change the authentik_host in the configuration. The weird thing is, when I click View Deployment Info it does show the external URL auth.domain.com in both authentik_host and authentik_host_browser even though the configuration itself is different.

Is your outpost deployed manually or automatically by the Authentik cluster integration?

luukrijnbende avatar Nov 05 '24 12:11 luukrijnbende

@luukrijnbende Okay, I think we get closer to the root of the problem.

My outpost is deployed manually via a helmchart (bjw-s universal library Helmchart). And my Authentik UI shows Logging in via https://authentik.my-external-domain.com. It also says my AUTHENTIK_HOST is "https://authentik.my-external-domain.com" even that I have configured AUTHENTIK_HOST to "http://authentik-server.security.cluster.local".

Also from what I can see in the logs of Authentik Server. The outpost definitely communicates through the internal network:

{"auth_via": "api_token", "domain_url": "authentik-server.security.svc.cluster.local", "event": "/api/v3/root/config/", "host": "authentik-server.security.svc.cluster.local", "level": "info", "logger": "authentik.asgi", "method": "GET", "pid": xxx, "remote": "10.42.xxx.xxx", "request_id": "xxxxxxxxxx", "runtime": 7, "schema_name": "public", "scheme": "http", "status": 200, "timestamp": "2024-11-05T14:50:07.215403", "user": "ak-outpost-xxxxxxxx", "user_agent": "goauthentik.io/outpost/2024.10.0"}

Another question when you watch the logs of Authentik Server and open "authentik.my-external-domain.com" in your Browser what do you see?

{"auth_via": "session", "domain_url": "authentik.my-external-domain.com", "event": "/api/v3/core/applications/?only_with_launch_url=true&ordering=name&page=1&page_size=100", "host": "authentik.my-external-domain.com", "level": "info", "logger": "authentik.asgi", "method": "GET", "pid": xxxxx, "remote": "xxxxxx", "request_id": "xxxxxxx", "runtime": 44, "schema_name": "public", "scheme": "https", "status": 200, "timestamp": "2024-11-05T15:01:56.608976", "user": "xxxxx", "user_agent": "xxxxxxxxxx"}

Under domain_url and host do you see your external or internal domain? Because when you see your internal domain, then definitely something is wrong.

chrootlogin avatar Nov 05 '24 15:11 chrootlogin

Hmm, mine is deployed automatically by the integration and the authentik_host in the configuration of the outpost is mapped to a secret for that outpost which is used for the environment variables of the deployment. So setting that apparently changes the issuing of the tokens and the URL used for the outpost to communicate with Authentik. Maybe @BeryJu can chime in here and shed some light on how this should work?

I was thinking about moving it to a separate helm chart to make the deployment of it explicit within my gitops. Which then, using the right environment variables, should fix this issue.

When I open Authentik and login as a user I see my external auth.domain.com in all the logs as both domain_url and host.

luukrijnbende avatar Nov 05 '24 16:11 luukrijnbende

@luukrijnbende Yes thats really interesting. What I can say from my side is that my pull-request should only modify requests when runned as external deployed Outpost.

See: https://github.com/chrootlogin/authentik/blob/1c474a22ef0561419acd4ec269c8d96a74df08de/internal/outpost/proxyv2/application/endpoint.go

// ...
// For the embedded outpost, we use the configure `authentik_host` for the browser URLs
// and localhost (which is what we've got from the API) for backchannel URLs
//
// For other outposts, when `AUTHENTIK_HOST_BROWSER` is set, we use that for the browser URLs
// and use what we got from the API for backchannel
hostBrowser := config.Get().AuthentikHostBrowser
if !embedded && hostBrowser == "" {
  return ep
}
var newHost *url.URL = aku
var newBrowserHost *url.URL
if embedded {
  if authentikHost == "" {
    log.Warning("Outpost has localhost/blank API Connection but no authentik_host is configured.")
    return ep
  }
  newBrowserHost = aku
} else if hostBrowser != "" {
  browser, err := url.Parse(hostBrowser)
  if err != nil {
    return ep
  }
}
newBrowserHost = browser
// ...
// This is only used in embedded outposts as there we can guarantee that the request
// is routed correctly
if embedded {
  ep.Issuer = updateURL(ep.Issuer, newHost.Scheme, newHost.Host)
  ep.JwksUri = updateURL(jwksUri, newHost.Scheme, newHost.Host)
} else {
  // Fixes: https://github.com/goauthentik/authentik/issues/9622 / ep.Issuer must be the HostBrowser URL
  ep.Issuer = updateURL(ep.Issuer, newBrowserHost.Scheme, newBrowserHost.Host)
}
// ...

So my code change ist actually the last else which overwrites the Issuer when having a "AUTHENTIK_HOST_BROWSER" and not running as embedded, e.g. deployed manually.

chrootlogin avatar Nov 06 '24 14:11 chrootlogin

So mine was deployed as an external outpost, not the embedded one, but managed by Authentik and the Kubernetes integration it provides. I have now switched to a non-managed deployment of the outpost to make stuff more explicit within my GitOps and to be able to block access from Authentik to the Kubernetes API.

The configuration in Authentik is now using auth.domain.com for the authentik_host where the environment for the outpost is using the internal URL and with that it works great with the latest version. Tokens are issued using the external URL while communication happens over the internal URL.

Thanks for all the help!

luukrijnbende avatar Nov 07 '24 20:11 luukrijnbende