goauthentik
NGINX auth_request 500 internal server error
Describe your question/ I will reopen issue #2594 due it has been closed for months now and there is no proper answer on how to fix it. I have ran to this exact same problem. Issue is that when running config provided by authentik + added my HTTP->HTTPS redirect and actual internal app which it needs to call after auth, I do get error 500 on browser, 502 in logs.
Relevant infos I am running NGINX as standalone and I have configured reverse proxy per Authentik documentation. Latest Authentik in use which Docker can provide on Hetzner VM.
domain1.tld - Authentik server running on different domain, domain2.tld - Domain which has the actual service running.
Authentik NGINX config:
# Upstream where your authentik server is hosted.
upstream authentik {
server localhost:9443;
# Improve performance by keeping some connections alive.
keepalive 10;
}
# Upgrade WebSocket if requested, otherwise use keepalive
map $http_upgrade $connection_upgrade_keepalive {
default upgrade;
'' '';
}
server {
# HTTP server config
listen 80;
server_name auth.domain1.tld;
# 301 redirect to HTTPS
location / {
return 301 https://$host$request_uri;
}
}
server {
# HTTPS server config
listen 443 ssl http2;
server_name auth.domain1.tld;
# TLS certificates
ssl_certificate /var/ssl/domain1/fullchain.pem;
ssl_certificate_key /var/ssl/domain1/privkey.pem;
# Proxy site
location / {
proxy_pass https://authentik;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade_keepalive;
}
}
Application config:
server {
# SSL and VHost configuration
listen 443 ssl http2;
server_name bans.domain2.tld;
ssl_certificate /etc/letsencrypt/live/subdomain.domain2.tld/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/subdomain.domain2.tld/privkey.pem;
# Increase buffer size for large headers
# This is needed only if you get 'upstream sent too big header while reading response
# header from upstream' error when trying to access an application protected by goauthentik
proxy_buffers 8 16k;
proxy_buffer_size 32k;
location / {
# Put your proxy_pass to your application here, and all the other statements you'll need
proxy_pass http://127.0.0.1:8888;
proxy_set_header Host $host;
##############################
# authentik-specific config
##############################
auth_request /outpost.goauthentik.io/auth/nginx;
error_page 401 = @goauthentik_proxy_signin;
auth_request_set $auth_cookie $upstream_http_set_cookie;
add_header Set-Cookie $auth_cookie;
# translate headers from the outposts back to the actual upstream
auth_request_set $authentik_username $upstream_http_x_authentik_username;
auth_request_set $authentik_groups $upstream_http_x_authentik_groups;
auth_request_set $authentik_email $upstream_http_x_authentik_email;
auth_request_set $authentik_name $upstream_http_x_authentik_name;
auth_request_set $authentik_uid $upstream_http_x_authentik_uid;
proxy_set_header X-authentik-username $authentik_username;
proxy_set_header X-authentik-groups $authentik_groups;
proxy_set_header X-authentik-email $authentik_email;
proxy_set_header X-authentik-name $authentik_name;
proxy_set_header X-authentik-uid $authentik_uid;
}
# all requests to /outpost.goauthentik.io must be accessible without authentication
location /outpost.goauthentik.io {
proxy_pass https://auth.domain1.tld/outpost.goauthentik.io;
# ensure the host of this vserver matches your external URL you've configured
# in authentik
proxy_set_header Host $host;
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
add_header Set-Cookie $auth_cookie;
auth_request_set $auth_cookie $upstream_http_set_cookie;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
}
# Special location for when the /auth endpoint returns a 401,
# redirect to the /start URL which initiates SSO
location @goauthentik_proxy_signin {
internal;
add_header Set-Cookie $auth_cookie;
return 302 /outpost.goauthentik.io/start?rd=$request_uri;
# For domain level, use the below error_page to redirect to your authentik server with the full redirect path
# return 302 https://auth.domain1.tld/outpost.goauthentik.io/start?rd=$scheme://$http_host$request_uri;
}
}
server {
if ($host = bans.domain2.tld) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name bans.domain2.tld;
return 404; # managed by Certbot
}
server {
server_name internal-bans.domain2.tld;
listen 127.0.0.1:8888;
root /var/www/litebans;
index index.php;
access_log /var/log/nginx/litebans.app-access.log;
error_log /var/log/nginx/litebans.app-error.log error;
# allow larger file uploads and longer script runtimes
client_max_body_size 100m;
client_body_timeout 120s;
sendfile off;
location / {
try_files $uri $uri/ /index.php?$query_string;
}
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param PHP_VALUE "upload_max_filesize = 100M \n post_max_size=100M";
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param HTTP_PROXY "";
fastcgi_intercept_errors off;
fastcgi_buffer_size 16k;
fastcgi_buffers 4 16k;
fastcgi_connect_timeout 300;
fastcgi_send_timeout 300;
fastcgi_read_timeout 300;
include /etc/nginx/fastcgi_params;
}
location ~ /\.ht {
deny all;
}
Logs NGINX error log
2022/12/13 13:00:00 [error] 1371849#0: *40 no live upstreams while connecting to upstream, client: 162.158.238.213, server: bans.domain2.tld, request: "GET / HTTP/2.0", subrequest: "/outpost.goauthentik.io/auth/nginx", upstream: "https://auth.domain1.tld/outpost.goauthentik.io/auth/nginx", host: "bans.domain2.tld"
2022/12/13 13:00:00 [error] 1371849#0: *40 auth request unexpected status: 502 while sending to client, client: 162.158.238.213, server: bans.domain2.tld, request: "GET / HTTP/2.0", host: "bans.domain2.tld"
2022/12/13 13:00:01 [error] 1371849#0: *46 no live upstreams while connecting to upstream, client: 162.158.239.46, server: bans.domain2.tld, request: "GET /favicon.ico HTTP/2.0", subrequest: "/outpost.goauthentik.io/auth/nginx", upstream: "https://auth.domain1.tld/outpost.goauthentik.io/auth/nginx", host: "bans.domain2.tld", referrer: "https://bans.domain2.tld/"
2022/12/13 13:00:01 [error] 1371849#0: *46 auth request unexpected status: 502 while sending to client, client: 162.158.239.46, server: bans.domain2.tld, request: "GET /favicon.ico HTTP/2.0", host: "bans.domain2.tld", referrer: "https://bans.domain2.tld/"
There is nothing on Authentik logs about this (that I can see), but I am more than happy to provide more logs if needed.
So small(ish) update. I managed to get this somehow working. Originally the domains are protected by Cloudflare. But once I configured in NGINX to use IP instead of domain and added the cert given by authentik + disabled ssl verification on proxy_pass due self signed cert, it started working just fine. So could this be something possibly to do with Cloudflare?
EDIT: Forgot to mention that both domains use Cloudflare SSL certificates which I made in CF dashboard.
The procedure I have been needing to do is edit lines 41 and 59 each time to my servers IP address, similar to what you mentioned.
I too use CF and an CF SSL Cert. on the nginx side and none on the proxy_pass.
I also ran into this same problem on 2023.3.0. I worked around it by hard-coding my Authentik server's IP address to bypass Cloudflare. If your NGINX can talk directly to Authentik, you could use the LAN IP. In my case, it could not, so I used the public WAN IP of my Authentik server.
Here are the relevant changes;
location /outpost.goauthentik.io {
proxy_pass https://0.0.0.0/outpost.goauthentik.io;
# ensure the host of this vserver matches your external URL you've configured
# in authentik
proxy_set_header Host login.acme.com;
...
Possibly related: https://github.com/goauthentik/authentik/issues/4381 https://github.com/goauthentik/authentik/issues/3147
@BeryJu comment in https://github.com/goauthentik/authentik/issues/4496#issuecomment-1400898062 seems to relate to the root of this issue as well
in the instance web site on the providers page, on the setup section, my standalone nginx instructions renders with a FQDN host, so it set me down the wrong path initially.
this restriction does introduce a constraint against setting up authentik that only exposes its services behind a reverse proxy (therefore lacking access by IP only).
it's unclear what options you would have in those scenarios other than to punch a hole directly into authentik to guarantee it can be reached by IP as well as by FQDN.
are there any other workarounds to get FQDN working in proxy_pass ?
Unsure if this helps anyone setting NGINX up, however take a peek at my comment for a different issue for configuration?
https://github.com/goauthentik/authentik/issues/4381#issuecomment-1605195711
i just did it
location /outpost.goauthentik.io {
proxy_pass http://192.168.0.1:9003/outpost.goauthentik.io;
...
proxy_redirect http://192.168.0.1:9003 https://auth.domen.com;
}
