headplane icon indicating copy to clipboard operation
headplane copied to clipboard

After updating headplane to 0.6.1, Auth0 OIDC is not working.

Open gunu3371 opened this issue 6 months ago • 7 comments

Description

After updating the headplane, the OIDC login feature is not working.

Server Log

2025-10-21T23:34:00.563Z [auth] ERROR: Unknown error: ClientError: unexpected JWT claim value encountered
    at e (file:///app/node_modules/.pnpm/[email protected]/node_modules/openid-client/build/index.js:116:12)
    at errorHandler (file:///app/node_modules/.pnpm/[email protected]/node_modules/openid-client/build/index.js:141:23)
    at Module.authorizationCodeGrant (file:///app/node_modules/.pnpm/[email protected]/node_modules/openid-client/build/index.js:953:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async finishAuthFlow (file:///app/build/server/assets/server-build.js:611:17)
    ... 2 lines matching cause stack trace ...
    at async commonRoute.loader (file:///app/node_modules/.pnpm/[email protected][email protected][email protected][email protected]/node_modules/react-router/dist/development/chunk-IFMMFE4R.mjs:658:19)
    at async file:///app/node_modules/.pnpm/[email protected][email protected][email protected][email protected]/node_modules/react-router/dist/development/chunk-UH6JLGW7.mjs:4327:19
    at async callLoaderOrAction (file:///app/node_modules/.pnpm/[email protected][email protected][email protected][email protected]/node_modules/react-router/dist/development/chunk-UH6JLGW7.mjs:4379:16) {
  code: 'OAUTH_JWT_CLAIM_COMPARISON_FAILED',
  [cause]: [OperationProcessingError]
}

Client Respond

{
  "code": 500,
  "error": {
    "name": "Internal Server Error",
    "description": "An unknown error occurred"
  }
}

config.yaml

oidc:
  issuer: "https://auth.example.app"
  client_id: ""
  client_secret: ""
  disable_api_key_login: true
  token_endpoint_auth_method: "client_secret_post"
  headscale_api_key: ""
  redirect_uri: "https://ts.example.app/admin/oidc/callback"

Headplane Version

0.6.1

Headscale Version

0.26.1

gunu3371 avatar Oct 21 '25 23:10 gunu3371

Honestly, this entirely boils down to the library that I use for OIDC and I'm not pleased with it either, I'll push this off for 0.7.0 as I want to fix several issues with auth for it and expand automated authentication details.

tale avatar Oct 30 '25 21:10 tale

Same here after update Pocket-ID OIDC stopped working, I reverted back to 0.6.0 and I could login again!

BobWs avatar Nov 27 '25 17:11 BobWs

I hate to hijack this problem thread but it is related, mostly ;). @BobWs I've reverted back to 0.6.0 but my Pocket-ID OIDC connection still doesn't work. If you don't mind, please post a gist or response with your headplane config. Mine is here: https://gist.github.com/kronenpj/f6e2eec9d34898f52f68dbcde3739796

kronenpj avatar Nov 27 '25 21:11 kronenpj

I don't get this, I'm using a Pocket ID instance for testing my OIDC work when I develop locally. The only things I can think of is if you are using PKCE or not and if the token endpoint authentication method is set correctly.

tale avatar Nov 27 '25 21:11 tale

I'm not using PKCE. The auth method is: "client_secret_post" (was basic) The callback is https://example.com/admin/oidc/callback right?

With 0.6.0 I get: {"code":"OAUTH_RESPONSE_BODY_ERROR","error":{"name":"Record not found"}} 0.6.1 and next (pulled 2025-11-27) instead gives this: {"code":500,"error":{"name":"Internal Server Error","description":"An unknown error occurred"}}

0.6.0 doesn't log anything, and next logs this: 2025-11-27T22:43:13.981Z [auth] ERROR: Unknown error: ClientError: invalid response encountered at e (file:///app/node_modules/.pnpm/[email protected]/node_modules/openid-client/build/index.js:116:12) at errorHandler (file:///app/node_modules/.pnpm/[email protected]/node_modules/openid-client/build/index.js:139:23) at Module.authorizationCodeGrant (file:///app/node_modules/.pnpm/[email protected]/node_modules/openid-client/build/index.js:916:21) at finishAuthFlow (file:///app/build/server/assets/server-build.js:611:30) ... 5 lines matching cause stack trace ... at async callLoaderOrAction (file:///app/node_modules/.pnpm/[email protected][email protected][email protected][email protected]/node_modules/react-router/dist/development/chunk-UH6JLGW7.mjs:4379:16) { code: 'OAUTH_INVALID_RESPONSE', [cause]: [OperationProcessingError] }

If it matters, I've tried Firefox and Chromium. Everything is Linux of one flavor or another.

kronenpj avatar Nov 27 '25 21:11 kronenpj

Ok, I got it working with 0.6.0 using client_secret_post.

kronenpj avatar Nov 27 '25 22:11 kronenpj

@gunu3371 @BobWs I identified the problem I was having. When headscale's issuer: URL contains a trailing slash the returning POST from Pocket-ID fails.

For the claim comparison problem, I'd suggest triple-checking the client_id and client_secret, and pkce (if you're using it) values correspond to the OIDC issuer's values for your application.

kronenpj avatar Nov 28 '25 16:11 kronenpj

Can you check if it works with v0.6.2-beta.2?

tale avatar Dec 04 '25 21:12 tale

Can you check if it works with v0.6.2-beta.2?

the beta version doesn't work for me I keep ending in a loop when trying to login with Pocket-ID. When I revert back to version 0.60.0 everything workings fine!

Portainer log with the beta installed:

2025-12-07T08:52:54.676Z [server] INFO: Running Node.js 22.21.1
(node:1) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
2025-12-07T08:52:54.738Z [config] WARN: oidc.token_endpoint_auth_method is deprecated and has no effect.
2025-12-07T08:52:54.753Z [config] WARN: oidc.token_endpoint_auth_method is deprecated and has no effect.
2025-12-07T08:52:54.754Z [config] WARN: oidc.redirect_uri is deprecated and will be removed in 0.7.0
2025-12-07T08:52:54.755Z [config] WARN: Please migrate to using `server.base_url` with a value of "http://hp.domain.tld/"
2025-12-07T08:52:54.870Z [config] INFO: Found a valid Headscale configuration file at /etc/headscale/config.yaml
2025-12-07T08:52:54.940Z [config] INFO: Found old user database file at /var/lib/headplane/users.json
2025-12-07T08:52:54.940Z [config] INFO: Migrating user database to the new SQL database
2025-12-07T08:52:54.942Z [config] INFO: Migrating 1 users from the old database
2025-12-07T08:52:54.955Z [config] INFO: Migrated 0 users successfully
2025-12-07T08:52:54.955Z [config] INFO: Removed old user database file /var/lib/headplane/users.json
2025-12-07T08:52:54.959Z [config] INFO: Using Docker integration
2025-12-07T08:52:54.960Z [config] INFO: Checking socket: /var/run/docker.sock
2025-12-07T08:52:54.965Z [config] ERROR: Could not request available Docker containers
2025-12-07T08:52:54.966Z [config] ERROR: Integration Docker is not available
2025-12-07T08:52:55.540Z [server] INFO: Running on 0.0.0.0:3000
2025-12-07T08:53:41.462Z [server] INFO: Received SIGTERM, shutting down...
2025-12-07T08:53:57.008Z [server] INFO: Running Node.js 22.21.1
(node:1) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
2025-12-07T08:53:57.034Z [config] WARN: oidc.token_endpoint_auth_method is deprecated and has no effect.
2025-12-07T08:53:57.048Z [config] WARN: oidc.token_endpoint_auth_method is deprecated and has no effect.
2025-12-07T08:53:57.050Z [config] WARN: oidc.redirect_uri is deprecated and will be removed in 0.7.0
2025-12-07T08:53:57.051Z [config] WARN: Please migrate to using `server.base_url` with a value of "http://hp.domain.tld/"
2025-12-07T08:53:57.156Z [config] INFO: Found a valid Headscale configuration file at /etc/headscale/config.yaml
2025-12-07T08:53:57.224Z [config] INFO: Found old user database file at /var/lib/headplane/users.json
2025-12-07T08:53:57.225Z [config] INFO: Migrating user database to the new SQL database
2025-12-07T08:53:57.227Z [config] INFO: Old user database file is empty, nothing to migrate
2025-12-07T08:53:57.227Z [config] INFO: You SHOULD remove oidc.user_storage_file from your config!
2025-12-07T08:53:57.230Z [config] INFO: Using Docker integration
2025-12-07T08:53:57.232Z [config] INFO: Checking socket: /var/run/docker.sock
2025-12-07T08:53:57.236Z [config] ERROR: Could not request available Docker containers
2025-12-07T08:53:57.238Z [config] ERROR: Integration Docker is not available
2025-12-07T08:53:57.850Z [server] INFO: Running on 0.0.0.0:3000

Portainer log when I revert back to version 0.60.0

(node:1) ExperimentalWarning: SQLite is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
tough-cookie: can't load punycode; won't use punycode for domain normalization
2025-12-07T09:09:59.795Z [server] INFO: Running Node.js 22.16.0
2025-12-07T09:09:59.840Z [config] INFO: Found a valid configuration file at /etc/headplane/config.yaml
2025-12-07T09:09:59.912Z [config] INFO: Found a valid Headscale configuration file at /etc/headscale/config.yaml
2025-12-07T09:09:59.969Z [config] INFO: Using user database file at /var/lib/headplane/users.json
2025-12-07T09:09:59.975Z [config] INFO: Using Docker integration
2025-12-07T09:09:59.977Z [config] INFO: Checking socket: /var/run/docker.sock
2025-12-07T09:10:00.016Z [config] INFO: Using container: /headscale (ID: 8b877ffeb5769902b674ee5343eb3a94364c7fd7a38fe9c7d686b4d84096a031)
2025-12-07T09:10:00.218Z [config] INFO: Using https://login.domain.tld/.well-known/openid-configuration as the OIDC issuer
2025-12-07T09:10:00.230Z [server] INFO: Running on 0.0.0.0:3000

headplane config.yml

server:
  host: "0.0.0.0"
  port: 3000
  cookie_secret: "crCX*********************E1v"
  cookie_secure: true  # Set to true if using HTTPS

headscale:
  url: "http://192.168.178.51:8080"
  public_url: "https://hs.domain.tld"
  config_path: "/etc/headscale/config.yaml"
  config_strict: true

integration:
  docker:
    enabled: true
    container_name: "headscale"
    socket: "unix:///var/run/docker.sock"

# OIDC Configuration for simpler authentication
oidc:
  issuer: "https://login.domain.tld/.well-known/openid-configuration"
  client_id: "265c********************************6"
  client_secret: "PYZ4jG**********************jPY"

  disable_api_key_login: true
  token_endpoint_auth_method: "client_secret_post"

  # This can be done with `headscale apikeys create --expiration 999d`
  headscale_api_key: "P1Uhwzy.*********************"

  # This should point to your publicly accessibly URL
  # for your Headplane instance with /admin/oidc/callback
  redirect_uri: "http://hp.domain.tld/admin/oidc/callback"

  # Stores the users and their permissions for Headplane
  # This is a path to a JSON file, default is specified below.
  user_storage_file: "/var/lib/headplane/users.json"

headscale config.yml

---
# Core Settings
server_url: https://hs.domain.tld
listen_addr: 0.0.0.0:8080
metrics_listen_addr: 127.0.0.1:9090 # Restrict metrics to localhost

# TS2021 Noise protocol
noise:
  private_key_path: /var/lib/headscale/noise_private.key

# Database
database:
  type: sqlite
  sqlite:
    path: /var/lib/headscale/db.sqlite
    write_ahead_log: true

# Network
prefixes:
  v4: 100.64.0.0/10
  v6: fd7a:115c:a1e0::/48

# headscale needs a list of DERP servers that can be presented to the clients.
derp:
  server:
    enabled: false
    region_id: 999
    region_code: "headscale"
    region_name: "Headscale Embedded DERP"
    stun_listen_addr: "0.0.0.0:3478"
    private_key_path: /var/lib/headscale/derp_server_private.key
    automatically_add_embedded_derp_region: true

    ipv4: 1.2.3.4
    ipv6: 2001:db8::1

  urls:
    - https://controlplane.tailscale.com/derpmap/default

  paths: []
  auto_update_enabled: true
  update_frequency: 24h

# Disables the automatic check for headscale updates on startup
disable_check_updates: false
ephemeral_node_inactivity_timeout: 30m

## Policy The mode can be "file" or "database" that defines
policy:
  mode: database
  # HuJSON file containing ACL policies.
  path: ""

# DNS (Split DNS + MagicDNS)
dns:
  magic_dns: true
  base_domain: ts.domain.tld # Must differ from server_url
  override_local_dns: true # Force clients to use Headscale DNS
  nameservers:
    global:
      - 192.168.178.55 # AdGuard Home
      - 192.168.178.1 # Router DNS (fallback)
    split:
      home.ts.domain.tld: []
  search_domains:
    - domain.tld

oidc:
  only_start_if_oidc_is_available: true
  issuer: "https://login.domain.tld"
  client_id: "2**********************************56"
  client_secret: "PY********************************Y"
  scope: [ "openid", "profile", "email", "groups" ]
  extra_params:
    domain_hint: domain.tld
  allowed_domains:
    - domain.tld
  allowed_groups:
    - administrators
    - users
  pkce:
    enabled: true
    method: S256

# Security
log:
  level: info # Production-safe logging
unix_socket: /var/run/headscale/headscale.sock
unix_socket_permission: "0770"
randomize_client_port: true # Better firewall compatibility

docker-compose.yml

version: '3.8'

services:
  headscale:              
    container_name: headscale
    image: headscale/headscale:v0.27.1
    command: serve
    hostname: headscale
    mem_limit: 512m
    mem_reservation: 256m
    ports:
      - "8080:8080"
      - "9090:9090"      
    volumes: 
      - /volume1/docker-sync/vpn/headscale/config:/etc/headscale
      - /volume1/docker-sync/vpn/headscale/data:/var/lib/headscale
      - /etc/localtime:/etc/localtime:ro
      - /etc/TZ:/etc/timezone:ro
    environment:
      TZ: "Europe/Berlin"
    restart: "unless-stopped"
    networks:
      proxy:

  headplane:
    container_name: headplane
    image: ghcr.io/tale/headplane:0.6.0
    hostname: headplane
    mem_limit: 512m
    mem_reservation: 256m
    ports:
      - "3000:3000"
    volumes:
      - /volume1/docker-sync/vpn/headplane/config/config.yml:/etc/headplane/config.yaml
      - /volume1/docker-sync/vpn/headscale/config/config.yml:/etc/headscale/config.yaml
      - /volume1/docker-sync/vpn/headplane/data:/var/lib/headplane
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      TZ: "Europe/Berlin"
    restart: "unless-stopped"
    networks:
      proxy:
        aliases:
          - headplane.local

networks:
  proxy:
    external: true

BobWs avatar Dec 07 '25 09:12 BobWs

Does not work. An NS_ERROR_REDIRECT_LOOP error occurs at https://example.com/admin/login?s=error_auth_failed.

Server Log

headplane  | 2025-12-07T10:20:03.580Z [server] INFO: Running Node.js 22.21.1
headplane  | (node:1) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
headplane  | (Use `node --trace-deprecation ...` to show where the warning was created)
headplane  | 2025-12-07T10:20:03.629Z [config] INFO: Found a valid Headscale configuration file at /etc/headscale/config.yaml
headplane  | 2025-12-07T10:20:03.646Z [config] INFO: Found old user database file at /var/lib/headplane/users.json
headplane  | 2025-12-07T10:20:03.646Z [config] INFO: Migrating user database to the new SQL database
headplane  | 2025-12-07T10:20:03.646Z [config] INFO: Old user database file is empty, nothing to migrate
headplane  | 2025-12-07T10:20:03.646Z [config] INFO: You SHOULD remove oidc.user_storage_file from your config!
headplane  | 2025-12-07T10:20:03.647Z [config] INFO: Using Docker integration
headplane  | 2025-12-07T10:20:03.648Z [config] INFO: Checking socket: /var/run/docker.sock
headplane  | 2025-12-07T10:20:03.650Z [config] INFO: Using container: /headscale (ID: ceeba5b71cd15ba15b3a59e3dd5a068ee1418304bfa09e536e99c21c0bcc9c6d)
headplane  | 2025-12-07T10:20:03.939Z [server] INFO: Running on 0.0.0.0:3000

gunu3371 avatar Dec 07 '25 13:12 gunu3371

@BobWs I've had Pocket ID working in 0.6.1 and all beta versions, it is most likely configuration error due to enabling something like PKCE in Pocket ID settings. Please refer to https://headplane.net/features/sso and reconfigure your SSO provider correctly.

tale avatar Dec 07 '25 19:12 tale

@gunu3371 you have not provided any kind of useful log or error, NS_ERROR implies a client side error in the browser signifying a redirect loop. I advise you to reconfigure your OIDC correctly, given that I test against Auth0 and can use it. See: https://headplane.net/features/sso

tale avatar Dec 07 '25 19:12 tale

@BobWs I've had Pocket ID working in 0.6.1 and all beta versions, it is most likely configuration error due to enabling something like PKCE in Pocket ID settings. Please refer to https://headplane.net/features/sso and reconfigure your SSO provider correctly.

I'm sorry but I can seem to get it to work! I double check the config settings with the descriptions here headplane.net/features/sso but can't find anything that is different. I also disabled the PKCE in Pocket ID (which was working fine in version 0.6.0) still didn't help.

Maybe you can tell me what is wrong with my config files, that's why I posted everything earlier above!

For now version 0.6.0 works with Pocket ID so I will stay on that version for now until I found a working solution for the update!

BobWs avatar Dec 08 '25 13:12 BobWs

@BobWs I've gone through each item in your headplane oidc configuration and added them to my configuration for 0.6.1. The only incorrect line I found is: scope: [ "openid", "profile", "email", "groups" ] It should be a string instead of a list: scope: "openid, profile, email, groups", but I believe Tale is working on making that a list in another issue. However, that would keep your service from starting as it emits an error.

Assuming you've fixed that already, the only other thing I see is that your headplane container can't see the headscale container:

2025-12-07T08:53:57.230Z [config] INFO: Using Docker integration
2025-12-07T08:53:57.232Z [config] INFO: Checking socket: /var/run/docker.sock
2025-12-07T08:53:57.236Z [config] ERROR: Could not request available Docker containers
2025-12-07T08:53:57.238Z [config] ERROR: Integration Docker is not available

You may have a permissions problem between the account starting the containers and ownership/permissions on /var/run/docker.sock.

One other note is that if you tried the 0.6.2 beta, the option for PKCE is now oidc.use_pkce with values of true or false. oidc.pkce is ignored.

kronenpj avatar Dec 08 '25 22:12 kronenpj

@BobWs I've gone through each item in your headplane oidc configuration and added them to my configuration for 0.6.1. The only incorrect line I found is: scope: [ "openid", "profile", "email", "groups" ] It should be a string instead of a list: scope: "openid, profile, email, groups", but I believe Tale is working on making that a list in another issue. However, that would keep your service from starting as it emits an error.

The scope: [ "openid", "profile", "email", "groups" ] is set in the headscale config file conform the headscale config example if I change that to as you suggested scope: "openid, profile, email, groups" I get this error:

2025-12-09T11:51:24.132Z [config] ERROR: Cannot parse Headscale configuration file at /etc/headscale/config.yaml
2025-12-09T11:51:24.132Z [config] ERROR:  - YAMLParseError: Unexpected scalar at node end at line 73, column 18:
  scope: "openid", "profile", "email", "groups" 
2025-12-07T08:53:57.230Z [config] INFO: Using Docker integration
2025-12-07T08:53:57.232Z [config] INFO: Checking socket: /var/run/docker.sock
2025-12-07T08:53:57.236Z [config] ERROR: Could not request available Docker containers
2025-12-07T08:53:57.238Z [config] ERROR: Integration Docker is not available

You may have a permissions problem between the account starting the containers and ownership/permissions on /var/run/docker.sock.

permissions for both containers are set to root don't think that is the problem.

One other note is that if you tried the 0.6.2 beta, the option for PKCE is now oidc.use_pkce with values of true or false. oidc.pkce is ignored.

Is already disabled

Still finding it strange the with version 0.6.0 everything is working fine but updating to version 0.6.1 or 0.6.2 beta it is not working and I keep getting errors or loop login.

Maybe someone could share their working config files and docker-compose files which is working with versions 0.6.1 or 0.6.2 beta then I can compare it with my settings and see what is wrong?!

BobWs avatar Dec 09 '25 12:12 BobWs

Scope is a space separated list, no commas.

tale avatar Dec 14 '25 20:12 tale