user_oidc icon indicating copy to clipboard operation
user_oidc copied to clipboard

OIDC with Synology SSO: "The received state does not match the expected value."

Open schnillerman opened this issue 1 year ago • 12 comments

Nextcloud version : 29.0.4.1 Operating system and version : Docker/Debian Apache or nginx version: stable-apache

Disclaimer

This issue is also posted on the Synology Forum. If issue is solved at either place, the corresponding issue will be udpated/closed as applicable.

Issue

OIDC Login with

returns

  1. Upon redirect from OIDC provider (Synology SSO Server) to OIDC client (Nextcloud): 504 Gateway timeout
  2. Upon refresh of page: Access denied The received state does not match the expected value. image
  3. When checking in Nextcloud → admin → users: New user has been created just fine with all scope details

Steps to replicate it

  1. Install OIDC user backend app in Nextcloud
  2. Configure Synology SSO Server with OpenID as service and Nextloud as client
  3. Migrate users that already exist in Nextcloud and OIDC provider → works fine with mapping of sud (client) to id (provider), e-mail gets updated on every login
  4. Log In with user that doesn’t exist in Nextcloud → Error described above appears.

Info exchanged between OIDC client and provider

The state parameters in both the redirect URI from client to provider and backwards are identical. The only difference is that the OIDC provider adds a code parameter which I'm pretty sure is hashed by S256 method.

Client call:

https://<redacted SSO Server URL>/
?client_id=<redacted>
&response_type=code
&scope=openid+email+profile
&redirect_uri=https%3A%2F%2F<redacted Nextcloud URL>%2Fapps%2Fuser_oidc%2Fcode
&claims=%7B%22id_token%22%3A%7B%22email%22%3Anull%2C%22name%22%3Anull%2C%22quota%22%3Anull%2C%22groups%22%3Anull%7D%2C%22userinfo%22%3A%7B%22email%22%3Anull%2C%22name%22%3Anull%2C%22quota%22%3Anull%2C%22groups%22%3Anull%7D%7D
&state=FB37D0DW5COM05LVQA96PDND69NBDBRG
&nonce=1NPDVOO8KMT6U1S7XVYWRLTPWBR38KBD
&synossoJSSDK=true
&force_login=#/signin

Provider call:

https://<redacted Nextcloud URL>/apps/user_oidc/code
?code=PsCX1tw0Gn1NwKd5OlZqLwN8ej9ToU9Z
&state=FB37D0DW5COM05LVQA96PDND69NBDBRG

The output of your Nextcloud log in Admin > Logging

No log entries pertaining to OIDC

The output of your config.php file in /path/to/nextcloud

The interesting part is in the last lines commented out with OpenID Connect user backend for Nextcloud

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',

//Cache
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => 'cloud-nextcloud-apache-redis',
    'password' => '',
    'port' => 6379,
  ),

//Apps
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'app_install_overwrite' => 
  array (
    0 => 'documentserver_community',
    1 => 'gpxmotion',
    2 => 'gpxedit',
    3 => 'apporder',
    4 => 'breezedark',
    5 => 'printer',
  ),
  'defaultapp' => 'files,dashboard',

//Instance
  'upgrade.disable-web' => true,
  'instanceid' => '<redacted>',
  'passwordsalt' => '<redacted>',
  'secret' => '<redacted>',
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'mysql',
  'version' => '29.0.4.1',
  'dbname' => 'nextcloud',
  'dbhost' => 'cloud-nextcloud-apache-db',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => '<redacted>',
  'dbpassword' => '<redacted>',
  'installed' => true,
  'trusted_domains' => 
  array (
    0 => 'cloud.<redacted>',
    1 => '192.168.1.11:10311',
  ),
  // intial value: 'overwrite.cli.url' => 'http://192.168.1.11:10311',
  'overwrite.cli.url' => 'https://cloud.<redacted>',
  'overwriteprotocol' => 'https',
  'updater.secret' => '<redacted>',
  'loglevel' => 2,
  'default_phone_region' => '<redacted>',
  'default_timezone' => '<redacted>',
  'maintenance' => false,
  'maintenance_window_start' => 3,
  'data-fingerprint' => '<redacted>',

//Proxy
  'trusted_proxies' => 
  array (
    0 => '172.16.238.0/24',
  ),
  'forwarded_for_headers' => 
  array (
    0 => 'HTTP_X_FORWARDED',
    1 => 'HTTP_FORWARDED_FOR',
    2 => 'X-Forwarded-For',
  ),

//Mail
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'tls',
  'mail_sendmailmode' => 'smtp',
  'mail_from_address' => '<redacted>',
  'mail_domain' => 'gmail.com',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtpauth' => 1,
  'mail_smtphost' => 'smtp.gmail.com',
  'mail_smtpport' => '587',
  'mail_smtpname' => '<redacted>',
  'mail_smtppassword' => '<redacted>',

//App: Memories
  'memories.exiftool' => '/var/www/html/custom_apps/memories/bin-ext/exiftool-amd64-glibc',
  'memories.vod.path' => '/var/www/html/custom_apps/memories/bin-ext/go-vod-amd64',
  'memories.db.triggers.fcu' => true,
  'memories.index.path.blacklist' => '\\/(#[Rr]ecycle|@eaDir|#snapshot|[Ll]ocked)\\/',
  'memories.vod.ffmpeg' => '/usr/bin/ffmpeg',
  'memories.vod.ffprobe' => '/usr/bin/ffprobe',
  'memories.gis_type' => 1,
  'preview_max_x' => 512,
  'preview_max_y' => 512,
  'enabledPreviewProviders' => 
  array (
    0 => 'OC\\Preview\\Image',
    1 => 'OC\\Preview\\HEIC',
    2 => 'OC\\Preview\\TIFF',
    3 => 'OC\\Preview\\Movie',
  ),

//App: ONLYOFFICE
  'onlyoffice' => 
  array (
    'verify_peer_off' => true,
  ),

//App: OpenID Connect user backend for Nextcloud https://github.com/nextcloud/user_oidc
  'user_oidc' => 
  array (
    'auto_provision' => true,
    'soft_auto_provision' => true,
	'use_pkce' => true,
  ),
);

Output errors in nextcloud.log

in /var/www/ or as admin user in top right menu (filtered for errors; use a pastebin service if necessary)

No log entries pertaining to OIDC

schnillerman avatar Aug 18 '24 21:08 schnillerman

From what I understand, the SSO Server software is proprietary and only works on Synology hardware, meaning that we unfortunately can't reproduce this if that's the case. Do you still get the same issue if you use an open-source OIDC provider such as Keycloak instead?

edward-ly avatar Oct 02 '24 23:10 edward-ly

I can confirm the same issue with Authentik and Nextcloud running in Docker on TrueNAS Scale (ElectricEel-24.10-RC.2). I am also receiving the message "Access forbidden State token does not match."

I've noticed that when I swipe from left to right on the screen, I end up back at Authentik, and when I swipe from right to left again, I return to Nextcloud, where the login then works, and I receive a "grant access" prompt. Image Image

alexstrassheim avatar Oct 24 '24 19:10 alexstrassheim

Same here with Nextcloud 30.0.4 and Authentik When i retry to login it usually works at the second time but sometimes i need to retry it multiple times.

Not sure if its an Authentik or Nextcloud user_oidc issue... but there is one clue that is could be an user_oidc issue for me; i have at least 20 different apps connected via oauth/oidc to my Authentik and this is the only app that generates a bug/error on login... whatever this means...

Nextcloud Serverlog:

{"reqId":"jOt7hVaZppP54LkW6Vgc","level":3,"time":"2025-01-21T10:04:09+00:00","remoteAddr":"*.*.*.*","user":"--","app":"core","method":"GET","url":"/","message":"Renewing session token failed: Token does not exist: token does not exist","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:134.0) Gecko/20100101 Firefox/134.0","version":"30.0.4.1","exception":{"Exception":"OC\\Authentication\\Exceptions\\InvalidTokenException","Message":"Token does not exist: token does not exist","Code":0,"Trace":[{"file":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenProvider.php","line":232,"function":"getToken","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/public/AppFramework/Db/TTransactional.php","line":45,"function":"OC\\Authentication\\Token\\{closure}","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenProvider.php","line":231,"function":"atomic","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":[{"__class__":"Closure"},{"__class__":"OC\\DB\\ConnectionAdapter"}]},{"file":"/var/www/html/lib/private/Authentication/Token/Manager.php","line":155,"function":"renewSessionToken","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":["*** sensitive parameters replaced ***","25d8a8f32a4c656884e44a0aafbbe59c"]},{"file":"/var/www/html/lib/private/User/Session.php","line":883,"function":"renewSessionToken","class":"OC\\Authentication\\Token\\Manager","type":"->","args":["*** sensitive parameters replaced ***","25d8a8f32a4c656884e44a0aafbbe59c"]},{"file":"/var/www/html/lib/base.php","line":1085,"function":"loginWithCookie","class":"OC\\User\\Session","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/base.php","line":992,"function":"handleLogin","class":"OC","type":"::","args":[{"__class__":"OC\\AppFramework\\Http\\Request"}]},{"file":"/var/www/html/index.php","line":24,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenProvider.php","Line":165,"Previous":{"Exception":"OCP\\AppFramework\\Db\\DoesNotExistException","Message":"token does not exist","Code":0,"Trace":[{"file":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenProvider.php","line":157,"function":"getToken","class":"OC\\Authentication\\Token\\PublicKeyTokenMapper","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenProvider.php","line":232,"function":"getToken","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/public/AppFramework/Db/TTransactional.php","line":45,"function":"OC\\Authentication\\Token\\{closure}","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenProvider.php","line":231,"function":"atomic","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":[{"__class__":"Closure"},{"__class__":"OC\\DB\\ConnectionAdapter"}]},{"file":"/var/www/html/lib/private/Authentication/Token/Manager.php","line":155,"function":"renewSessionToken","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":["*** sensitive parameters replaced ***","25d8a8f32a4c656884e44a0aafbbe59c"]},{"file":"/var/www/html/lib/private/User/Session.php","line":883,"function":"renewSessionToken","class":"OC\\Authentication\\Token\\Manager","type":"->","args":["*** sensitive parameters replaced ***","25d8a8f32a4c656884e44a0aafbbe59c"]},{"file":"/var/www/html/lib/base.php","line":1085,"function":"loginWithCookie","class":"OC\\User\\Session","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/base.php","line":992,"function":"handleLogin","class":"OC","type":"::","args":[{"__class__":"OC\\AppFramework\\Http\\Request"}]},{"file":"/var/www/html/index.php","line":24,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenMapper.php","Line":81},"message":"Renewing session token failed: Token does not exist: token does not exist","user":"1","exception":[],"CustomMessage":"Renewing session token failed: Token does not exist: token does not exist"},"id":"67920ec5414b6"}

if i can provide further tests/results/logs just let me know.

motey avatar Jan 23 '25 09:01 motey

Update: Sorry, i did not realize there is a pending patch for user_oidc in my nextcloud instance (6.2.0->6.2.1)... it seems to have fixed the issue for me 💟 (will come back here if the problem reoccurs)

motey avatar Jan 23 '25 16:01 motey

Update: Sorry, the issue still occurs randomly with user_oidc :( After 1-4 retries or a browser switch it usually works suddenly again.

{"reqId":"tvGSw2D6wDA84SoHVlMf","level":3,"time":"2025-02-10T11:18:54+00:00","remoteAddr":"146.107.213.240","user":"--","app":"core","method":"GET","url":"/","message":"Renewing session token failed: Token does not exist: token does not exist","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:134.0) Gecko/20100101 Firefox/134.0","version":"30.0.5.1","exception":{"Exception":"OC\\Authentication\\Exceptions\\InvalidTokenException","Message":"Token does not exist: token does not exist","Code":0,"Trace":[{"file":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenProvider.php","line":232,"function":"getToken","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/public/AppFramework/Db/TTransactional.php","line":45,"function":"OC\\Authentication\\Token\\{closure}","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenProvider.php","line":231,"function":"atomic","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":[{"__class__":"Closure"},{"__class__":"OC\\DB\\ConnectionAdapter"}]},{"file":"/var/www/html/lib/private/Authentication/Token/Manager.php","line":155,"function":"renewSessionToken","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":["*** sensitive parameters replaced ***","8abd059e2dc0db839065da7cc59a7cda"]},{"file":"/var/www/html/lib/private/User/Session.php","line":883,"function":"renewSessionToken","class":"OC\\Authentication\\Token\\Manager","type":"->","args":["*** sensitive parameters replaced ***","8abd059e2dc0db839065da7cc59a7cda"]},{"file":"/var/www/html/lib/base.php","line":1085,"function":"loginWithCookie","class":"OC\\User\\Session","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/base.php","line":992,"function":"handleLogin","class":"OC","type":"::","args":[{"__class__":"OC\\AppFramework\\Http\\Request"}]},{"file":"/var/www/html/index.php","line":24,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenProvider.php","Line":165,"Previous":{"Exception":"OCP\\AppFramework\\Db\\DoesNotExistException","Message":"token does not exist","Code":0,"Trace":[{"file":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenProvider.php","line":157,"function":"getToken","class":"OC\\Authentication\\Token\\PublicKeyTokenMapper","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenProvider.php","line":232,"function":"getToken","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/public/AppFramework/Db/TTransactional.php","line":45,"function":"OC\\Authentication\\Token\\{closure}","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenProvider.php","line":231,"function":"atomic","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":[{"__class__":"Closure"},{"__class__":"OC\\DB\\ConnectionAdapter"}]},{"file":"/var/www/html/lib/private/Authentication/Token/Manager.php","line":155,"function":"renewSessionToken","class":"OC\\Authentication\\Token\\PublicKeyTokenProvider","type":"->","args":["*** sensitive parameters replaced ***","8abd059e2dc0db839065da7cc59a7cda"]},{"file":"/var/www/html/lib/private/User/Session.php","line":883,"function":"renewSessionToken","class":"OC\\Authentication\\Token\\Manager","type":"->","args":["*** sensitive parameters replaced ***","8abd059e2dc0db839065da7cc59a7cda"]},{"file":"/var/www/html/lib/base.php","line":1085,"function":"loginWithCookie","class":"OC\\User\\Session","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/base.php","line":992,"function":"handleLogin","class":"OC","type":"::","args":[{"__class__":"OC\\AppFramework\\Http\\Request"}]},{"file":"/var/www/html/index.php","line":24,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"/var/www/html/lib/private/Authentication/Token/PublicKeyTokenMapper.php","Line":81},"message":"Renewing session token failed: Token does not exist: token does not exist","user":"1","exception":[],"CustomMessage":"Renewing session token failed: Token does not exist: token does not exist"},"id":"67a9e0c481d46"}

I happily will provide any additional data if necessary.

motey avatar Feb 10 '25 11:02 motey

I recently ran into this, in my case this seems to have been caused by having multiple nextcloud nodes and not having session affinity setup on my proxy. After setting cookie affinity in my proxy, the issue is fixed.

rouke-broersma avatar May 18 '25 12:05 rouke-broersma

thx for the hint. I also have multiple nextcloud nodes each under a custom subdomain with traefik as the reverse proxy. As far as i understand session affinity cant be the issue in this setup.

motey avatar May 30 '25 09:05 motey

thx for the hint. I also have multiple nextcloud nodes each under a custom subdomain with traefik as the reverse proxy. As far as i understand session affinity cant be the issue in this setup.

Yes that is possibly the issue. Your client connect to your traefik, trafik sends the connection to node 1, node 1 creates and signs the token and returns the token to your client, your client then offers the token to traefik, traefik now offers the token to node 2 since there is no session affinity. Node 2 did not sign the token /does not recognize the session created by node1 and denies authorization. With session affinity enabled in traefik, traefik knows your client originally connected to node 1 and tries to always offer node1 to your client. Node 1 created the session so it knows about it and approves the authorization. This explains why it works after some refresh. The default load balancing in most reverse proxy is round robin. So if you have for example 3 nodes in your nextcloud cluster, 1 in 3 refreshes work, because 1 in 3 refreshes round-robins to the original node that created the token.

Normally in a distributed system you want the session information to be shared between nodes in the cluster but this does not seem to be the case with this plugin (or it's possible but requires additional configuration that I don't know about).

rouke-broersma avatar May 30 '25 10:05 rouke-broersma

Thank you very much for the explanation ❤ . I'll give it a try (https://doc.traefik.io/traefik/routing/services/#sticky-sessions) and if it works i will document here what i did

motey avatar May 30 '25 18:05 motey

Thank you very much for the explanation ❤ . I'll give it a try (https://doc.traefik.io/traefik/routing/services/#sticky-sessions) and if it works i will document here what i did

Please do!

schnillerman avatar Jun 01 '25 08:06 schnillerman

Hi all,

same problem with Authentik, NginX Proxy manager and Nextcloud. All setup as Dockers on Unraid.

Maybe something to add to the problem, even if I login via User/password, the error occurs as long as the App is enabled in Nextcloud.

rouloy avatar Jun 01 '25 21:06 rouloy

Thank you very much for the explanation ❤ . I'll give it a try (https://doc.traefik.io/traefik/routing/services/#sticky-sessions) and if it works i will document here what i did

After some weeks of testing the sticky cookie, it seems the error occures less often and less stuborn. one reload is enough to get rid of the error. Before that i needed to reload the site at least 3 times to get a successfull login.

So better, but not resolved...

These are the docker compose Treafik-labels i used to enable sticky cooies for my nextcloud instances

...

      - "traefik.http.services.srv-nextcloud01.loadbalancer.server.port=80"
      - "traefik.http.services.srv-nextcloud01.loadbalancer.sticky.cookie=true"
      - "traefik.http.services.srv-nextcloud01.loadbalancer.sticky.cookie.name=sticky_nextcloud01"
      - "traefik.http.services.srv-nextcloud01.loadbalancer.sticky.cookie.secure=true"
      - "traefik.http.services.srv-nextcloud01.loadbalancer.sticky.cookie.domain=nextclou01.mydomain.org"

...

@schnillerman

motey avatar Aug 04 '25 10:08 motey