synapse Error at OIDC initialization

Description

I've configured my OIDC server (Authentik) for Synapse using the official guide.

My homeserver.yml is populated as this (the URL of the matrix server is matrix.tld):

oidc_providers:
    - idp_id: authentik
      idp_name: authentik
      # discover: true
      skip_verification: true
      issuer: "https://authentik.tld/application/o/matrix/"
      client_id: <redacted>
      client_secret: <redacted>
      authorization_endpoint: "https://authentik.tld/application/o/matrix/"
      token_endpoint: "https://authentik.tld/application/o/token/"
      userinfo_endpoint: "https://authentik.tld/application/o/userinfo/"
      jwks_uri: "https://authentik.tld/application/o/matrix/jwks/"
      scopes:
          - "openid"
          - "profile"
          - "email"
      user_mapping_provider:
          config:
              localpart_template: "{{ user.preferred_username }}"
              display_name_template: "{{ user.name|capitalize }}"

However, at the docker compose up, the synapse server restarts in loop because it can't initialize the provider. It seems it reaches a timeout, from the logs, although it doesn't even wait half a second and directly restarts.

Here are the logs:

matrix-server exited with code 0
matrix-server   | Starting synapse with args -m synapse.app.homeserver --config-path /data/homeserver.yaml
matrix-server   | Error during startup:
matrix-server   | Traceback (most recent call last):
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/synapse/handlers/oidc.py", line 148, in load_metadata
matrix-server   |     await p.load_metadata()
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/synapse/handlers/oidc.py", line 610, in load_metadata
matrix-server   |     return await self._provider_metadata.get()
matrix-server   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/synapse/util/caches/cached_call.py", line 143, in get
matrix-server   |     return await self._cachedcall.get()
matrix-server   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/synapse/util/caches/cached_call.py", line 113, in get
matrix-server   |     self._result.raiseException()
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/twisted/python/failure.py", line 505, in raiseException
matrix-server   |     raise self.value.with_traceback(self.tb)
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 1999, in _inlineCallbacks
matrix-server   |     result = context.run(
matrix-server   |              ^^^^^^^^^^^^
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/twisted/python/failure.py", line 519, in throwExceptionIntoGenerator
matrix-server   |     return g.throw(self.value.with_traceback(self.tb))
matrix-server   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/synapse/util/caches/cached_call.py", line 133, in _wrapper
matrix-server   |     return await f()
matrix-server   |            ^^^^^^^^^
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/synapse/handlers/oidc.py", line 622, in _load_metadata
matrix-server   |     metadata_response = await self._http_client.get_json(url)
matrix-server   |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/synapse/http/client.py", line 571, in get_json
matrix-server   |     body = await self.get_raw(uri, args, headers=actual_headers)
matrix-server   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/synapse/http/client.py", line 656, in get_raw
matrix-server   |     response = await self.request("GET", uri, headers=Headers(actual_headers))
matrix-server   |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/synapse/http/client.py", line 423, in request
matrix-server   |     response = await make_deferred_yieldable(request_deferred)
matrix-server   |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
matrix-server   |     current.result = callback(  # type: ignore[misc]
matrix-server   |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/synapse/http/client.py", line 964, in _timeout_to_request_timed_out_error
matrix-server   |     raise RequestTimedOutError("Timeout waiting for response from remote server")
matrix-server   | synapse.http.RequestTimedOutError: 504: Timeout waiting for response from remote server
matrix-server   | 
matrix-server   | The above exception was the direct cause of the following exception:
matrix-server   | 
matrix-server   | Traceback (most recent call last):
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/synapse/app/_base.py", line 259, in wrapper
matrix-server   |     await cb(*args, **kwargs)
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/synapse/app/homeserver.py", line 368, in start
matrix-server   |     await oidc.load_metadata()
matrix-server   |   File "/usr/local/lib/python3.11/site-packages/synapse/handlers/oidc.py", line 152, in load_metadata
matrix-server   |     raise Exception(
matrix-server   | Exception: Error while initialising OIDC provider 'oidc-authentik'

But I can browse to the Element homepage, and then when clicking on "Sign in", I get these log lines:

matrix-element  | 172.18.0.5 - - [30/Aug/2024:13:08:58 +0000] "GET /i18n/languages.241c278.json HTTP/1.1" 200 904 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "172.18.0.1"
matrix-element  | 172.18.0.5 - - [30/Aug/2024:13:08:58 +0000] "GET /_matrix/client/versions HTTP/1.1" 404 555 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "172.18.0.1"
matrix-element  | 2024/08/30 13:08:58 [error] 29#29: *6 open() "/usr/share/nginx/html/_matrix/client/versions" failed (2: No such file or directory), client: 172.18.0.5, server: localhost, request: "GET /_matrix/client/versions HTTP/1.1", host: "matrix.tld"
matrix-element  | 172.18.0.5 - - [30/Aug/2024:13:08:58 +0000] "GET /img/element-icons/i.cf33278.svg HTTP/1.1" 200 516 "https://matrix.tld/bundles/e9035f460125f26b7b0e/theme-light.css" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "172.18.0.1"
matrix-element  | 172.18.0.5 - - [30/Aug/2024:13:08:58 +0000] "GET /_matrix/client/v3/login HTTP/1.1" 404 555 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "172.18.0.1"
matrix-element  | 2024/08/30 13:08:58 [error] 29#29: *6 open() "/usr/share/nginx/html/_matrix/client/v3/login" failed (2: No such file or directory), client: 172.18.0.5, server: localhost, request: "GET /_matrix/client/v3/login HTTP/1.1", host: "matrix.tld"

I have this page:

I don't know why is there a timeout. My OIDC server is using a self-signed certificate, but I've patched the Dockerfile to add the CA to the trusted store.

How could I further debug this ?

Thanks

Steps to reproduce

Setup Authentik as SSO
Configure Synapse and Matrix with the official guides
docker compose up
witness the error when clicking on "Sign in"

Homeserver

self-hosted, not federated

Synapse Version

1.113.0

Installation Method

Docker (matrixdotorg/synapse)

Database

PostgreSQL

Workers

Single process

Platform

Dockerized with docker compose.

Configuration

No response

Relevant log output

matrix-element  | 172.18.0.5 - - [30/Aug/2024:13:08:58 +0000] "GET /i18n/languages.241c278.json HTTP/1.1" 200 904 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "172.18.0.1"
matrix-element  | 172.18.0.5 - - [30/Aug/2024:13:08:58 +0000] "GET /_matrix/client/versions HTTP/1.1" 404 555 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "172.18.0.1"
matrix-element  | 2024/08/30 13:08:58 [error] 29#29: *6 open() "/usr/share/nginx/html/_matrix/client/versions" failed (2: No such file or directory), client: 172.18.0.5, server: localhost, request: "GET /_matrix/client/versions HTTP/1.1", host: "matrix.tld"
matrix-element  | 172.18.0.5 - - [30/Aug/2024:13:08:58 +0000] "GET /img/element-icons/i.cf33278.svg HTTP/1.1" 200 516 "https://matrix.tld/bundles/e9035f460125f26b7b0e/theme-light.css" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "172.18.0.1"
matrix-element  | 172.18.0.5 - - [30/Aug/2024:13:08:58 +0000] "GET /_matrix/client/v3/login HTTP/1.1" 404 555 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "172.18.0.1"
matrix-element  | 2024/08/30 13:08:58 [error] 29#29: *6 open() "/usr/share/nginx/html/_matrix/client/v3/login" failed (2: No such file or directory), client: 172.18.0.5, server: localhost, request: "GET /_matrix/client/v3/login HTTP/1.1", host: "matrix.tld"

Anything else that would be useful to know?

No response

Aug 30 '24 13:08 3isenHeiM

EDIT : I've tried with let's encrypt certificates and the issue is still present.

Sep 05 '24 10:09 3isenHeiM

I've modified the code to avoid raising a RequestTimedOutErrors anytime there is an error to catch, because then it masks the real error.

Commenting the line https://github.com/element-hq/synapse/blob/v1.114.0/synapse/http/client.py#L933

request_deferred.addErrback(_timeout_to_request_timed_out_error)

And re-running the server allowed me to get the true error:

twisted.web._newclient.ResponseNeverReceived: [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', '', 'tlsv1 alert internal error')]>]

Here are the ciphers offered by my IdP server (thanks this comment):

nmap --script ssl-enum-ciphers -p 443 auth.rtss.ninja
Starting Nmap 7.95 ( https://nmap.org ) at 2024-09-06 13:21 CEST
Nmap scan report for auth.rtss.ninja (172.18.0.1)
Host is up (0.00010s latency).

PORT    STATE SERVICE
443/tcp open  https
| ssl-enum-ciphers: 
|   TLSv1.2: 
|     ciphers: 
|       TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
|       TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
|       TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (secp256r1) - A
|     compressors: 
|       NULL
|     cipher preference: client
|   TLSv1.3: 
|     ciphers: 
|       TLS_AKE_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_AKE_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|     cipher preference: server
|_  least strength: A

Nmap done: 1 IP address (1 host up) scanned in 0.37 seconds

Sep 06 '24 12:09 3isenHeiM

Can you confirm the length of the client secret ?

Sep 09 '24 20:09 daedric7

The client secret is 32 hex chars long.

Sep 09 '24 20:09 3isenHeiM

The client secret is 32 hex chars long.

Increase it to at least 128 chars long.

Sep 09 '24 21:09 daedric7

My mistake, it was 64 chars long. I generate them using openssl rand -hex 32.

Will try, thanks for the hint

Sep 10 '24 07:09 3isenHeiM

An EMS customer ran into this when their IdP went down. It would be useful is Synapse would start when unable to connect to the IdP, just with user authentication disabled.

Not sure if it's easily doable, but maybe differentiate between when initially configuring OIDC vs. an OIDC config that has worked for a long time but the IdP is just down temporarily.

Synapse OIDC config:

oidc_config:
    client_auth_method: client_secret_basic
    client_id: redacted
    client_secret: redacted
    discover: true
    enabled: true
    idp_id: oidc
    issuer: https://auth.their-idp.com:8443/...
    scopes:
        ... redacted
    skip_verification: false
    user_mapping_provider:
        config:
            display_name_template: '{{ user.firstName }}'
            localpart_template: '{{ user.name }}'
            subject_template: '{{ user.sub }}'

curl -vi https://auth.their-idp.com:8443

*   Trying ip.address:8443...
* Connected to auth.their-idp.com (ip.address) port 8443 (#0)
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=their-idp.com
*  start date: 
*  expire date: 
*  subjectAltName: host "auth.their-idp.com" matched cert's "*.their-idp.com"
*  issuer: 
*  SSL certificate verify ok.
* using HTTP/2
* h2h3 [:method: GET]
* h2h3 [:path: /]
* h2h3 [:scheme: https]
* h2h3 [:authority: auth.their-idp.com:8443]
* h2h3 [user-agent: curl/7.88.1]
* h2h3 [accept: */*]
* Using Stream ID: 1 (easy handle 0xaaab13dbb600)
> GET / HTTP/2
> Host: auth.their-idp.com:8443
> user-agent: curl/7.88.1
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/2 522 
HTTP/2 522 
< date: Tue, 05 Nov 2024 09:23:35 GMT
date: Tue, 05 Nov 2024 09:23:35 GMT
< content-type: text/plain; charset=UTF-8
content-type: text/plain; charset=UTF-8
< content-length: 15
content-length: 15
< report-to: 
report-to: 
< nel: 
nel: 
< x-frame-options: SAMEORIGIN
x-frame-options: SAMEORIGIN
< referrer-policy: same-origin
referrer-policy: same-origin
< cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< expires: Thu, 01 Jan 1970 00:00:01 GMT
expires: Thu, 01 Jan 1970 00:00:01 GMT
< server: cloudflare
server: cloudflare
< cf-ray: 
cf-ray: 
< alt-svc: h3=":8443"; ma=86400
alt-svc: h3=":8443"; ma=86400
< server-timing: 
server-timing: 

< 
* Connection #0 to host auth.their-idp.com left intact
error code: 522

Their IdP is behind Cloudflare: https://developers.cloudflare.com/support/troubleshooting/cloudflare-errors/troubleshooting-cloudflare-5xx-errors/#error-522-connection-timed-out

Synapse Logs

2024-11-05 09:57:48.552	Exception: Error while initialising OIDC provider 'oidc'
2024-11-05 09:57:48.552	    raise Exception(
2024-11-05 09:57:48.552	  File "/usr/local/lib/python3.11/site-packages/synapse/handlers/oidc.py", line 152, in load_metadata
2024-11-05 09:57:48.552	    await oidc.load_metadata()
2024-11-05 09:57:48.552	  File "/usr/local/lib/python3.11/site-packages/synapse/app/homeserver.py", line 382, in start
2024-11-05 09:57:48.552	    await cb(*args, **kwargs)
2024-11-05 09:57:48.552	  File "/usr/local/lib/python3.11/site-packages/synapse/app/_base.py", line 259, in wrapper
2024-11-05 09:57:48.552	Traceback (most recent call last):
2024-11-05 09:57:48.552	
2024-11-05 09:57:48.552	The above exception was the direct cause of the following exception:
2024-11-05 09:57:48.552	
2024-11-05 09:57:48.552	synapse.api.errors.HttpResponseException: 522: 
2024-11-05 09:57:48.552	    raise HttpResponseException(
2024-11-05 09:57:48.552	  File "/usr/local/lib/python3.11/site-packages/synapse/http/client.py", line 665, in get_raw
2024-11-05 09:57:48.552	           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-05 09:57:48.552	    body = await self.get_raw(uri, args, headers=actual_headers)
2024-11-05 09:57:48.552	  File "/usr/local/lib/python3.11/site-packages/synapse/http/client.py", line 573, in get_json
2024-11-05 09:57:48.552	                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-05 09:57:48.552	    metadata_response = await self._http_client.get_json(url)
2024-11-05 09:57:48.552	  File "/usr/local/lib/python3.11/site-packages/synapse/handlers/oidc.py", line 622, in _load_metadata
2024-11-05 09:57:48.552	           ^^^^^^^^^
2024-11-05 09:57:48.552	    return await f()
2024-11-05 09:57:48.552	  File "/usr/local/lib/python3.11/site-packages/synapse/util/caches/cached_call.py", line 133, in _wrapper
2024-11-05 09:57:48.551	             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-05 09:57:48.551	    result = context.run(gen.send, result)
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2014, in _inlineCallbacks
2024-11-05 09:57:48.551	    raise self.value.with_traceback(self.tb)
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/twisted/python/failure.py", line 535, in raiseException
2024-11-05 09:57:48.551	    self._result.raiseException()
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/util/caches/cached_call.py", line 113, in get
2024-11-05 09:57:48.551	           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-05 09:57:48.551	    return await self._cachedcall.get()
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/util/caches/cached_call.py", line 143, in get
2024-11-05 09:57:48.551	           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-05 09:57:48.551	    return await self._provider_metadata.get()
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/handlers/oidc.py", line 610, in load_metadata
2024-11-05 09:57:48.551	    await p.load_metadata()
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/handlers/oidc.py", line 148, in load_metadata
2024-11-05 09:57:48.551	Traceback (most recent call last):
2024-11-05 09:57:48.551	Error during startup:
2024-11-05 09:57:48.551	Exception: Error while initialising OIDC provider 'oidc'
2024-11-05 09:57:48.551	    raise Exception(
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/handlers/oidc.py", line 152, in load_metadata
2024-11-05 09:57:48.551	    await oidc.load_metadata()
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/app/homeserver.py", line 382, in start
2024-11-05 09:57:48.551	    await cb(*args, **kwargs)
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/app/_base.py", line 259, in wrapper
2024-11-05 09:57:48.551	Traceback (most recent call last):
2024-11-05 09:57:48.551	
2024-11-05 09:57:48.551	The above exception was the direct cause of the following exception:
2024-11-05 09:57:48.551	
2024-11-05 09:57:48.551	synapse.api.errors.HttpResponseException: 522: 
2024-11-05 09:57:48.551	    raise HttpResponseException(
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/http/client.py", line 665, in get_raw
2024-11-05 09:57:48.551	           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-05 09:57:48.551	    body = await self.get_raw(uri, args, headers=actual_headers)
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/http/client.py", line 573, in get_json
2024-11-05 09:57:48.551	                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-05 09:57:48.551	    metadata_response = await self._http_client.get_json(url)
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/handlers/oidc.py", line 622, in _load_metadata
2024-11-05 09:57:48.551	           ^^^^^^^^^
2024-11-05 09:57:48.551	    return await f()
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/util/caches/cached_call.py", line 133, in _wrapper
2024-11-05 09:57:48.551	             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-05 09:57:48.551	    result = context.run(gen.send, result)
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 2014, in _inlineCallbacks
2024-11-05 09:57:48.551	    raise self.value.with_traceback(self.tb)
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/twisted/python/failure.py", line 535, in raiseException
2024-11-05 09:57:48.551	    self._result.raiseException()
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/util/caches/cached_call.py", line 113, in get
2024-11-05 09:57:48.551	           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-05 09:57:48.551	    return await self._cachedcall.get()
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/util/caches/cached_call.py", line 143, in get
2024-11-05 09:57:48.551	           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-05 09:57:48.551	    return await self._provider_metadata.get()
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/handlers/oidc.py", line 610, in load_metadata
2024-11-05 09:57:48.551	    await p.load_metadata()
2024-11-05 09:57:48.551	  File "/usr/local/lib/python3.11/site-packages/synapse/handlers/oidc.py", line 148, in load_metadata
2024-11-05 09:57:48.551	Traceback (most recent call last):
2024-11-05 09:57:48.551	2024-11-05 09:57:48,547 - synapse.app._base - 274 - CRITICAL - sentinel- Error during startup
2024-11-05 09:57:48.548	2024-11-05 09:57:48,547 - synapse.http.client - 428 - INFO - sentinel- Received response to GET https://auth.their-idp.com:8443/.../.well-known/openid-configuration: 522
2024-11-05 09:57:48.548	2024-11-05 09:57:39,265 - synapse.storage.databases.main.event_push_actions - 1770 - INFO - rotate_notifs-0- Rotating notifications, deleted 0 push actions

Nov 05 '24 10:11 Twi1ightSparkle

Same error but in another scenario: when oidc provider is not available, synapse can't start. Why access to oidc provider is hard dependency in synapse? Maybe possible to add some parameter to ignore it? There are many cases it useful - for example oidc provider in maintenance mode, but users still can login by login and pass.

Dec 10 '24 19:12 vaminakov

I had this issue,

I had used the use_insecure_ssl_client_just_for_testing_do_not_use option as a workaround for another previous issue. i tried using metadata_url: unsuccesfully

In the end i commented out: discover: metadata_url: use_insecure_ssl_client_just_for_testing_do_not_use:

and the issue went away

Jul 31 '25 05:07 domwolfy

Error at OIDC initialization - restart in loop

Description

Steps to reproduce

Homeserver

Synapse Version

Installation Method

Database

Workers

Platform

Configuration

Relevant log output

Anything else that would be useful to know?