synapse icon indicating copy to clipboard operation
synapse copied to clipboard
verified publisher icon element-hq

Synapse 1.111.0 in Worker mode does not register new authenticated media endpoints

Open AlexanderBabel opened this issue 1 year ago • 3 comments

Description

I use a workers configuration for my synapse with a media worker. I already updated my ingress to forward the new authenticated endpoints to the media worker instance. But all of them are returned with 404.

It looks like that the new endpoints are not registered on startup on the media server

Steps to reproduce

  • Use the 1.111.0 synapse version
  • Try to upload an image through the new endpoints/Retrieve images through the new endpoint

Homeserver

babel1.eu

Synapse Version

v1.111.0

Installation Method

Docker (matrixdotorg/synapse)

Database

postgres

Workers

Multiple workers

Platform

Ubuntu 22.04, Kubernetes

Configuration

general-config:
  enable_media_repo: false

media-worker-config:
  worker_app: "synapse.app.media_repository"
  
  worker_listeners:
    - port: 9090
      tls: false
      bind_addresses: ["::"]
      type: http
  
      resources:
        - names: [metrics]
          compress: false
  
    - port: 8083
      tls: false
      bind_addresses: ["::"]
      type: http
      x_forwarded: true
  
      resources:
        - names:
            - media
          compress: false
  
  worker_log_config: /synapse/config/log.yaml

Relevant log output

2024-07-23 13:52:49,139 - twisted - 282 - INFO - sentinel- SynapseSite starting on 9090
2024-07-23 13:52:49,139 - synapse.app._base - 439 - INFO - sentinel- Synapse now listening on TCP port 9090
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for POST /_matrix/media/v1/create
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for POST /_matrix/media/(r0|v3|v1)/upload$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for PUT /_matrix/media/v3/upload/(?P<server_name>[^/]*)/(?P<media_id>[^/]*)$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for GET /_matrix/media/(r0|v3|v1)/download/(?P<server_name>[^/]*)/(?P<media_id>[^/]*)(/(?P<file_name>[^/]*))?$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for GET /_matrix/media/(r0|v3|v1)/thumbnail/(?P<server_name>[^/]*)/(?P<media_id>[^/]*)$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for GET /_matrix/media/(r0|v3|v1)/preview_url$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for GET /_matrix/media/(r0|v3|v1)/config$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for POST ^/_synapse/admin/v1/purge_media_cache$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for POST ^/_synapse/admin/v1/room/(?P<room_id>[^/]*)/media/quarantine$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for POST ^/_synapse/admin/v1/quarantine_media/(?P<room_id>[^/]*)$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for POST ^/_synapse/admin/v1/media/quarantine/(?P<server_name>[^/]*)/(?P<media_id>[^/]*)$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for POST ^/_synapse/admin/v1/media/unquarantine/(?P<server_name>[^/]*)/(?P<media_id>[^/]*)$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for POST ^/_synapse/admin/v1/user/(?P<user_id>[^/]*)/media/quarantine$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for POST ^/_synapse/admin/v1/media/protect/(?P<media_id>[^/]*)$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for POST ^/_synapse/admin/v1/media/unprotect/(?P<media_id>[^/]*)$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for GET ^/_synapse/admin/v1/room/(?P<room_id>[^/]*)/media$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for POST ^/_synapse/admin/v1/media/delete$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for POST ^/_synapse/admin/v1/media/(?P<server_name>[^/]*)/delete$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for DELETE ^/_synapse/admin/v1/media/(?P<server_name>[^/]*)/(?P<media_id>[^/]*)$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for GET ^/_synapse/admin/v1/users/(?P<user_id>[^/]*)/media$
2024-07-23 13:52:49,140 - synapse.http.server - 485 - DEBUG - sentinel- Registering for DELETE ^/_synapse/admin/v1/users/(?P<user_id>[^/]*)/media$
2024-07-23 13:52:49,140 - synapse.util.httpresourcetree - 56 - INFO - sentinel- Attaching <synapse.rest.health.HealthResource object at 0xe63816596b90> to path b'/health'
2024-07-23 13:52:49,141 - synapse.util.httpresourcetree - 56 - INFO - sentinel- Attaching <synapse.rest.media.media_repository_resource.MediaRepositoryResource object at 0xe638165c30d0> to path b'/_matrix/media/r0'
2024-07-23 13:52:49,141 - synapse.util.httpresourcetree - 56 - INFO - sentinel- Attaching <synapse.rest.media.media_repository_resource.MediaRepositoryResource object at 0xe638165c30d0> to path b'/_matrix/media/v3'
2024-07-23 13:52:49,141 - synapse.util.httpresourcetree - 56 - INFO - sentinel- Attaching <synapse.rest.media.media_repository_resource.MediaRepositoryResource object at 0xe638165c30d0> to path b'/_matrix/media/v1'
2024-07-23 13:52:49,141 - synapse.util.httpresourcetree - 56 - INFO - sentinel- Attaching <synapse.http.server.JsonResource object at 0xe63816368250> to path b'/_synapse/admin'
2024-07-23 13:52:49,141 - twisted - 282 - INFO - sentinel- SynapseSite starting on 8083
2024-07-23 13:52:49,141 - synapse.app._base - 439 - INFO - sentinel- Synapse now listening on TCP port 8083


2024-07-23 13:57:27,646 - synapse.access.http.8083 - 428 - DEBUG - GET-56- ::ffff: - 8083 - Received request: GET /_matrix/client/v1/media/download/babel1.eu/OHagbmtHsZRLwhrdDCGxRBOQ?allow_redirect=true
2024-07-23 13:57:27,647 - synapse.http.server - 130 - INFO - GET-56- <XForwardedForRequest at 0xe638160dc7d0 method='GET' uri='/_matrix/client/v1/media/download/babel1.eu/OHagbmtHsZRLwhrdDCGxRBOQ?allow_redirect=true' clientproto='HTTP/1.1' site='8083'> SynapseError: 404 - Unrecognized request
2024-07-23 13:57:27,648 - synapse.access.http.8083 - 473 - INFO - GET-56- ::ffff: - 8083 - {None} Processed request: 0.001sec/-0.000sec (0.001sec, 0.000sec) (0.000sec/0.000sec/0) 59B 404 "GET /_matrix/client/v1/media/download/babel1.eu/OHagbmtHsZRLwhrdDCGxRBOQ?allow_redirect=true HTTP/1.1" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) ElementNightly/2024072201 Chrome/126.0.6478.127 Electron/31.2.0 Safari/537.36" [0 dbevts]
2024-07-23 13:57:27,655 - synapse.replication.tcp.handler - 592 - DEBUG - process-replication-data-507- Received rdata presence (master) -> 78074332

Anything else that would be useful to know?

No response

AlexanderBabel avatar Jul 23 '24 13:07 AlexanderBabel

The issue is that client and federation are not listed in resources, but I'm not sure if requiring them for the new endpoints is intentional or not. If it's intentional, it should probably be highlighted in release notes. If not, then it's a bug.

tulir avatar Jul 23 '24 14:07 tulir

Agreed. Either we should:

  • update the docs to mention adding client and federation resources during worker configuration ...or
  • have the media resource also start the relevant client/federation servlets for the new endpoints

devonh avatar Jul 23 '24 23:07 devonh

I think the preferred approach going forward should be to ensure that when specifying the media resource, all appropriate handlers are started in that situation.

devonh avatar Jul 24 '24 15:07 devonh

It might be nice to add a link to this to the upgrade notes as well. As of right now, this is really confusing because it still does not work, even after modify the reverse proxy config as instructed, if you are deploying with a seperate media worker. (https://element-hq.github.io/synapse/latest/upgrade)

4censord avatar Sep 03 '24 10:09 4censord

I don't think there's any point in adding upgrade notes for a bug that was already fixed and released 🤔

@devonh the issue can probably be closed now?

tulir avatar Sep 03 '24 11:09 tulir

Ah, then I seem to have overlooked it in the release notes.

4censord avatar Sep 03 '24 14:09 4censord

Just to clarify, in case someone else is confused with this: As of 1.114, this fix is included. 1.113 does not have it yet.

4censord avatar Sep 13 '24 13:09 4censord