Update to 1.9.0 breaks authentication via LDAP (ma1sd / rest client)

[jacotec]

Not sure whether this is a synapse issue or a ma1sd issue, so I cross-post this here as well:

Description

After updating synapse to 1.9.0 no user can login. Users are stored in a LDAP database, authentication via REST-API / ma1sd.

Steps to reproduce

Open Riot Web Log in with any existing user Error message: Fehler: Problem bei der Kommunikation mit dem angegebenen Home-Server. (M_UNKNOWN)

Version information

Synapse 1.9.0 installed via official repo. Ubuntu 18.04 LTS

Log file of the issue:

2020-01-23 14:09:46,680 - synapse.access.https.8448 - 233 - INFO - POST-44 - 87.79.201.63 - 8448 - Received request: POST /_matrix/client/r0/login
2020-01-23 14:09:46,681 - synapse.rest.client.v1.login - 176 - INFO - POST-44 - Got login request with identifier: {'type': 'm.id.user', 'user': 'admin'}, medium: None, address: None, user: None
2020-01-23 14:09:46,681 - rest_auth_provider - 46 - INFO - POST-44 - Got password check for @admin:jaychat.de
2020-01-23 14:09:46,695 - rest_auth_provider - 62 - INFO - POST-44 - User @admin:jaychat.de authenticated
2020-01-23 14:09:46,699 - rest_auth_provider - 76 - INFO - POST-44 - User @admin:jaychat.de already exists, registration skipped
2020-01-23 14:09:46,699 - rest_auth_provider - 79 - INFO - POST-44 - Handling profile data
2020-01-23 14:09:46,700 - synapse.http.server - 110 - ERROR - POST-44 - Failed handle request via 'LoginRestServlet': <XForwardedForRequest at 0x7f18edeedda0 method='POST' uri='/_matrix/client/r0/login' clientproto='HTTP/1.0' site=8448>
Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
StopIteration: {}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
StopIteration: {}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
StopIteration: ('@admin:jaychat.de', '$2b$12$YEY5SVN7Yy43yFq3f6saLuAR/uaNK5m4Res6hgnLLc/70KODPt7ZW')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
StopIteration: @admin:jaychat.de

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.6/site-packages/synapse/http/server.py", line 78, in wrapped_request_handler
    await h(self, request)
  File "/opt/venvs/matrix-synapse/lib/python3.6/site-packages/synapse/http/server.py", line 331, in _async_render
    callback_return = await callback_return
  File "/opt/venvs/matrix-synapse/lib/python3.6/site-packages/synapse/rest/client/v1/login.py", line 150, in on_POST
    result = await self._do_other_login(login_submission)
  File "/opt/venvs/matrix-synapse/lib/python3.6/site-packages/synapse/rest/client/v1/login.py", line 281, in _do_other_login
    identifier["user"], login_submission
  File "/opt/venvs/matrix-synapse/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/opt/venvs/matrix-synapse/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.6/site-packages/synapse/handlers/auth.py", line 622, in validate_login
    is_valid = yield provider.check_password(qualified_user_id, password)
  File "/opt/venvs/matrix-synapse/lib/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/opt/venvs/matrix-synapse/lib/python3.6/site-packages/rest_auth_provider.py", line 82, in check_password
    store = yield self.account_handler.hs.get_profile_handler().store
AttributeError: 'ModuleApi' object has no attribute 'hs'

[MurzNN]

Same problem, here is Synapse issue about this problem: https://github.com/matrix-org/synapse/issues/6772

[jacotec]

@ma1uta Seems that needs a final fixing in ma1sd: https://github.com/matrix-org/synapse/issues/6772#issuecomment-577723490

[ma1uta]

I have pushed to the master the temporary fix of the rest_password_provider. Also I am going to create PR to add missing methods to the public API.

[anoadragon453]

@ma1uta Were these ever done?

[ma1uta]

Maybe. I don't well with python and too busy. PR welcome.

[anoadragon453]

@ma1uta I can take a look. What do you need exactly? Where is the python code?

[ma1uta]

@anoadragon453 thanks for help. The problem code: https://github.com/ma1uta/matrix-synapse-rest-password-provider/blob/master/rest_auth_provider.py#L86 One of the cases of the matrix-synapse-rest-password-provider:

1. a new user is trying to log in
2. synapse delegates passwords check to the matrix-synapse-rest-password-provider
3. rest-provider sends a request to the ma1sd
4. ma1sd finds login/password in an one of the storages
5. ma1sd returns profile of a new user
6. rest-provider registers a new user if necessary and update the profile

What wrong with that? I don't think that register and update profile a new user on the password checks is a good idea, but synapse doesn't have another module or user/profile/... providers to implement this feature.

Currently rest-provider uses a protected profile handler. I am not so good with the synapse code and cannot find the best alternative.

[anoadragon453]

So, it looks like the two features of matrix-synapse-rest-password-provider that use the datastore here are:

• Setting the displayname of the user on login/register
• Updating/replacing the current set of 3PIDs for the user upon login/registration

The only relevant things the current module API provides is:

• Setting displayname on register (not login)
• Setting initial emails on register (not login) (not phone numbers)

So at this time, the features you would need from the ModuleApi to stop reaching into the datastore would be:

• Ability to set user displayname
• Ability to set the user's threepids (and an option to erase existing threepids)

I've added an issue for adding this functionality here: https://github.com/matrix-org/synapse/issues/7734 (you could continue to use the database-reaching functionality for older versions of Synapse).

While browsing through the module's code... I realized it could really use some cleaning up. I may submit a PR for that at some point :slightly_smiling_face:

Do you know what the minimum Synapse version you'd like to support is? And do you still want https://github.com/ma1uta/matrix-synapse-rest-password-provider to support Python 2?

[ma1uta]

Thanks a lot for help!

Do you know what the minimum Synapse version you'd like to support is? And do you still want https://github.com/ma1uta/matrix-synapse-rest-password-provider to support Python 2?

I think to support only Synapse versions which officially supported. Since Synapse support only Python 3.5-3.8 https://github.com/matrix-org/synapse/blob/develop/INSTALL.md#installing-from-source we can drop Python 2 support.