`Database` constructor fails with unix sockets

[jtracey]

When trying to launch maubot, I get the following exception: raise ValueError("Invalid URL: host is required for absolute urls")

This originates from yarl, called via mautrix's Database constructor: https://github.com/mautrix/python/blob/735d0c139e48777cbd2385ed15fdb549113914c4/mautrix/util/async_db/database.py#L66

If you consult Postgres' documentation, you'll see in that the netloc portion of the URI is optional:

postgresql://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]

In fact, not only is it optional, omitting it is the only way to indicate connection to a Unix socket:

In particular, a Unix-domain socket connection is chosen if the host part is either empty or starts with a slash, otherwise a TCP/IP connection is initiated.

This is fairly important for security purposes, as Unix sockets are the standard way to ensure a local database can never be remotely accessed. I don't think urllib has this problem, and the check is fairly minimal and explicit, so I suspect it's a design decision on yarl's part to parse URLs, not arbitrary URIs.

For now, users can be work around this by explicitly encoding the default Unix socket path, using percent escapes to avoid yarl parsing the path as the URI hierarchy, e.g.: postgresql://username:secret@%2Fvar%2Frun%2Fpostgresql/dbname, but this really should be fixed.

[tulir]

The usual way is postgres:///dbname, but escaped host or using the ?host parameter work too, so there isn't really any reason to fix this.

[cyberb]

this does not work either:

postgresql://matrix:matrix@/telegram?host=/var/snap/matrix/current/database&port=5436

error:

ValueError: Invalid URL: host is required for absolute urls

[tulir]

That's the same as in the original issue. URL-encoded host in the netloc field or specifying user/password via query params works

[cyberb]

this still does not work:

postgresql://matrix:password-redacted@%2Fvar%2Fsnap%2Fmatrix%2Fcurrent%2Fdatabase/telegram?port=5436

produces:

Traceback (most recent call last):
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/site-packages/mautrix/bridge/bridge.py", line 215, in start_db
    await self.db.start()
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/site-packages/mautrix/util/async_db/asyncpg.py", line 71, in start
    self._pool = await asyncpg.create_pool(str(self.url), **self._db_args)
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/site-packages/asyncpg/pool.py", line 409, in _async__init__
    await self._initialize()
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/site-packages/asyncpg/pool.py", line 437, in _initialize
    await first_ch.connect()
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/site-packages/asyncpg/pool.py", line 129, in connect
    self._con = await self._pool._get_new_connection()
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/site-packages/asyncpg/pool.py", line 507, in _get_new_connection
    con = await connection.connect(
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/site-packages/asyncpg/connection.py", line 2092, in connect
    return await connect_utils._connect(
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 895, in _connect
    raise last_error
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 881, in _connect
    return await _connect_addr(
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 768, in _connect_addr
    return await __connect_addr(params, timeout, False, *args)
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 825, in __connect_addr
    tr, pr = await compat.wait_for(connector, timeout=timeout)
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/site-packages/asyncpg/compat.py", line 56, in wait_for
    return await asyncio.wait_for(fut, timeout)
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/asyncio/tasks.py", line 494, in wait_for
    return fut.result()
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/asyncio/unix_events.py", line 244, in create_unix_connection
    await self.sock_connect(sock, path)
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/asyncio/selector_events.py", line 496, in sock_connect
    return await fut
  File "/snap/matrix/x2/python/usr/local/lib/python3.8/asyncio/selector_events.py", line 501, in _sock_connect
    sock.connect(address)
FileNotFoundError: [Errno 2] No such file or directory

while this is ok:

postgresql:///telegram?host=/var/snap/matrix/current/database&user=matrix&password=matrix&port=5436