postgresql is required on macOS Monterey

[Johennes]

Description

On macOS Monterey postgresql is required in order to install Synapse.

Steps to reproduce

Run poetry install --extras all and notice how it fails.

$ poetry install --extras all
Installing dependencies from lock file

Package operations: 1 install, 0 updates, 0 removals

  • Installing psycopg2 (2.9.3): Failed

  EnvCommandError

  Command ['/Users/jm/Library/Caches/pypoetry/virtualenvs/matrix-synapse-9UQhfgjM-py3.8/bin/pip', 'install', '--no-deps', 'file:///Users/jm/Library/Caches/pypoetry/artifacts/3c/7a/60/e4a0d26c4cee1923be3b7ae2cb545f115a822e112a6c5b5434bd25eacd/psycopg2-2.9.3.tar.gz'] errored with the following return code 1, and output:
  Processing /Users/jm/Library/Caches/pypoetry/artifacts/3c/7a/60/e4a0d26c4cee1923be3b7ae2cb545f115a822e112a6c5b5434bd25eacd/psycopg2-2.9.3.tar.gz
    Preparing metadata (setup.py): started
    Preparing metadata (setup.py): finished with status 'error'
    error: subprocess-exited-with-error

    × python setup.py egg_info did not run successfully.
    │ exit code: 1
    ╰─> [25 lines of output]
        running egg_info
        creating /private/var/folders/zv/cnks6dx14rl25kvgqslhpv740000gn/T/pip-pip-egg-info-qrjfl19z/psycopg2.egg-info
        writing /private/var/folders/zv/cnks6dx14rl25kvgqslhpv740000gn/T/pip-pip-egg-info-qrjfl19z/psycopg2.egg-info/PKG-INFO
        writing dependency_links to /private/var/folders/zv/cnks6dx14rl25kvgqslhpv740000gn/T/pip-pip-egg-info-qrjfl19z/psycopg2.egg-info/dependency_links.txt
        writing top-level names to /private/var/folders/zv/cnks6dx14rl25kvgqslhpv740000gn/T/pip-pip-egg-info-qrjfl19z/psycopg2.egg-info/top_level.txt
        writing manifest file '/private/var/folders/zv/cnks6dx14rl25kvgqslhpv740000gn/T/pip-pip-egg-info-qrjfl19z/psycopg2.egg-info/SOURCES.txt'
        /Users/jm/Library/Caches/pypoetry/virtualenvs/matrix-synapse-9UQhfgjM-py3.8/lib/python3.8/site-packages/setuptools/config/setupcfg.py:463: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
          warnings.warn(msg, warning_class)

        Error: pg_config executable not found.

        pg_config is required to build psycopg2 from source.  Please add the directory
        containing pg_config to the $PATH or specify the full executable path with the
        option:

            python setup.py build_ext --pg-config /path/to/pg_config build ...

        or with the pg_config option in 'setup.cfg'.

        If you prefer to avoid building psycopg2 from source, please install the PyPI
        'psycopg2-binary' package instead.

        For further information please check the 'doc/src/install.rst' file (also at
        <https://www.psycopg.org/docs/install.html>).

        [end of output]

        ...

After running brew install postgresql it succeeds.

I think we should add this in the platform-specific installation instructions for macOS: https://matrix-org.github.io/synapse/latest/setup/installation.html#macos

Homeserver

n/a

Synapse Version

Current develop

Installation Method

n/a

Platform

macOS Monterey

Relevant log output

n/a

Anything else that would be useful to know?

n/a

[clokep]

We should probably be requiring or allowing psycopg2-binary instead. That should allow you to install it without compiling anything.

What version of Python is this? Is this an M1/M2 Mac or an Intel chipset?

[Johennes]

Sorry, this is Python 3.8.9 on an M1.

[clokep]

Sorry, this is Python 3.8.9 on an M1.

Unfortunately I don't think using psycopg2-binary would help here -- it seems to only have wheels for Intel Macs. 😢

[richvdh]

on Linux, you only need the development libraries (hence the recommendations to install libpq-devel in some sections). Is there nothing similar in the macOS world?

[richvdh]

We should probably be requiring or allowing psycopg2-binary instead.

This seems to be a mess - cf https://www.psycopg.org/articles/2018/02/08/psycopg-274-released/ and https://github.com/psycopg/psycopg2/issues/674. To cut a long story short: use of psycopg2-binary seems to be risky. Previously, allowing an either/or situation was tricky, but maybe that has now changed?

Unfortunately I don't think using psycopg2-binary would help here -- it seems to only have wheels for Intel Macs. :cry:

This seems to be https://github.com/psycopg/psycopg2/issues/1482

[Johennes]

on Linux, you only need the development libraries (hence the recommendations to install libpq-devel in some sections). Is there nothing similar in the macOS world?

Hm, homebrew has libpq which appears to contain the same headers that the postgres formula includes. So I assume that would have been sufficient as well.