django.db.utils.OperationalError: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory

[karambaq]

I've docker-compose configuration for django and postgres, it works fine. However, when I'm trying to run pytest inside a django container it fails with an error:

pytest apps/service/tests/test_api.py::TestCreate::test_new
====================================================================== test session starts =======================================================================
platform linux -- Python 3.8.18, pytest-7.4.1, pluggy-1.3.0
django: settings: project.settings.local (from env)
rootdir: /app/code
configfile: pytest.ini
plugins: mock-3.11.1, django-4.5.2, Faker-19.6.1, celery-4.4.2
collected 1 item

apps/service/tests/test_api.py E                                                                                                                           [100%]

============================================================================= ERRORS =============================================================================
__________________________________________________ ERROR at setup of TestCreate.test_new __________________________________________________

self = <django.db.backends.postgresql.base.DatabaseWrapper object at 0xffff71cedf70>

    @async_unsafe
    def ensure_connection(self):
        """Guarantee that a connection to the database is established."""
        if self.connection is None:
            with self.wrap_database_errors:
>               self.connect()

/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:219:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py:33: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:200: in connect
    self.connection = self.get_new_connection(conn_params)
/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py:33: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py:187: in get_new_connection
    connection = Database.connect(**conn_params)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

dsn = 'dbname=postgres', connection_factory = None, cursor_factory = None, kwargs = {'database': 'postgres'}, kwasync = {}

    def connect(dsn=None, connection_factory=None, cursor_factory=None, **kwargs):
        """
        Create a new database connection.

        The connection parameters can be specified as a string:

            conn = psycopg2.connect("dbname=test user=postgres password=secret")

        or using a set of keyword arguments:

            conn = psycopg2.connect(database="test", user="postgres", password="secret")

        Or as a mix of both. The basic connection parameters are:

        - *dbname*: the database name
        - *database*: the database name (only as keyword argument)
        - *user*: user name used to authenticate
        - *password*: password used to authenticate
        - *host*: database host address (defaults to UNIX socket if not provided)
        - *port*: connection port number (defaults to 5432 if not provided)

        Using the *connection_factory* parameter a different class or connections
        factory can be specified. It should be a callable object taking a dsn
        argument.

        Using the *cursor_factory* parameter, a new default cursor factory will be
        used by cursor().

        Using *async*=True an asynchronous connection will be created. *async_* is
        a valid alias (for Python versions where ``async`` is a keyword).

        Any other keyword parameter will be passed to the underlying client
        library: the list of supported parameters depends on the library version.

        """
        kwasync = {}
        if 'async' in kwargs:
            kwasync['async'] = kwargs.pop('async')
        if 'async_' in kwargs:
            kwasync['async_'] = kwargs.pop('async_')

        if dsn is None and not kwargs:
            raise TypeError('missing dsn and no parameters')

        dsn = _ext.make_dsn(dsn, **kwargs)
>       conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
E       psycopg2.OperationalError: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory
E       	Is the server running locally and accepting connections on that socket?

/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py:127: OperationalError

The above exception was the direct cause of the following exception:

self = <django.db.backends.postgresql.base.DatabaseWrapper object at 0xffff7f398d30>

    @contextmanager
    def _nodb_cursor(self):
        try:
>           with super()._nodb_cursor() as cursor:

/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py:301:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.8/contextlib.py:113: in __enter__
    return next(self.gen)
/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:620: in _nodb_cursor
    with conn.cursor() as cursor:
/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py:33: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:259: in cursor
    return self._cursor()
/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:235: in _cursor
    self.ensure_connection()
/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py:33: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:219: in ensure_connection
    self.connect()
/usr/local/lib/python3.8/site-packages/django/db/utils.py:90: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:219: in ensure_connection
    self.connect()
/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py:33: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:200: in connect
    self.connection = self.get_new_connection(conn_params)
/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py:33: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py:187: in get_new_connection
    connection = Database.connect(**conn_params)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

dsn = 'dbname=postgres', connection_factory = None, cursor_factory = None, kwargs = {'database': 'postgres'}, kwasync = {}

    def connect(dsn=None, connection_factory=None, cursor_factory=None, **kwargs):
        """
        Create a new database connection.

        The connection parameters can be specified as a string:

            conn = psycopg2.connect("dbname=test user=postgres password=secret")

        or using a set of keyword arguments:

            conn = psycopg2.connect(database="test", user="postgres", password="secret")

        Or as a mix of both. The basic connection parameters are:

        - *dbname*: the database name
        - *database*: the database name (only as keyword argument)
        - *user*: user name used to authenticate
        - *password*: password used to authenticate
        - *host*: database host address (defaults to UNIX socket if not provided)
        - *port*: connection port number (defaults to 5432 if not provided)

        Using the *connection_factory* parameter a different class or connections
        factory can be specified. It should be a callable object taking a dsn
        argument.

        Using the *cursor_factory* parameter, a new default cursor factory will be
        used by cursor().

        Using *async*=True an asynchronous connection will be created. *async_* is
        a valid alias (for Python versions where ``async`` is a keyword).

        Any other keyword parameter will be passed to the underlying client
        library: the list of supported parameters depends on the library version.

        """
        kwasync = {}
        if 'async' in kwargs:
            kwasync['async'] = kwargs.pop('async')
        if 'async_' in kwargs:
            kwasync['async_'] = kwargs.pop('async_')

        if dsn is None and not kwargs:
            raise TypeError('missing dsn and no parameters')

        dsn = _ext.make_dsn(dsn, **kwargs)
>       conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
E       django.db.utils.OperationalError: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory
E       	Is the server running locally and accepting connections on that socket?

/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py:127: OperationalError

During handling of the above exception, another exception occurred:

self = <django.db.backends.postgresql.base.DatabaseWrapper object at 0xffff71593ca0>

    @async_unsafe
    def ensure_connection(self):
        """Guarantee that a connection to the database is established."""
        if self.connection is None:
            with self.wrap_database_errors:
>               self.connect()

/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:219:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py:33: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:200: in connect
    self.connection = self.get_new_connection(conn_params)
/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py:33: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py:187: in get_new_connection
    connection = Database.connect(**conn_params)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

dsn = 'dbname=test_project', connection_factory = None, cursor_factory = None, kwargs = {'database': 'test_project'}, kwasync = {}

    def connect(dsn=None, connection_factory=None, cursor_factory=None, **kwargs):
        """
        Create a new database connection.

        The connection parameters can be specified as a string:

            conn = psycopg2.connect("dbname=test user=postgres password=secret")

        or using a set of keyword arguments:

            conn = psycopg2.connect(database="test", user="postgres", password="secret")

        Or as a mix of both. The basic connection parameters are:

        - *dbname*: the database name
        - *database*: the database name (only as keyword argument)
        - *user*: user name used to authenticate
        - *password*: password used to authenticate
        - *host*: database host address (defaults to UNIX socket if not provided)
        - *port*: connection port number (defaults to 5432 if not provided)

        Using the *connection_factory* parameter a different class or connections
        factory can be specified. It should be a callable object taking a dsn
        argument.

        Using the *cursor_factory* parameter, a new default cursor factory will be
        used by cursor().

        Using *async*=True an asynchronous connection will be created. *async_* is
        a valid alias (for Python versions where ``async`` is a keyword).

        Any other keyword parameter will be passed to the underlying client
        library: the list of supported parameters depends on the library version.

        """
        kwasync = {}
        if 'async' in kwargs:
            kwasync['async'] = kwargs.pop('async')
        if 'async_' in kwargs:
            kwasync['async_'] = kwargs.pop('async_')

        if dsn is None and not kwargs:
            raise TypeError('missing dsn and no parameters')

        dsn = _ext.make_dsn(dsn, **kwargs)
>       conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
E       psycopg2.OperationalError: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory
E       	Is the server running locally and accepting connections on that socket?

/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py:127: OperationalError

The above exception was the direct cause of the following exception:

request = <SubRequest '_django_db_marker' for <Function test_new>>

    @pytest.fixture(autouse=True)
    def _django_db_marker(request) -> None:
        """Implement the django_db marker, internal to pytest-django."""
        marker = request.node.get_closest_marker("django_db")
        if marker:
>           request.getfixturevalue("_django_db_helper")

/usr/local/lib/python3.8/site-packages/pytest_django/plugin.py:465:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.8/site-packages/pytest_django/fixtures.py:122: in django_db_setup
    db_cfg = setup_databases(
/usr/local/lib/python3.8/site-packages/django/test/utils.py:179: in setup_databases
    connection.creation.create_test_db(
/usr/local/lib/python3.8/site-packages/django/db/backends/base/creation.py:57: in create_test_db
    self._create_test_db(verbosity, autoclobber, keepdb)
/usr/local/lib/python3.8/site-packages/django/db/backends/base/creation.py:191: in _create_test_db
    with self._nodb_cursor() as cursor:
/usr/local/lib/python3.8/contextlib.py:113: in __enter__
    return next(self.gen)
/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py:319: in _nodb_cursor
    with conn.cursor() as cursor:
/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py:33: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:259: in cursor
    return self._cursor()
/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:235: in _cursor
    self.ensure_connection()
/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py:33: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:219: in ensure_connection
    self.connect()
/usr/local/lib/python3.8/site-packages/django/db/utils.py:90: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:219: in ensure_connection
    self.connect()
/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py:33: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py:200: in connect
    self.connection = self.get_new_connection(conn_params)
/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py:33: in inner
    return func(*args, **kwargs)
/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py:187: in get_new_connection
    connection = Database.connect(**conn_params)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

dsn = 'dbname=test_project', connection_factory = None, cursor_factory = None, kwargs = {'database': 'test_project'}, kwasync = {}

    def connect(dsn=None, connection_factory=None, cursor_factory=None, **kwargs):
        """
        Create a new database connection.

        The connection parameters can be specified as a string:

            conn = psycopg2.connect("dbname=test user=postgres password=secret")

        or using a set of keyword arguments:

            conn = psycopg2.connect(database="test", user="postgres", password="secret")

        Or as a mix of both. The basic connection parameters are:

        - *dbname*: the database name
        - *database*: the database name (only as keyword argument)
        - *user*: user name used to authenticate
        - *password*: password used to authenticate
        - *host*: database host address (defaults to UNIX socket if not provided)
        - *port*: connection port number (defaults to 5432 if not provided)

        Using the *connection_factory* parameter a different class or connections
        factory can be specified. It should be a callable object taking a dsn
        argument.

        Using the *cursor_factory* parameter, a new default cursor factory will be
        used by cursor().

        Using *async*=True an asynchronous connection will be created. *async_* is
        a valid alias (for Python versions where ``async`` is a keyword).

        Any other keyword parameter will be passed to the underlying client
        library: the list of supported parameters depends on the library version.

        """
        kwasync = {}
        if 'async' in kwargs:
            kwasync['async'] = kwargs.pop('async')
        if 'async_' in kwargs:
            kwasync['async_'] = kwargs.pop('async_')

        if dsn is None and not kwargs:
            raise TypeError('missing dsn and no parameters')

        dsn = _ext.make_dsn(dsn, **kwargs)
>       conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
E       django.db.utils.OperationalError: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory
E       	Is the server running locally and accepting connections on that socket?

/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py:127: OperationalError
--------------------------------------------------------------------- Captured stderr setup ----------------------------------------------------------------------


/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py:304: RuntimeWarning: Normally Django will use a connection to the 'postgres' database to avoid running initialization queries against the production database when it's not needed (for example, when running tests). Django was unable to create a connection to the 'postgres' database and will use the first PostgreSQL database instead.
  warnings.warn(
==================================================================== short test summary info =====================================================================
ERROR apps/service/tests/test_api.py::TestCreate::test_new - django.db.utils.OperationalError: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory
======================================================================== 1 error in 5.25s ========================================================================

This is actually true, there is no such file in django container, it's inside the psql container, but I don't understand what to do with this error. I checked that test database created.

I can fix this error if add this fixture in conftest.py

import pytest

@pytest.fixture()
def django_db_setup():
    pass

But using this approach it using my actual database, not the "test_*" one.

I don't think that this is docker-compose problem because other than tests it works fine.

[karambaq]

I found the solution, but not sure that if this is the right one.

DATABASES = {
    'default': {
        'ENGINE': 'django.contrib.gis.db.backends.postgis',
        'NAME': os.environ.get('DB_NAME'),
        'USER': os.environ.get('DB_USERNAME'),
        'PASSWORD': os.environ.get('DB_PASSWORD'),
        'HOST': os.environ.get('DB_HOSTNAME'),
        'PORT': os.environ.get('DB_PORT'),
    },
    LOG_DB: {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ.get('LOG_DB_NAME'),
        'USER': os.environ.get('LOG_DB_USERNAME'),
        'PASSWORD': os.environ.get('LOG_DB_PASSWORD'),
        'HOST': os.environ.get('LOG_DB_HOSTNAME'),
        'PORT': os.environ.get('LOG_DB_PORT'),
    }
}

In my settings I had two databases in DATABASES dict, after I comment out the second one, all works as expected. But how can I specify to use only first database for pytest?