pytest-django icon indicating copy to clipboard operation
pytest-django copied to clipboard

Circular dependency in TEST[DEPENDENCIES] arises after upgrade from 4.10.0 => 4.11.0

Open micahjsmith opened this issue 5 months ago • 1 comments

When I upgrade from 4.10.0 => 4.11.0, the following error starts occurring

_______________________ ERROR at setup of test_foo ________________________

request = <SubRequest '_django_db_marker' for <Function test_foo>>

    @pytest.fixture(autouse=True)
    def _django_db_marker(request: pytest.FixtureRequest) -> None:
        """Implement the django_db marker, internal to pytest-django."""
        marker = request.node.get_closest_marker("django_db")
        if marker:
>           request.getfixturevalue("_django_db_helper")

.venv/lib/python3.11/site-packages/pytest_django/plugin.py:552: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.venv/lib/python3.11/site-packages/pytest_django/fixtures.py:198: in django_db_setup
    db_cfg = setup_databases(
.venv/lib/python3.11/site-packages/django/test/utils.py:187: in setup_databases
    test_databases, mirrored_aliases = get_unique_databases_and_mirrors(aliases)
.venv/lib/python3.11/site-packages/django/test/utils.py:369: in get_unique_databases_and_mirrors
    test_databases = dict(dependency_ordered(test_databases.items(), dependencies))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

test_databases = dict_items([(('localhost', 5433, 'django.db.backends.postgresql', 'test_postgres'), ('postgres', ['myseconddb']))])
dependencies = {'myseconddb': ['default']}

    def dependency_ordered(test_databases, dependencies):
        """
        Reorder test_databases into an order that honors the dependencies
        described in TEST[DEPENDENCIES].
        """
        ordered_test_databases = []
        resolved_databases = set()
    
        # Maps db signature to dependencies of all its aliases
        dependencies_map = {}
    
        # Check that no database depends on its own alias
        for sig, (_, aliases) in test_databases:
            all_deps = set()
            for alias in aliases:
                all_deps.update(dependencies.get(alias, []))
            if not all_deps.isdisjoint(aliases):
                raise ImproperlyConfigured(
                    "Circular dependency: databases %r depend on each other, "
                    "but are aliases." % aliases
                )
            dependencies_map[sig] = all_deps
    
        while test_databases:
            changed = False
            deferred = []
    
            # Try to find a DB that has all its dependencies met
            for signature, (db_name, aliases) in test_databases:
                if dependencies_map[signature].issubset(resolved_databases):
                    resolved_databases.update(aliases)
                    ordered_test_databases.append((signature, (db_name, aliases)))
                    changed = True
                else:
                    deferred.append((signature, (db_name, aliases)))
    
            if not changed:
>               raise ImproperlyConfigured("Circular dependency in TEST[DEPENDENCIES]")
E               django.core.exceptions.ImproperlyConfigured: Circular dependency in TEST[DEPENDENCIES]

.venv/lib/python3.11/site-packages/django/test/utils.py:312: ImproperlyConfigured

My test case looks like this

@pytest.mark.django_db(databases=["myseconddb"])
def test_foo():
    assert True

My databases look like this

DATABASES = {
    "default": {
        "ENGINE": "django.contrib.gis.db.backends.postgis",
        "NAME": "postgres",
        "USER": "postgres",
        "PASSWORD": "postgres",
        "HOST": "localhost",
        "PORT": 5432,
    },
    "myseconddb": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": "postgres",
        "USER": "postgres",
        "PASSWORD": "postgres",
        "HOST": "localhost",
        "PORT": 5433,
    },
    "myseconddb-replica": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": "postgres",
        "USER": "postgres",
        "PASSWORD": "postgres",
        "HOST": "localhost",
        "PORT": 5433,
        "TEST": {
            "MIRROR": "myseconddb",
        },
    },
}

My test session

python -m pytest --durations=5  --ds myproject.settings_test -vvx -k test_foo path/to/foo/test_foo.py
============================= test session starts ==============================
platform darwin -- Python 3.11.11, pytest-7.4.4, pluggy-1.5.0 -- /Users/me/workspace/myproject/.venv/bin/python
cachedir: .pytest_cache
django: version: 4.2.20, settings: myproject.settings_test (from option)
rootdir: /Users/micahsmith/workspace/myproject
configfile: pyproject.toml
plugins: allure-pytest-2.14.2, asyncio-0.23.8, freezegun-0.4.2, anyio-4.9.0, socket-0.7.0, Faker-37.1.0, langsmith-0.3.31, myproject-pytest-plugin-0.1, django-4.11.0, mock-3.14.0, xdist-3.6.1
asyncio: mode=Mode.STRICT

I suspect this is related to https://github.com/pytest-dev/pytest-django/commit/8000db04f07822861331d0df8ef52f9c67eafc00

If I tweak my test case

@pytest.mark.django_db(databases=["default", "myseconddb"])
def test_foo():
    assert True

Then it passes

micahjsmith avatar Aug 06 '25 02:08 micahjsmith

Hmm, didn't dig into it yet, but just from reading the issue, I figure that databases can have dependencies on other databases, and maybe we need to consider that?

Interestingly the Django docs say that all databases depend on default by default. I wonder what happens in your case if you set "DEPENDENCIES": [] for the myseconddb database?

Also, while I understand how myseconddb depends on default (previous paragraph), I don't understand how default depends on myseconddb, such that there is a circular dependency...

bluetech avatar Aug 09 '25 20:08 bluetech