Testing fails with `pytest-xdist` installed and disabled

[mtelka]

I'm packaging pytest for OpenIndiana and while testing the package I noticed that some tests are failing when there is pytest-xdist installed and disabled using -p no:xdist option set in the PYTEST_ADDOPTS environment variable.

The list of failed tests is here:

FAILED testing/test_junitxml.py::test_random_report_log_xdist - FileNotFoundE...
FAILED testing/test_junitxml.py::test_runs_twice_xdist - FileNotFoundError: [...
FAILED testing/test_terminal.py::TestProgressOutputStyle::test_xdist_normal
FAILED testing/test_terminal.py::TestProgressOutputStyle::test_xdist_normal_count
FAILED testing/test_terminal.py::TestProgressOutputStyle::test_xdist_verbose
FAILED testing/test_terminal.py::TestProgressWithTeardown::test_xdist_normal

Here is typical test failure:

__________________ TestProgressOutputStyle.test_xdist_normal ___________________

self = <test_terminal.TestProgressOutputStyle object at 0x7fffabbf7df0>
many_tests_files = None
pytester = <Pytester PosixPath('/tmp/pytest-of-marcel/pytest-23/test_xdist_normal0')>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fffa8f834c0>

    def test_xdist_normal(
        self, many_tests_files, pytester: Pytester, monkeypatch
    ) -> None:
        pytest.importorskip("xdist")
        monkeypatch.delenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", raising=False)
        output = pytester.runpytest("-n2")
>       output.stdout.re_match_lines([r"\.{20} \s+ \[100%\]"])
E       Failed: remains unmatched: '\\.{20} \\s+ \\[100%\\]'
$(BUILD_DIR)/testing/test_terminal.py:2178: Failed
----------------------------- Captured stderr call -----------------------------
ERROR: usage: __main__.py [options] [file_or_dir] [file_or_dir] [...]
__main__.py: error: unrecognized arguments: -n2
  inifile: None
  rootdir: /tmp/pytest-of-marcel/pytest-23/test_xdist_normal0

[ghost]

Can you try removing "-n2" from thet pytest command, especially if you're not planning to run the tests in parallel?

This is the part of your error message that points to this:

unrecognized arguments: -n2

Since pytest-xdist is disabled, it wouldn't recognise this argument

[mtelka]

The -n2 is in the pytest sources so your suggestion to remove it is hardly possible :-). The actual problem is that the test of pytest itself is trying to import xdist (this pass, even the xdist is disabled) and then immediately it tries to run with -n2, which fails, since xdist is disabled.

This pytest test (TestProgressOutputStyle.test_xdist_normal) needs better check for availability and usability of xdist because, apparently, the xdist is currently considered as optional test dependency so it is legitimate to have the plugin installed and disabled.

The other option would be to list pytest-xdist as a (hard) test dependency to make sure it is always installed (and so enabled).