pytest-xdist icon indicating copy to clipboard operation
pytest-xdist copied to clipboard

Use a different exit code when collection fails

Open Redoubts opened this issue 1 year ago • 0 comments

I noticed https://pytest-xdist.readthedocs.io/en/stable/known-limitations.html#order-and-amount-of-test-must-be-consistent

And indeed I expect pytest to fail if I have some randomness in my test definitions (say a random parameterized fixture).

Different tests were collected between gw1 and gw2. The difference is:
...

But I noticed that pytest exits with code 1 in this case. This code is reserved for "collected but failed" https://docs.pytest.org/en/stable/reference/exit-codes.html. I think this should be either code 3 or 5, but at least not code 1 since this feels like a collection or otherwise "internal" error.

Redoubts avatar Aug 27 '24 13:08 Redoubts