coveragepy
coveragepy copied to clipboard
Running with pytest-xdist leads to "No data to report."
Describe the bug
pytest-xdist allows running tests in parallel.
So we can do something like pytest -n auto, and our tests take 5 minutes instead of 15.
But coverage run --parallel-mode -m pytest -n auto seems to not work: coverage report returns "No data to report."
I thought the --parallel-mode was intended for this, is this correct?
To Reproduce
https://github.com/khink/django-tutorial-tournament-pool/pull/5 illustrates the issue.
Steps to reproduce:
git clone [email protected]:khink/django-tutorial-tournament-pool.git -b measure-coverage-with-tests-in-parallelcd django-tutorial-tournament-poolmake coverageworks finemake coverage-paralleldoesn't.
Some more information about versions:
$ python --version
Python 3.10.12
$ pip freeze
asgiref==3.7.2
beautifulsoup4==4.12.2
coverage==7.3.2
Django==4.2.6
django-webtest==1.9.11
exceptiongroup==1.1.3
execnet==2.0.2
iniconfig==2.0.0
packaging==23.2
pluggy==1.3.0
pytest==7.4.2
pytest-django==4.5.2
pytest-xdist==3.3.1
soupsieve==2.5
sqlparse==0.4.4
tomli==2.0.1
typing_extensions==4.8.0
waitress==2.1.2
WebOb==1.8.7
WebTest==3.0.0
It works with extra setup, but I think that we should do something to make coverage require less setup for pytest+xdist as this combination is by far the most popular way of running tests on python projects.
We do currently have few helper projects suck https://pypi.org/project/pytest-cov/ , but still the overall experience is far from ideal, as we all endup spending many hours debugging unexpected behaviors, like this one.