coveragepy icon indicating copy to clipboard operation
coveragepy copied to clipboard

Running with pytest-xdist leads to "No data to report."

Open khink opened this issue 2 years ago • 8 comments

Describe the bug

pytest-xdist allows running tests in parallel. So we can do something like pytest -n auto, and our tests take 5 minutes instead of 15.

But coverage run --parallel-mode -m pytest -n auto seems to not work: coverage report returns "No data to report."

I thought the --parallel-mode was intended for this, is this correct?

To Reproduce

https://github.com/khink/django-tutorial-tournament-pool/pull/5 illustrates the issue.

Steps to reproduce:

  1. git clone [email protected]:khink/django-tutorial-tournament-pool.git -b measure-coverage-with-tests-in-parallel
  2. cd django-tutorial-tournament-pool
  3. make coverage works fine
  4. make coverage-parallel doesn't.

Some more information about versions:

$ python --version
Python 3.10.12
$ pip freeze
asgiref==3.7.2
beautifulsoup4==4.12.2
coverage==7.3.2
Django==4.2.6
django-webtest==1.9.11
exceptiongroup==1.1.3
execnet==2.0.2
iniconfig==2.0.0
packaging==23.2
pluggy==1.3.0
pytest==7.4.2
pytest-django==4.5.2
pytest-xdist==3.3.1
soupsieve==2.5
sqlparse==0.4.4
tomli==2.0.1
typing_extensions==4.8.0
waitress==2.1.2
WebOb==1.8.7
WebTest==3.0.0

khink avatar Oct 18 '23 08:10 khink

It works with extra setup, but I think that we should do something to make coverage require less setup for pytest+xdist as this combination is by far the most popular way of running tests on python projects.

We do currently have few helper projects suck https://pypi.org/project/pytest-cov/ , but still the overall experience is far from ideal, as we all endup spending many hours debugging unexpected behaviors, like this one.

ssbarnea avatar May 22 '24 08:05 ssbarnea