pytest-xdist
pytest-xdist copied to clipboard
pytest-dev
Do not spawn more workers than testcases
I switched from a Core i5 (4 cores) to a Ryzen 7 (16 cores). The time pytest-xdist takes to spawn the workers is consequently greater, but it is expected.
Sometimes I run pytest for just several tests: sometimes 1 test, sometimes 1 file with a few tests. I thought it would save some time if pytest-xdist would only spawn the workers needed.
What do you think?
when xdist starts the workers, it has zero idea how many tests are there, so currently its not implementable
I guess it would be possible to launch a single worker, collect the tests, and then based on the number of tests spawn the rest of the workers as needed, but this would probably require quite a bit of refactoring to accomplish.
@Krilivye the master does not run any collection, all collection is on the workers
Yes, because transferring tests and fixtures over the wire (between workers and the master node) is quite difficult/impossible, so every worker collects all tests and the master just schedules which test to run based on their node id (take a look at OVERVIEW for more details).
Some related issues: #270, #271