pytest-benchmark defer disabled-by-xdist warning until a benchmark is actually disabled

Currently, if pytest-benchmark and pytest-xdist are both installed, pytest -n $foo will always cause pytest-benchmark to emit a disabled-by-xdist warning even if the current test suite does not include any benchmarks. Perhaps the warning could be deferred until a benchmark actually gets skipped?

Jan 08 '17 18:01 anntzer

(Upon further thought, I wonder whether pytest-benchmark could just arrange to run all the benchmarks in a single process at the beginning or the end of the test suite, as this could be helpful if other parts of the test suite can be usefully parallelized -- but that's pure speculation for now.)

Sep 20 '17 01:09 anntzer

Running them at the beginning would imply that it won't work well for fixtures using the benchmark fixture, see #86

Running them at the end could work but it's hard to implement - basically we'd need to send info from slaves to master about which tests are using benchmark.

Sep 20 '17 08:09 ionelmc

Oooof.... I give up on this. If anyone knows a simple solution to this make a pr, otherwise this seems like a minor issue to me.

Jan 02 '19 22:01 ionelmc