requests
requests copied to clipboard
Measure and report test coverage
This runs pytest within coverage, enables pytest-xdist to improve test run times, and reports the coverage on GitHub Actions for us.
This also ensures that the parallel mode coverage files are ignored and adds things to pytest configuration to make tox command simpler
I haven't had a chance to dig into the failure yet, but it's showing 26% coverage which seems extremely low. I tried to go back to find older examples of when codecov was running and am seeing closer to 89% coverage. Do we possibly have something misconfigured?
Running in tox I get north of 80% as well but we're not running in tox on GitHub actions so that may be part of the problem
It looks like we're possibly hitting an issue with coverage combine similar to what's noted in https://github.com/nedbat/coveragepy/issues/1036.
We're generating 21 files for the 21 tests across 7 interpreters. From Ned's response it sounds like those combinations have subtle variances that may result in false flags for missing coverage? When running the tests locally without parallel=True and generating a single file it shows 87% like I'd expect. If I use parallel = True and don't use combine, we get the same results we're seeing in CI as if we're missing coverage data.
Have you used this workflow in other projects? If not, we might consider dividing coverage by interpreter or another partitioning strategy.
I mostly stole this from urllib3 and https://hynek.me/articles/ditch-codecov-python/ but maybe I missed something important