Missing coverage related to sys.exit()

[hynek]

Describe the bug

I have a new (conveniently public) project where I can verify that certain lines are executed, but are reported as missing.

Here's a permalink: https://github.com/hynek/hatch-fancy-pypi-readme/blob/c84005d716b663f9125b0acba250e7b9ddf23661/src/hatch_fancy_pypi_readme/_cli.py#L43-L49

Coverage says: src/hatch_fancy_pypi_readme/_cli.py 23 0 4 1 96% 43->49

Adding xxx to line 43 and/or 48 crashes as expected.

I tried nailing down the first version of Coverage where it happens and it appears to be 5.0.

4.5.5 gives me src/hatch_fancy_pypi_readme/_cli.py 26 0 100%

Caveat: since 4.x doesn't support toml, it ran unconfigured.

---

While writing this I tried pulling out the sys.exit(1) out of _fail() and append it to the except blocks and that fixes it.

If I leave the sys.exits in the except blocks, but add one to _fail, it correctly reports that the sys.exit lines aren't executed.

So there must be some special handling of sys.exit in coverage?

Replacing sys.exit with a raise SystemExit does not fix it though.

To Reproduce

1. In my tox suite are 3.7, 3.10, and 3.11 that all execute that lines but don't report it.
2. This appears to be happening from 4.5.5 until 6.4.3 on both macOS (my notebook) and Linux (GHA)
3. https://github.com/hynek/hatch-fancy-pypi-readme current main is broken. It started with this commit: https://github.com/hynek/hatch-fancy-pypi-readme/commit/d8f5a7ecc17079af80e1e37b18ad9fa35dc3a0282. (you can download its coverage data from the bottom of https://github.com/hynek/hatch-fancy-pypi-readme/actions/runs/2804037630).
4. you can reproduce it easily by cloning https://github.com/hynek/hatch-fancy-pypi-readme , and running on Python 3.10 or 3.11:

• pip install -e .[dev]
• remove the # pragma: no cover from _cli.py
• python -m coverage erase
• python -m coverage run -m pytest -m "not slow"
• python -m coverage combine
• python -m coverage report

OR by running tox, but that takes a lot longer.

The quick version will report a bunch of other files (__main__.py and _fragments.py) missing coverage but that's expected, it needs different Python versions. The only coverage that's important here is in src/hatch_fancy_pypi_readme/_cli.py (the rest comes

Expected behavior That lines are executed and hence should be reported as such.

Additional context ¯\_(ツ)_/¯

I hope all of this makes some sense. lmk if you need anything, I'm happy to assist.

[nedbat]

Coverage says: src/hatch_fancy_pypi_readme/_cli.py 23 0 4 1 96% 43->49

the 43->49 doesn't mean "lines 43 through 49", it means, "the branch from 43 to 49". Coverage things execution should be able to flow from line 43 to 49 (because 49 is the line after 43). It doesn't know that _fail always raises an exception, so it will never see execution flow from 43 to 49.

You could fix this by adding # pragma: no branch to line 43, to indicate that not all of its branches are taken. You could also add this to your .coveragerc (or other configuration file):

[report]
partial_branches =
    pragma: no branch
    ^_fail\(

That will automatically exclude _fail() calls from branch consideration.

(BTW, I didn't try the code, but I can tomorrow if this isn't right.)

[hynek]

Ah! Thanks, I use partial_branches. As a small nit for anyone coming with the same problem: it's "^\s*_fail\(" (or "^\\s*_fail\\(" in toml).

I guess this is expected then? It would be nice to have an annotation like NoReturn for coverage I guess but feel free to close.

[vsajip]

Not sure if related, but I get a line with sys.exit() reported as not covered when I'm pretty sure it is executed - this is with coverage 6.4.4. See the last line of code on this page on CodeCov. The source repo with the coverage run in its workflow is public.

[nedbat]

Are you sure there's no exception being raised? Lines 266-277 are catching specific exceptions, but others would fly by and skip line 278.

[vsajip]

Oh, good catch. I'd missed a catch SystemExit: - sorry for the noise.