coveragepy icon indicating copy to clipboard operation
coveragepy copied to clipboard

Regression in branch coverage using Coverage 7.6.3 on Python 3.12+

Open freakboy3742 opened this issue 4 months ago • 0 comments

Describe the bug

Coverage 7.6.3 appears to have introduced a regression in the calculation of branch coverage related to the handling of while loops inside context managers.

To Reproduce How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

  1. What version of Python are you using?

Observed using 3.12.7 and 3.13.0. It doesn't appear in 3.11 or earlier.

  1. What version of coverage.py shows the problem? The output of coverage debug sys is helpful.

7.6.3. The problem doesn't exist in 7.6.1.

  1. What versions of what packages do you have installed? The output of pip freeze is helpful.
  2. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.

The problem has been found in this dependabot update on Briefcase, which bumps coverage.py from 7.6.1 to 7.6.3.

  1. What commands should we run to reproduce the problem? Be specific. Include everything, even git clone, pip install, and so on. Explain like we're five!

$ git clone [email protected]:beeware/briefcase.git $ cd briefcase $ git checkout -b coverage-7.6.3 origin/dependabot/pip/coverage-toml--7.6.3 $ python3.12 -m venv venv $ source venv/bin/activate (venv) $ pip install tox $ tox -m test312

This will run the full test suite, reporting 2 branch coverage misses:

...
coverage312: commands[1]> python -m coverage report --fail-under=100
Name                                   Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------------------
src/briefcase/platforms/iOS/xcode.py     219      0     48      2  99.3%   525->528, 541->544
----------------------------------------------------------------------------------
TOTAL                                   7084      0   1598      2  99.9%

67 files skipped due to complete coverage.

Expected behavior

If you omit the git checkout -b ... line, the main branch uses 7.6.1, and shows 100% coverage.

...
coverage312: commands[1]> python -m coverage report --fail-under=100
Name    Stmts   Miss Branch BrPart   Cover   Missing
----------------------------------------------------
TOTAL    7104      0   2354      0  100.0%

68 files skipped due to complete coverage.

Additional context

It seems likely this is related to #1876, and commit 378c321ba0.

Both of the new "missing" branches follow the same basic pattern of a while loop with a walrus operator, inside a context manager:

https://github.com/beeware/briefcase/blob/563c6acd7b0875ee3f8e808c70065a10c077ed05/src/briefcase/platforms/iOS/xcode.py#L525-L528

https://github.com/beeware/briefcase/blob/563c6acd7b0875ee3f8e808c70065a10c077ed05/src/briefcase/platforms/iOS/xcode.py#L541-L544

Unfortunately, I haven't had any luck reducing this to a simpler example case.

freakboy3742 avatar Oct 15 '24 03:10 freakboy3742