coveragepy
coveragepy copied to clipboard
nedbat
Branch coverage false positive for exit context
Describe the bug
In https://github.com/twisted/towncrier/pull/591#issuecomment-2081516763, I encountered what appears to be a spurious false positive. The coverage report is indicating that 20->exit is not covered. even though that line seems to be unequivocally covered (how could that context not exit?).
To Reproduce How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:
- What version of Python are you using? All (3.8->3.12)
- What version of coverage.py shows the problem?
towncrier debt/pathlib-read @ .nox/coverage_report/bin/coverage debug sys
-- sys -------------------------------------------------------
coverage_version: 7.4.1
coverage_module: /Users/jaraco/code/twisted/towncrier/.nox/coverage_report/lib/python3.12/site-packages/coverage/__init__.py
core: -none-
CTracer: available
plugins.file_tracers: -none-
plugins.configurers: -none-
plugins.context_switchers: -none-
configs_attempted: .coveragerc
setup.cfg
tox.ini
pyproject.toml
configs_read: /Users/jaraco/code/twisted/towncrier/pyproject.toml
config_file: /Users/jaraco/code/twisted/towncrier/pyproject.toml
config_contents: b'[build-system]\nrequires = [\n "hatchling",\n "incremental == 22.10.0",\n]\nbuild-backend = "hatchling.build"\n\n\n[project]\ndynamic = ["version"]\nname = "towncrier"\ndescription = "Building newsfiles for your project."\nreadme = "README.rst"\nlicense = "MIT"\n# Keep version list in-sync with noxfile/tests & ci.yml/test-linux.\nclassifiers = [\n "Intended Audience :: Developers",\n "License :: OSI Approved :: MIT License",\n "Operating System :: POSIX :: Linux",\n "Operating System :: MacOS :: MacOS X",\n "Operating System :: Microsoft :: Windows",\n "Programming Language :: Python :: 3.8",\n "Programming Language :: Python :: 3.9",\n "Programming Language :: Python :: 3.10",\n "Programming Language :: Python :: 3.11",\n "Programming Language :: Python :: 3.12",\n "Programming Language :: Python :: Implementation :: CPython",\n "Programming Language :: Python :: Implementation :: PyPy",\n]\nrequires-python = ">=3.8"\ndependencies = [\n "click",\n "importlib-resources>=5; python_version<\'3.10\'",\n "incremental",\n "jinja2",\n "tomli; python_version<\'3.11\'",\n]\n\n[project.optional-dependencies]\ndev = [\n "packaging",\n "sphinx >= 5",\n "furo",\n "twisted",\n "nox",\n]\n\n[project.scripts]\ntowncrier = "towncrier._shell:cli"\n\n[project.urls]\nDocumentation = "https://towncrier.readthedocs.io/"\nChat = "https://web.libera.chat/?channels=%23twisted"\n"Mailing list" = "https://mail.python.org/mailman3/lists/twisted.python.org/"\nIssues = "https://github.com/twisted/towncrier/issues"\nRepository = "https://github.com/twisted/towncrier"\nTests = "https://github.com/twisted/towncrier/actions?query=branch%3Atrunk"\nCoverage = "https://codecov.io/gh/twisted/towncrier"\nDistribution = "https://pypi.org/project/towncrier"\n\n\n[tool.hatch.version]\nsource = "code"\npath = "src/towncrier/_version.py"\nexpression = "_hatchling_version"\n\n[tool.hatch.build]\nexclude = [\n "admin",\n "bin",\n "docs",\n ".readthedocs.yaml",\n "src/towncrier/newsfragments",\n]\n\n\n[tool.towncrier]\n package = "towncrier"\n package_dir = "src"\n filename = "NEWS.rst"\n issue_format = "`#{issue} <https://github.com/twisted/towncrier/issues/{issue}>`_"\n\n [[tool.towncrier.section]]\n path = ""\n\n [[tool.towncrier.type]]\n directory = "feature"\n name = "Features"\n showcontent = true\n\n [[tool.towncrier.type]]\n directory = "bugfix"\n name = "Bugfixes"\n showcontent = true\n\n [[tool.towncrier.type]]\n directory = "doc"\n name = "Improved Documentation"\n showcontent = true\n\n [[tool.towncrier.type]]\n directory = "removal"\n name = "Deprecations and Removals"\n showcontent = true\n\n [[tool.towncrier.type]]\n directory = "misc"\n name = "Misc"\n showcontent = false\n\n\n[tool.black]\nexclude = \'\'\'\n\n(\n /(\n \\.eggs # exclude a few common directories in the\n | \\.git # root of the project\n | \\.nox\n | \\.venv\n | \\.env\n | env\n | _build\n | _trial_temp.*\n | build\n | dist\n | debian\n )/\n)\n\'\'\'\n\n\n[tool.isort]\nprofile = "attrs"\nline_length = 88\n\n\n[tool.ruff.isort]\n# Match isort\'s "attrs" profile\nlines-after-imports = 2\nlines-between-types = 1\n\n\n[tool.mypy]\nstrict = true\n# 2022-09-04: Trial\'s API isn\'t annotated yet, which limits the usefulness of type-checking\n# the unit tests. Therefore they have not been annotated yet.\nexclude = \'^src/towncrier/test/test_.*\\.py$\'\n\n[[tool.mypy.overrides]]\nmodule = \'towncrier.click_default_group\'\n# Vendored module without type annotations.\nignore_errors = true\n\n[[tool.mypy.overrides]]\nmodule = \'incremental\'\n# No released version with type hints.\nignore_missing_imports = true\n\n\n[tool.coverage.run]\nparallel = true\nbranch = true\nsource = ["towncrier"]\n\n[tool.coverage.paths]\nsource = ["src", ".nox/tests-*/**/site-packages"]\n\n[tool.coverage.report]\nshow_missing = true\nskip_covered = true\nexclude_lines = [\n "pragma: no cover",\n "if TYPE_CHECKING:",\n]\nomit = [\n "src/towncrier/__main__.py",\n "src/towncrier/test/*",\n "src/towncrier/click_default_group.py",\n]\n'
data_file: -none-
python: 3.12.3 (main, Apr 9 2024, 08:09:14) [Clang 15.0.0 (clang-1500.3.9.4)]
platform: macOS-14.4.1-arm64-arm-64bit
implementation: CPython
executable: /Users/jaraco/code/twisted/towncrier/.nox/coverage_report/bin/python
def_encoding: utf-8
fs_encoding: utf-8
pid: 38077
cwd: /Users/jaraco/code/twisted/towncrier
path: /Users/jaraco/code/twisted/towncrier/.nox/coverage_report/bin
/opt/homebrew/Cellar/[email protected]/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python312.zip
/opt/homebrew/Cellar/[email protected]/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12
/opt/homebrew/Cellar/[email protected]/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload
/Users/jaraco/code/twisted/towncrier/.nox/coverage_report/lib/python3.12/site-packages
environment: HOME = /Users/jaraco
PIPX_DEFAULT_PYTHON = /opt/homebrew/bin/python3
PIP_NO_PYTHON_VERSION_WARNING = 1
command_line: .nox/coverage_report/bin/coverage debug sys
sqlite3_sqlite_version: 3.45.3
sqlite3_temp_store: 0
sqlite3_compile_options: ATOMIC_INTRINSICS=1, COMPILER=clang-15.0.0, DEFAULT_AUTOVACUUM,
DEFAULT_CACHE_SIZE=-2000, DEFAULT_FILE_FORMAT=4,
DEFAULT_JOURNAL_SIZE_LIMIT=-1, DEFAULT_MMAP_SIZE=0, DEFAULT_PAGE_SIZE=4096,
DEFAULT_PCACHE_INITSZ=20, DEFAULT_RECURSIVE_TRIGGERS,
DEFAULT_SECTOR_SIZE=4096, DEFAULT_SYNCHRONOUS=2,
DEFAULT_WAL_AUTOCHECKPOINT=1000, DEFAULT_WAL_SYNCHRONOUS=2,
DEFAULT_WORKER_THREADS=0, DIRECT_OVERFLOW_READ, ENABLE_API_ARMOR,
ENABLE_COLUMN_METADATA, ENABLE_DBSTAT_VTAB, ENABLE_FTS3,
ENABLE_FTS3_PARENTHESIS, ENABLE_FTS4, ENABLE_FTS5, ENABLE_GEOPOLY,
ENABLE_MATH_FUNCTIONS, ENABLE_MEMORY_MANAGEMENT, ENABLE_PREUPDATE_HOOK,
ENABLE_RTREE, ENABLE_SESSION, ENABLE_STAT4, ENABLE_UNLOCK_NOTIFY,
MALLOC_SOFT_LIMIT=1024, MAX_ATTACHED=10, MAX_COLUMN=2000,
MAX_COMPOUND_SELECT=500, MAX_DEFAULT_PAGE_SIZE=8192, MAX_EXPR_DEPTH=1000,
MAX_FUNCTION_ARG=127, MAX_LENGTH=1000000000, MAX_LIKE_PATTERN_LENGTH=50000,
MAX_MMAP_SIZE=0x7fff0000, MAX_PAGE_COUNT=0xfffffffe, MAX_PAGE_SIZE=65536,
MAX_SQL_LENGTH=1000000000, MAX_TRIGGER_DEPTH=1000,
MAX_VARIABLE_NUMBER=250000, MAX_VDBE_OP=250000000, MAX_WORKER_THREADS=8,
MUTEX_PTHREADS, SYSTEM_MALLOC, TEMP_STORE=1, THREADSAFE=1, USE_URI
- What versions of what packages do you have installed?
towncrier debt/pathlib-read @ .nox/coverage_report/bin/pip freeze
coverage==7.4.1
- What code shows the problem? https://github.com/jaraco/towncrier/blob/e5c6df77d40c8a5ae7aee7a5ac4fbb74c25a3bcd/src/towncrier/_writer.py#L20-L21
- What commands should we run to reproduce the problem?
- install nox
- git clone https://github.com/jaraco/towncrier
- cd towncrier
- git checkout e5c6df77d
- nox
Expected behavior That line is covered by the code, so there should be no coverage concerns. Moreover, an earlier version of the code does not exhibit the issue (even though it also opens the file in a context).
Am I missing some crucial detail here?
git checkout e5c6df77d
This commit doesn't seem to be in the repo anymore, and when I ran nox on trunk, I got:
nox > Running session coverage_report
nox > Creating virtual environment (virtualenv) using python in .nox/coverage_report
nox > python -m pip install 'coverage[toml]'
nox > coverage combine
Combined data file .coverage.geometer.lan.5627.XAuKZYsx
Combined data file .coverage.geometer.lan.5871.XazxqWlx
Combined data file .coverage.geometer.lan.6111.XTCIvhpx
Combined data file .coverage.geometer.lan.6360.XbLLYRNx
Combined data file .coverage.geometer.lan.6611.XtIfvEOx
Combined data file .coverage.geometer.lan.6851.XzORrlFx
nox > coverage report
Name Stmts Miss Branch BrPart Cover Missing
---------------------------------------------------
TOTAL 674 0 265 0 100%
13 files skipped due to complete coverage.
Is this still happening somewhere I can look at?
I pushed a tag for the commit. git checkout coverage-1770 should get the commit. It's definitely there, as the github link to the code above is pinned to that commit. You may need to fetch it explicitly, something like git fetch https://github.com/jaraco/towncrier e5c6df77d.
I just double-checked and ran the repro steps in a clean docker image and they do work for me. So maybe pushing that tag helped keep that commit active.
OK, I got the commit, but I still have 100% coverage: https://gist.github.com/nedbat/3ad003d2c6428282e3249fd33a5f8e2d
So hard to reproduce the issue reliably :(
I see in your run that the typecheck job failed, but the others passed. In my run, the docs job fails, but the others pass. When I run it locally on my mac, all the jobs pass (but tests-pypy3.8 is skipped). In both of my cases, the coverage reports show the missing branch.
Since I supplied the Docker repro instructions, are you able to use those to replicate the missed expectation?
Since you're not able to reproduce the issue, that suggests the issue may be sensitive to the environment, and since the issue occurs on my mac and in a Linux docker image I created, it makes me think there may be something about my environment that's triggering the behavior. I'll try paring the Docker image down to its minimal requirements.
Here's a more minimal docker image that replicates the missed branch:
FROM ubuntu:noble
ENV TZ=UTC
RUN apt update
RUN apt install -y software-properties-common
RUN apt-add-repository -y ppa:deadsnakes
RUN apt update
RUN DEBIAN_FRONTEND=noninteractive apt install -y git python3 pipx
RUN apt install -y python3.9-dev python3.9-venv
RUN git clone https://github.com/jaraco/towncrier
WORKDIR towncrier
RUN git checkout e5c6df77d
RUN pipx install nox
ENV PATH=/root/.local/bin:$PATH
CMD nox