coveragepy icon indicating copy to clipboard operation
coveragepy copied to clipboard

coverage fails to combine paths for multiple test environments

Open dakotahorstman opened this issue 2 years ago • 1 comments

Describe the bug coverage is failing to combine duplicate source files from different environments. For example, take the below screenshot.

image

To Reproduce How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

  1. What version of Python are you using? Python 3.9.13
  2. What version of coverage.py shows the problem? The output of coverage debug sys is helpful.
-- sys -------------------------------------------------------
               coverage_version: 7.0.1
                coverage_module: /home/dur10420/miniconda3/envs/python-step-series/lib/python3.9/site-packages/coverage/__init__.py
                         tracer: -none-
                        CTracer: available
           plugins.file_tracers: -none-
            plugins.configurers: -none-
      plugins.context_switchers: -none-
              configs_attempted: .coveragerc
                                 setup.cfg
                   configs_read: /home/dur10420/Documents/python-step-series/setup.cfg
                    config_file: /home/dur10420/Documents/python-step-series/setup.cfg
                config_contents: b'# This file is used to configure your project.\n# Read more about the various options under:\n# https://setuptools.pypa.io/en/latest/userguide/declarative_config.html\n# https://setuptools.pypa.io/en/latest/references/keywords.html\n\n[metadata]\nname = python-step-series\ndescription = A Python library for OSC communication with the Ponoor Step-series devices.\nauthor = Julian_Orteil\nauthor_email =\nlicense = MIT\nlicense_files = LICENSE.txt\nlong_description = file: README.rst\nlong_description_content_type = text/x-rst; charset=UTF-8\nurl = https://github.com/ponoor/python-step-series\nproject_urls =\n    Documentation = https://python-step-series.readthedocs.io/en/stable/index.html\n    Source = https://github.com/ponoor/python-step-series\n    Changelog = https://github.com/ponoor/python-step-series/blob/main/CHANGELOG.rst\n    Tracker = https://github.com/ponoor/python-step-series/issues\n    Download = https://pypi.org/project/python-step-series/#files\n\n# Change if running only on Windows, Mac or Linux (comma-separated)\nplatforms = any\n\n# Add here all kinds of additional classifiers as defined under\n# https://pypi.org/classifiers/\nclassifiers =\n    Development Status :: 5 - Production/Stable\n    Programming Language :: Python\n    Programming Language :: Python :: 3 :: Only\n    Intended Audience :: Developers\n    Natural Language :: English\n    Natural Language :: Japanese\n    License :: OSI Approved :: MIT License\n    Operating System :: OS Independent\n    Typing :: Typed\nkeywords = ponoor, step400, step800, motor\n\n\n[options]\nzip_safe = False\npackages = find_namespace:\ninclude_package_data = True\npackage_dir =\n    =src\n\n# Require a min/specific Python version (comma-separated conditions)\npython_requires = >=3.7\n\n# Add here dependencies of your project (line-separated), e.g. requests>=2.2,<3.0.\n# Version specifiers like >=2.2,<3.0 avoid problems due to API changes in\n# new major versions. This works if the required packages follow Semantic Versioning.\n# For more information, check out https://semver.org/.\ninstall_requires =\n    importlib-metadata; python_version<"3.8"\n    python-osc==1.8.0\n\n\n[options.packages.find]\nwhere = src\nexclude =\n    __pycache__\n    .envs\n    *.egg\n    .eggs\n    .github\n    .tox\n    .vscode\n    assets\n    bin\n    build\n    dist\n    docs/conf.py\n    locale\n    requirements\n    tests\n\n[options.extras_require]\n# Add here additional requirements for extra features, to install with:\n# `pip install python-step-series[PDF]` like:\n# PDF = ReportLab; RXP\n\n# Add here test requirements (semicolon/line-separated)\ntesting =\n    coverage==7.0.1\n    pytest==7.2.0\n    pytest-sugar==0.9.6\n    pytest-order==1.0.1\n\n[options.entry_points]\n# Add here console scripts like:\n# console_scripts =\n#     script_name = stepseries.module:function\n# For example:\n# console_scripts =\n#     fibonacci = stepseries.skeleton:run\n# And any other entry points, for example:\n# pyscaffold.cli =\n#     awesome = pyscaffoldext.awesome.extension:AwesomeExtension\n\n[tool:pytest]\n# Specify command line options as you would do when invoking pytest directly.\n# e.g. --cov-report html (or xml) for html/xml output or --junitxml junit.xml\n# in order to write a coverage file that can be read by Jenkins.\n# CAUTION: --cov flags may prohibit setting breakpoints while debugging.\n#          Comment those flags to avoid this pytest issue.\naddopts =\n    --strict-markers\n    --strict-config\n    --tb=short\n    --verbose\n    --capture=no\nnorecursedirs =\n    __pycache__\n    .envs\n    *.egg\n    .eggs\n    .github\n    .tox\n    .vscode\n    assets\n    bin\n    build\n    dist\n    docs/conf.py\n    locale\n    requirements\ntestpaths = tests\nxfail_strict = True\n# Use pytest markers to select/deselect specific tests\nmarkers =\n    incremental: xfails further tests in a class if the current fails\n\n    skip_400_disconnected: skips marked tests (or suites) if STEP400 hardware is not detected\n    check_400_embrake: verifies configuration and device statuses before running tests that involve the EM-brake\n    check_400_homesw: verifies configuration and device statuses before running tests that involve the home sw\n    check_400_limitsw: verifies configuration and device statuses before running tests that involve the limit sw\n    check_400_motors: verifies configuration and device statuses before allowing a test to physically run motors\n    reset_400_device: after a test suite (class) finishes, reset the device to provide a blank slate\n    skip_400_not_configured: skips marked tests (or suites) if test presets have not been flagged as configured\n\n    skip_800_disconnected: skips marked tests (or suites) if STEP800 hardware is not detected\n    check_800_embrake: verifies configuration and device statuses before running tests that involve the EM-brake\n    check_800_homesw: verifies configuration and device statuses before running tests that involve the home sw\n    check_800_motors: verifies configuration and device statuses before allowing a test to physically run motors\n    reset_800_device: after a test suite (class) finishes, reset the device to provide a blank slate\n    skip_800_not_configured: skips marked tests (or suites) if test presets have not been flagged as configured\n\n[devpi:upload]\n# Options for the devpi: PyPI server and packaging tool\n# VCS export must be deactivated since we are using setuptools-scm\nno_vcs = 1\nformats = bdist_wheel\n\n[flake8]\n# Some sane defaults for the code style checker flake8\nstrictness = long\nshow-source = True\nmax-line-length = 100\nmax-complexity = 6\ndocstring-style = google\nextend_ignore = E203, W503\n                # ^  Black-compatible\n                #    E203 and W503 have edge cases handled by black\nexclude =\n    __pycache__\n    .envs\n    *.egg\n    .eggs\n    .github\n    .tox\n    .vscode\n    assets\n    bin\n    build\n    dist\n    docs/conf.py\n    locale\n    requirements\nignore = D100,D104,D401,DAR103,DAR203,E800,N802,Q000,RST303,RST304,S101,W504,WPS326\nper-file-ignores =\n    setup.py: WPS421, C813\n    tests/conftest.py: D208, E800\nrst-directives =\n    # These are sorted alphabetically - but that does not matter\n    autosummary,data,currentmodule,deprecated,\n    glossary,moduleauthor,plot,testcode,\n    versionadded,versionchanged,\nrst-roles =\n    attr,class,func,meth,mod,obj,ref,term,\n    # C programming language:\n    c:member,\n    # Python programming language:\n    py:func,py:mod,\n\n[isort]\ninclude_trailing_comma = True\nuse_parentheses = True\nmulti_line_output = 3\nline_length = 100\nprofile = black\nknown_first_party = stepseries\n\n[mypy]\nallow_redefinition = False\ncheck_untyped_defs = True\nignore_errors = False\nignore_missing_imports = True\nimplicit_reexport = False\nlocal_partial_types = True\nstrict_optional = True\nstrict_equality = True\nno_implicit_optional = True\nwarn_unused_ignores = True\nwarn_redundant_casts = True\nwarn_unused_configs = True\nwarn_unreachable = True\nwarn_no_return = True\n\n[coverage:run]\nbranch = True\nsource = stepseries\nconcurrency = multiprocessing\nparallel = True\nomit =\n    *tests*\n\n[coverage:paths]\nsource =\n    */site-packages/\n    */stepseries/\n\n[coverage:html]\nshow_contexts = True\n\n[coverage:report]\nfail_under = 100\nshow_missing = True\nexclude_lines =\n    pragma: no cover\n\n    def __repr__\n    if self\\.debug\n\n    raise AssertionError\n    raise NotImplementedError\n\n    if 0:\n    if __name__ == .__main__.:\n\n[pyscaffold]\n# PyScaffold\'s parameters when the project was created.\n# This will be used when updating. Do not change!\nversion = 4.1.1\npackage = stepseries\nextensions =\n    no_skeleton\n    pre_commit\n'
                      data_file: -none-
                         python: 3.9.15 (main, Nov 24 2022, 14:31:59) [GCC 11.2.0]
                       platform: Linux-5.15.0-52-generic-x86_64-with-glibc2.31
                 implementation: CPython
                     executable: /home/dur10420/miniconda3/envs/python-step-series/bin/python
                   def_encoding: utf-8
                    fs_encoding: utf-8
                            pid: 2865841
                            cwd: /home/dur10420/Documents/python-step-series
                           path: /home/dur10420/miniconda3/envs/python-step-series/bin
                                 /home/dur10420/miniconda3/envs/python-step-series/lib/python39.zip
                                 /home/dur10420/miniconda3/envs/python-step-series/lib/python3.9
                                 /home/dur10420/miniconda3/envs/python-step-series/lib/python3.9/lib-dynload
                                 /home/dur10420/miniconda3/envs/python-step-series/lib/python3.9/site-packages
                    environment: CONDA_PYTHON_EXE = /home/dur10420/miniconda3/bin/python
                                 HOME = /home/dur10420
                   command_line: /home/dur10420/miniconda3/envs/python-step-series/bin/coverage debug sys
         sqlite3_sqlite_version: 3.40.1
             sqlite3_temp_store: 0
        sqlite3_compile_options: ATOMIC_INTRINSICS=1, COMPILER=gcc-11.2.0, DEFAULT_AUTOVACUUM,
                                 DEFAULT_CACHE_SIZE=-2000, DEFAULT_FILE_FORMAT=4,
                                 DEFAULT_JOURNAL_SIZE_LIMIT=-1, DEFAULT_MMAP_SIZE=0, DEFAULT_PAGE_SIZE=4096,
                                 DEFAULT_PCACHE_INITSZ=20, DEFAULT_RECURSIVE_TRIGGERS,
                                 DEFAULT_SECTOR_SIZE=4096, DEFAULT_SYNCHRONOUS=2,
                                 DEFAULT_WAL_AUTOCHECKPOINT=1000, DEFAULT_WAL_SYNCHRONOUS=2,
                                 DEFAULT_WORKER_THREADS=0, ENABLE_COLUMN_METADATA, ENABLE_DBSTAT_VTAB,
                                 ENABLE_FTS3, ENABLE_FTS3_TOKENIZER, ENABLE_FTS4, ENABLE_FTS5,
                                 ENABLE_GEOPOLY, ENABLE_MATH_FUNCTIONS, ENABLE_RTREE, ENABLE_UNLOCK_NOTIFY,
                                 MALLOC_SOFT_LIMIT=1024, MAX_ATTACHED=10, MAX_COLUMN=2000,
                                 MAX_COMPOUND_SELECT=500, MAX_DEFAULT_PAGE_SIZE=8192, MAX_EXPR_DEPTH=10000,
                                 MAX_FUNCTION_ARG=127, MAX_LENGTH=1000000000, MAX_LIKE_PATTERN_LENGTH=50000,
                                 MAX_MMAP_SIZE=0x7fff0000, MAX_PAGE_COUNT=1073741823, MAX_PAGE_SIZE=65536,
                                 MAX_SQL_LENGTH=1000000000, MAX_TRIGGER_DEPTH=1000,
                                 MAX_VARIABLE_NUMBER=250000, MAX_VDBE_OP=250000000, MAX_WORKER_THREADS=8,
                                 MUTEX_PTHREADS, SECURE_DELETE, SYSTEM_MALLOC, TEMP_STORE=1, THREADSAFE=1
  1. What versions of what packages do you have installed? The output of pip freeze is helpful.
alabaster==0.7.12
attrs==22.2.0
Babel==2.11.0
black==22.12.0
certifi @ file:///croot/certifi_1671487769961/work/certifi
cfgv==3.3.1
charset-normalizer==2.1.1
click==8.1.3
coverage==7.0.1
detect-secrets==1.4.0
distlib==0.3.6
docutils==0.19
exceptiongroup==1.1.0
filelock==3.9.0
flake8==4.0.1
flake8-isort==5.0.3
identify==2.5.13
idna==3.4
imagesize==1.4.1
importlib-metadata==6.0.0
iniconfig==2.0.0
isort==5.11.4
Jinja2==3.1.2
MarkupSafe==2.1.1
mccabe==0.6.1
mypy-extensions==0.4.3
nodeenv==1.7.0
packaging==23.0
pathspec==0.10.3
platformdirs==2.6.2
pluggy==1.0.0
pre-commit==2.21.0
py==1.11.0
pycodestyle==2.8.0
pyflakes==2.4.0
Pygments==2.14.0
pytest==7.2.0
pytest-order==1.0.1
pytest-sugar==0.9.6
python-osc==1.8.0
pytz==2022.7
PyYAML==6.0
requests==2.28.1
ruamel.yaml==0.17.21
ruamel.yaml.clib==0.2.7
setuptools-scm==7.1.0
six==1.16.0
snowballstemmer==2.2.0
Sphinx==6.1.3
sphinx-intl==2.0.1
sphinx-rtd-theme==0.5.1
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
sphinxcontrib.applehelp==1.0.3
termcolor==2.2.0
tomli==2.0.1
tox==3.28.0
tox-conda==0.10.1
typing_extensions==4.4.0
urllib3==1.26.14
virtualenv==20.17.1
zipp==3.11.0
  1. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix. https://github.com/ponoor/python-step-series/commit/965981f060941e7c32f076c748832a91b1763d41

  2. What commands did you run? Through tox, coverage run --context={envname} -m pytest. Replace {envname} with py37, py38, py39, or py310.

Expected behavior The HTML report to just show one entry for each source file.

Additional context I use coverage through tox and tox-conda. This means my separate environments are separate conda environments in a .tox folder. This allows me to easily test against different versions of Python.

To reproduce, simply install Miniconda or Anaconda, then run the following:

  • conda create -n coverage-issue python=3.9 -y
  • conda activate coverage-issue
  • pip install -r requirements.txt
  • tox

You may see tests fail, in which case, just comment those out if it impedes reproducing this issue. I'm currently reworking parts of this library, so excuse the current transitory state of the codebase.

My coverage config lives at the bottom of setup.cfg.

dakotahorstman avatar Jan 11 '23 23:01 dakotahorstman