Run doctests but do not include in coverage report

[hy144328]

I like to use function doctests as smoke tests. It makes nice documentation. And if the interface of my function changes, the failed doctest serves as a reminder to overhaul the entire docstring, e.g. args. In doctests, I usually set things up to execute the function in question but without any assertions; hence a smoke test. Making assertions is left to proper test_*.py modules.

Nevertheless, I do not want my smoke tests to count towards code coverage. For new pieces of code, I tend to forget to provide proper test_*.py modules when I see that the coverage is already at 100%. More concretely, I execute these two lines:

coverage run -m pytest
coverage report

pytest.ini:

[pytest]
addopts = --doctest-modules
testpaths =
    PACKAGE_FOLDER/
    tests/

Thus, I set up pytest such that it executes doctests in PACKAGE_FOLDER.

.coveragerc:

[report]
fail_under = 100
include = sfi_airflow_etl/*
show_missing = True

Is it possible to provide a [report] option such that doctests are ignored, or tests (but not source code) in certain paths are run but not reported? Thanks! 😄

[nedbat]

Coverage.py mostly doesn't know what tests are running, it just notes what lines of code get executed. If you can instruct pytest to skip doctests, you can run your tests in two phases: once without doctests with coverage, and ones with doctests without coverage.

Would that work for you?

[nedbat]

Feel free to re-open this when you come back to it.