Why is coverage increasing when using doctest modules, even when no doctests exist?

[caldwellst]

I am trying to better understand why coverage increases when using --doctest-modules and pytest-cov, even when there are no docstrings, let alone doctest examples within the docstring. Here's a simple module that reproduces my problem.

temp
│   setup.cfg
│
└───src
    │  tmp.py

tmp.py has the simple structure:

def tempo(x):
    return x

And when setup.cfg is empty and run pytest --cov=src, I get:

============================= test session starts ==============================
platform darwin -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /Users/caldwellst/Desktop/temp
plugins: doctestplus-0.11.0, anyio-3.3.2, cov-3.0.0
collected 0 items
/Users/caldwellst/.pyenv/versions/3.9.7/lib/python3.9/site-packages/coverage/control.py:761: CoverageWarning: No data was collected. (no-data-collected)
  self._warn("No data was collected.", slug="no-data-collected")


---------- coverage: platform darwin, python 3.9.7-final-0 -----------
Name         Stmts   Miss  Cover
--------------------------------
src/tmp.py       2      2     0%
--------------------------------
TOTAL            2      2     0%

However, if I run it with --doctest-modules, I get:

============================= test session starts ==============================
platform darwin -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /Users/caldwellst/Desktop/temp
plugins: doctestplus-0.11.0, anyio-3.3.2, cov-3.0.0
collected 0 items


---------- coverage: platform darwin, python 3.9.7-final-0 -----------
Name         Stmts   Miss  Cover
--------------------------------
src/tmp.py       2      1    50%
--------------------------------
TOTAL            2      1    50%

In more complex instances, I am getting 100% coverage on files that get 0% without using doctest (and have no docstring examples). What could the cause of this be? I would expect the behavior to have the same coverage, except for .py files that explicitly have examples in the docstring. However, am I misunderstanding what should be happening?

[nedbat]

In the first case, you are getting a warning that no data was collected. pytest finds no code to run, so it never executes src/tmp.py. You can verify this with:

COVERAGE_DEBUG=trace COVERAGE_DEBUG_FILE=/tmp/dbg.out pytest --cov=src

This writes all of the trace/no-trace decisions to /tmp/dbg.out. src/tmp.py isn't mentioned at all. It was never executed.

I'm not familiar with --doctest-modules, but my guess is that it is finding and importing modules itself, to find docstrings. This runs src/tmp.py, which then gets you the 50% coverage. You can use COVERAGE_DEBUG=trace again to see that the two files in src are now being considered and traced.

[jaraco]

Just to confirm - I run with --doctest-modules as a matter of course to get this coverage. That is, even if no doctests are run, if the module is executed, that proves that the module has no syntax errors and any behavior on import is not failing.