Coverage now fails if there are no covered Python files even if test succeeds

[phst]

🐞 bug report

Affected Rule

The issue is caused by the rule: py_test

Is this a regression?

Yes, the previous version in which this bug was not present was: 1.2.0

Description

If there are no covered source files, the coverage.py lcov step fails, causing the entire test to fail when run under coverage (see repro below).

🔬 Minimal Reproduction

See https://github.com/phst/py-cover-bug, and the next section.

🔥 Exception or Error

After cloning the above repository, run bazel coverage --test_output=all //a:test. This fails with an error

[...]
INFO: Using default value for --instrumentation_filter: "^//a[/:]".
[...]
INFO: From Testing //a:test:
==================== Test output for //a:test:
Hello world
[...]/coverage/inorout.py:504: CoverageWarning: Module [...]/_main/bazel-out/darwin_arm64-fastbuild/bin/b was never imported. (module-not-imported)
  self.warn(f"Module {pkg} was never imported.", slug="module-not-imported")
[...]/coverage/control.py:894: CoverageWarning: No data was collected. (no-data-collected)
  self._warn("No data was collected.", slug="no-data-collected")
--
Coverage runner: Not collecting coverage for failed test.
The following commands failed with status 1
[...]/bin/a/test.runfiles/_main/a/test
================================================================================
[...]

Running this with bazel test instead of coverage succeeds. Downgrading to rules_python 1.2.0 also makes it succeed.

Relevant additional output with verbose coverage enabled:

[coveragepy] Instrumented Files:
[...]/test.runfiles/_main/bazel-out/darwin_arm64-fastbuild/bin/b/bin
[...]/test.runfiles/_main/bazel-out/darwin_arm64-fastbuild/bin/b/binruntime_objects_list.txt
[coveragepy] Sources:
[...]/test.runfiles/_main/bazel-out/darwin_arm64-fastbuild/bin/b
Coverage entrypoint: [...]/coverage/__main__.py
[...]
Converting coveragepy database to lcov: /private/var/tmp/_bazel_p/35f69852d1c41a85f4769530320f22fe/sandbox/darwin-sandbox/26/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/_coverage/a/test/test/pylcov.dat
No data to report.
+ TEST_STATUS=1
[...]

Note that no directories containing Python files are listed in "Sources", so no coverage is generated for them. This is expected given the instrumentation filter, but causes coverage.py lcov to err out since it doesn't find any coverage data. My guess is that this is fallout from https://github.com/bazel-contrib/rules_python/pull/2607.

🌍 Your Environment

Operating System:

macOS

Output of bazel version:

8.1.1

Rules_python version:

1.3.0

Anything else relevant?

[BurnzZ]

Hi @phst ! Cheers for the report!

I can't seem to reproduce your logs, as I get this on mac and linux with bazel coverage --test_output=all //a:test --nocache_test_results:

INFO: Using default value for --instrumentation_filter: "^//a[/:]".
INFO: Override the above default with --instrumentation_filter
INFO: Analyzed target //a:test (83 packages loaded, 3921 targets configured).
INFO: From Testing //a:test:
==================== Test output for //a:test:
Hello world
Apr. 14, 2025 2:44:11 AM com.google.devtools.coverageoutputgenerator.Main runWithArgs
WARNING: There was no coverage found.
================================================================================
INFO: Found 1 test target...
Target //a:test up-to-date:
  bazel-bin/a/test
INFO: Elapsed time: 6.829s, Critical Path: 2.04s
INFO: 7 processes: 15 action cache hit, 5 internal, 2 darwin-sandbox.
INFO: Build completed successfully, 7 total actions
//a:test                                                                 PASSED in 1.9s

Executed 1 out of 1 test: 1 test passes

The response status code after running bazel coverage also looks good:

> echo $?
0

Reverting back to rules_python 1.2.0, 1.1.0, and 1.0.0 also gets the same result on my end.

---

I was wondering if you might have some env vars set that I'm missing?

[phst]

I tried to trim down my environment as much as possible and still see the issue:

env -i "HOME=$HOME" "PATH=/bin:/usr/bin" /opt/homebrew/bin/bazel --nohome_rc --nosystem_rc coverage --test_output=all --experimental_generate_llvm_lcov --nocache_test_results //a:test

[phst]

Maybe the code in https://github.com/bazel-contrib/rules_python/blob/1.5.3/python/private/python_bootstrap_template.txt#L460-L476 should be skipped if the previous step hasn't produced a .coverage file?

[phst]

Maybe the code in https://github.com/bazel-contrib/rules_python/blob/1.5.3/python/private/python_bootstrap_template.txt#L460-L476 should be skipped if the previous step hasn't produced a .coverage file?

OK, even that isn't enough because coverage run will generate a .coverage database in all cases, and coverage lcov fails unconditionally if the database is empty. The only way seems to be to ignore the return code of coverage lcov.