rules_py icon indicating copy to clipboard operation
rules_py copied to clipboard

[Bug]: py_test does not work with bazel coverage

Open jab opened this issue 1 year ago • 5 comments

What happened?

Using bazel coverage with an aspect py_test target always results in no coverage data being collected.

Version

Development (host) and target OS/architectures:

Output of bazel --version: aspect 5.10.1

Version of the Aspect rules, or other relevant rules from your WORKSPACE or MODULE.bazel file: Current latest releases. See repro linked below.

Language(s) and/or frameworks involved: Python

How to reproduce

Please see https://github.com/jab/aspect-py-test-coverage-bug for a minimal reproduction.

jab avatar Jun 18 '24 15:06 jab

https://github.com/aspect-build/rules_py/issues/303 seems related - I see you aren't using a py_pytest_main so I'm not sure if the workaround there is useful.

alexeagle avatar Jun 19 '24 14:06 alexeagle

I didn't use py_pytest_main so as to provide a more minimal reproduction. But I am interested in experimenting with py_pytest_main more at some point.

I did take a quick a look at #303, but since I prefer not to use pytest-cov, it didn't seem like it would apply to me. (I agree with the reasons in https://www.youtube.com/watch?v=sPgvHGkmd0U for preferring to use coverage.py to invoke my test runner directly, as bazel coverage does with py_test targets.)

jab avatar Jun 19 '24 16:06 jab

I happened to reproduce this today while working on the Python course and debugging for a client.

alexeagle avatar Jan 18 '25 00:01 alexeagle

I'm seeing no coverage data on 1.3.0, not using pytest. Using absltest. Do we have to use pytest to get coverage? I prefer to not use pytest because it makes test startup noticably slower.

garymm avatar Feb 03 '25 06:02 garymm

@garymm yes you're right, my change closing this issue is specific to pytest.

alexeagle avatar Feb 21 '25 02:02 alexeagle

I believe the pytest is also suffering from lack of coverage, created https://github.com/kwesolowski/rules_py/tree/repro/coverage-validation-test-clean as reproduction. Should i file new issue while keeping this for generic py_test

kwesolowski avatar Oct 20 '25 12:10 kwesolowski