rules_jest icon indicating copy to clipboard operation
rules_jest copied to clipboard

[Bug]: `bazel coverage` doesn't report coverage correctly when dealing with generated files

Open Aghassi opened this issue 1 year ago • 1 comments

What happened?

We currently run bazel coverage on all our tests. The input to our tests are cjs files that come from ts. These files are generated with rules_swc and have maps. Right now, the jest config hard codes the following https://github.com/aspect-build/rules_jest/blob/main/jest/private/jest_config_template.mjs#L174-L179

However this is wrong for two reasons

  1. We are passing in generated files, which bazel won't try and look for because it only collects coverage on source files. So ideally rules_jest needs to remap the extensions for collected files based on the sourcemap in the files
  2. It is missing transitive file information. rules_go for example does this https://github.com/bazelbuild/rules_go/blob/0f1e0ae83d2a6f0746b9c083880b5c2239e3256f/go/private/rules/test.bzl#L187

What this means is we get coverage for only files in folders that have test files AND are imported by the test files. But we don't get coverage from files in the same folder that have no test files. This leads to an inaccurate representation of coverage.

Version

Development (host) and target OS/architectures:

Output of bazel --version: 7.1.1

Version of the Aspect rules, or other relevant rules from your WORKSPACE or MODULE.bazel file:

Language(s) and/or frameworks involved:

How to reproduce

I'm going to describe it beccause I don't have a public repro

Have source files which are ts and have them transformed to .js or .cjs using either rules_swc or ts_project with source maps. Take the output of that target and put them into the rules_jest with a custom config. have a three files in the test folder

  1. a source file
  2. a test file that references the source file
  3. another source file that is not tested whatsoever

Lastly run bazel coverage with

bazel coverage --combined_report=lcov //<project>/... --instrumentation_filter="^//" --test_output=all

Any other information?

No response

Aghassi avatar Apr 19 '24 21:04 Aghassi

I also want to add to this that the rootDir has to be set relative to where the package.json not relative to the test target or you end up only collecting coverage on what is visible from that package or lower. In a JS world, it's possible you import something from a sibling folder and that folder may not be under your pwd when you have something like gazelle running and generating build files.

Aghassi avatar Apr 24 '24 21:04 Aghassi