google-cloud-rust icon indicating copy to clipboard operation
google-cloud-rust copied to clipboard

Reconsider code coverage approach

Open coryan opened this issue 1 month ago • 1 comments

We can close this issue when we have documented why we should not do these things or we have implemented them.

With the code coverage build moving to GCB (Google Cloud Build, see #3880) it may be time to reconsider how we run this build. We use cargo tarpaulin because it was easy to set up, and was not too slow when compared to cargo llvm-cov. That may have changed, as the code size has grown.

We also exclude all the integration tests because we could not run them in GHA (GitHub Actions). That is no longer true.

In summary:

  • [ ] Use cargo llvm-cov to run coverage builds.
  • [ ] Enable as many integration tests as possible as part of the coverage builds.
  • [ ] Trim the list of directories excluded from code coverage analysis.

coryan avatar Nov 22 '25 16:11 coryan

I ran an experiment in #3920 to use llvm-cov.

Pros

  • Seems faster (5m vs. 7m, but that was a single run).
  • Provides branch coverage, so lines with only a single branch tested appear as "partial".

Cons

  • Includes the tests module. In modules with a lot of tests, this makes the coverage look higher than it is.
  • It also shows a lot of false positives in assert!() or assert_eq!() lines that do not fail.

The coverage for the tests modules is enough of a bummer that I don't feel like pushing for this change.

coryan avatar Nov 25 '25 19:11 coryan