Reconsider code coverage approach
We can close this issue when we have documented why we should not do these things or we have implemented them.
With the code coverage build moving to GCB (Google Cloud Build, see #3880) it may be time to reconsider how we run this build. We use cargo tarpaulin because it was easy to set up, and was not too slow when compared to cargo llvm-cov. That may have changed, as the code size has grown.
We also exclude all the integration tests because we could not run them in GHA (GitHub Actions). That is no longer true.
In summary:
- [ ] Use
cargo llvm-covto run coverage builds. - [ ] Enable as many integration tests as possible as part of the coverage builds.
- [ ] Trim the list of directories excluded from code coverage analysis.
I ran an experiment in #3920 to use llvm-cov.
Pros
- Seems faster (5m vs. 7m, but that was a single run).
- Provides branch coverage, so lines with only a single branch tested appear as "partial".
Cons
- Includes the
testsmodule. In modules with a lot of tests, this makes the coverage look higher than it is. - It also shows a lot of false positives in
assert!()orassert_eq!()lines that do not fail.
The coverage for the tests modules is enough of a bummer that I don't feel like pushing for this change.