code coverage is broken - details of the error

[sylvestre]

I am going to collect some finding of my work (@cakebaker probably has other).

• -Zprofile has been removed from rustc - https://github.com/rust-lang/rust/pull/131829 we should use -Cinstrument-coverage instead

• we have to switch to a different method described here https://github.com/mozilla/grcov/issues/1240 (cc @marco-c)

• we need to pass LLVM_PROFILE_FILE as a new env variable, which wasn't passed to some commands by ucmd() i fixed it in 0cd58c63cf29fd9bcc2e1b50a927d48de064bfa1 it was leaving some coverage artifacts in the pwd which was breaking some tests like ls. LLVM_PROFILE_FILE: "${{ github.workspace }}/build/coverage-%p-%m.profraw"

• because of the change of format, files have to be exported into a different directory to make sure they don't pollute the tests

• the coverage files are HUGE 56G /Users/runner/work/coreutils/coreutils/build/ on a mac it causes the linux machine to fails with System.IO.IOException: No space left on device : '/home/runner/runners/2.320.0/_diag/Worker_20241116-105056-utc.log' and doesn't show anything in the log

• it takes 16 minutes for grcov to process everything - it was a few seconds before

• During one of my numerous experiment, i got <LLVM Profile Warning: Unable to track new values: Running out of static counters. Consider using option -mllvm -vp-counters-per-site=<n> to allocate more value profile counters at compile time. despite passing -Clink-arg=-Wl,-mllvm -Clink-arg=-Wl,-vp-counters-per-site=6, still happened.

• Using cargo llvm-cov show-env like in https://github.com/foundationdb-rs/foundationdb-rs/blob/c6623a84a7f35597eb7f68357a3c9cb91831501e/.github/workflows/coverage.yml fails the same way

My experimentations: https://github.com/uutils/coreutils/pull/6850

[chadbrewbaker]

I was able to get reports (with branch coverage) by installing the https://github.com/taiki-e/cargo-llvm-cov tool. Should I add this to the developer docs?

cargo llvm-cov --html -p uu_sort
cargo llvm-cov --html run factor -- 1234567 

[sylvestre]

Oh, excellent news. Yes, it would be great :)

[sylvestre]

the %p in LLVM_PROFILE_FILE: "${{ github.workspace }}/build/coverage-%p-%m.profraw" might be the reason why we have some many files. “%p” expands out to the process ID.

[sylvestre]

https://github.com/mozilla/grcov/pull/1287 might be the fix

[dan-hipschman]

The coverage upload stopped working around Apr 7, 2025:

Here's some sample output from the CI job (the job passes even though upload fails):

==> Running upload-coverage
      ./codecov  upload-coverage -t <redacted> --git-service github --flag ubuntu_latest --gcov-executable gcov --name codecov-umbrella
info - 2025-04-25 22:24:30,323 -- ci service found: github-actions
warning - 2025-04-25 22:24:30,348 -- xcrun is not installed or can't be found.
warning - 2025-04-25 22:24:30,402 -- No gcov data found.
warning - 2025-04-25 22:24:30,402 -- coverage.py is not installed or can't be found.
info - 2025-04-25 22:24:30,490 -- Found 0 coverage files to report
Error: No coverage reports found. Please make sure you're generating reports successfully.

It looks like this may be a known issue since there's some recent work being done on it, but I didn't see any issues reported about the upload failure so thought I'd comment here in case it was missed.

[sylvestre]

it is now back https://app.codecov.io/github/uutils/coreutils/

[cakebaker]

@sylvestre what did you do to make it work again?

[sylvestre]

@RenjiSann deserves all the credits: https://github.com/uutils/coreutils/pull/7401