`cargo build` support in `cargo-llvm-cov`

[psandana]

Request This is a request to consider calling cargo build from within cargo-llvm-cov.

Reasoning CI systems are mostly building dev profile and running tests with coverage. As cargo-llvm-cov introduces special setup for tests, dev builds intermediate and final objects cannot be resused by the pipeline, implying recompilation of the whole codebase. This waste time and resources. This is more critical on CI systems, where building can take an order of tens of minutes.

If we could execute cargo build from within cargo-llvm-cov we could reuse dev profile builds for the test execution, saving important time ⌚ and CPU (think greener 🍃!).

I already tried using caching steps for our target folder in our pipeline, but even for that, we require twice the storage to host build dev and cargo llvm-cov test caches.

Solution A solution for this, would make the following call valid:

cargo llvm-cov build

All arguments for cargo build should be available and pass it down to cargo build.

Extra This solution may also work for complex integration tests, which can make use of the binaries generated in special environments (for instance, a micro-services test deployment).

[taiki-e]

Unfortunately, the proposed API does not work well because specific environment variables must also be set at runtime.

The way it is currently available for this kind of use case is to use show-env.

[wiktor-k]

I've got a similar (but not identical?) use-case: I'd like to grab the binary that cargo llvm-cov run produces and run it in a container (that wouldn't need Rust tools or llvm-preview tools) and then extract the profiling files from that container. (The need for an additional container is due to integration tooling that's hard to replicate on the host system).

If I'm understanding it correctly cargo llvm-cov run already produces some kind of binary that then gets executed with correct environment variables (e.g. LLVM_PROFILE_FILE) so in theory it would be possible (correct me if I'm wrong here)... That's what I'd understand by cargo llvm-cov build - building a binary with instrumentation that when run collects the profiling data.

I hope this doesn't sound too crazy... but if so... I'd be glad to hear your opinion 😅

Thanks for your time! 👋

[taiki-e]

@wiktor-k Does cargo llvm-cov run --no-report + cargo llvm-cov report not work?

If you want to do something like cargo llvm-cov run --no-run, show-env mentioned above is the right way to go.

[psandana]

@wiktor-k Does cargo llvm-cov run --no-report + cargo llvm-cov report not work?

If you want to do something like cargo llvm-cov run --no-run, show-env mentioned above is the right way to go.

Not sure if the same than @wiktor-k , but on my case you need to even set the target folder, so artifacts can be reused. Moreover, any addition of any other flag or change would not be immediately compatible with the build.

For instance, if I set manually RUSTFLAGS = "-C instrument-coverage", build will execute with -C instrument-coverage, and cargo llvm-cov will execute with -C instrument-coverage -C instrument-coverage (duplicated), which invalidates previous compiled targets.

It is so painful to maintain. And seems so easy to just cargo llvm-cov build for convenience.

[wiktor-k]

If you want to do something like cargo llvm-cov run --no-run, show-env mentioned above is the right way to go.

That's exactly what I wanted. I guess I was just confused why (or even if) cargo build now includes instrumentation and overlooked that part of envs: RUSTFLAGS='-C instrument-coverage' (I also didn't know show-env existed and reverse-engineered the other variables 🤦 ).

I think my case is now closed (if you don't mind I'll file a tiny improvement to the README).

Thanks for help, folks! 👋