cargo-binstall
cargo-binstall copied to clipboard
Investigate using cg_clif for debug builds in CI
Some benchmarking shows a significant speedup, from 30s on nightly to 23s on cg_clif. I'm doing some more benchmarking to check the numbers further.
$ hyperfine -p 'cargo clean' -p 'cargo +nightly-2022-07-18 clean' -p '/home/Downloads/cg_clif/cargo-clif clean' 'cargo build' 'cargo +nightly-2022-07-18 build' '/home/Downloads/cg_clif/cargo-clif build'
Benchmark 1: cargo build
Time (mean ± σ): 34.141 s ± 1.219 s [User: 220.332 s, System: 18.720 s]
Range (min … max): 31.949 s … 36.452 s 10 runs
Benchmark 2: cargo +nightly-2022-07-18 build
Time (mean ± σ): 30.792 s ± 1.525 s [User: 202.360 s, System: 17.527 s]
Range (min … max): 29.376 s … 33.309 s 10 runs
Benchmark 3: /home/Downloads/cg_clif/cargo-clif build
Time (mean ± σ): 22.960 s ± 0.348 s [User: 109.599 s, System: 12.014 s]
Range (min … max): 22.302 s … 23.339 s 10 runs
Summary
'/home/Downloads/cg_clif/cargo-clif build' ran
1.34 ± 0.07 times faster than 'cargo +nightly-2022-07-18 build'
1.49 ± 0.06 times faster than 'cargo build'
That particular nightly is the same nightly that the clif tooling uses.
Trying out release builds too (all on my 12-thread x86_64 Linux):
Toolchain | Profile | Time | Size |
---|---|---|---|
stable | debug | 34s | 173MB |
stable | release | 58s | 4.7MB |
nightly-2022-07-18 | debug | 31s | 171MB |
nightly-2022-07-18 | release | 62s | 4.7MB |
cg_clif-2022-07-18 | debug | 23s | 76MB |
cg_clif-2022-07-18 | release | 23s | 30MB |
cg_clif doesn't support LTO, either
So we could build in release mode with cg_clif in CI for testing, and do (llvm) nightly builds in release mode for the releases
Amazing, I didn't think of using an alternative codegen.
So we could build in release mode with cg_clif in CI for testing, and do nightly builds in release mode for the releases
I think building in debug mode with cg_clif
in CI is enough for testing, we don't need to build in release mode just for testing.
As for the actual release, IMO it is better to keep using the llvm
codegen with release mode, as LTO can make the binary smaller.
I think building in debug mode with cg_clif in CI is enough for testing, we don't need to build in release mode just for testing.
We don't need to, but given the timing it's essentially free.
Agree on keeping LLVM for actual releases.
Now the second part of this investigation is how to use clif in CI in a reasonable manner...
Draft workflow would be:
- Query GHAs on https://github.com/bjorn3/rustc_codegen_cranelift/actions
- Find the latest successful
CI
run - Download the linux artifact via https://docs.github.com/en/rest/actions/artifacts#download-an-artifact
- Extract the archives (a tar.xz inside a zip)
- Cache some of that somehow
- Run
Maybe would be worth making a custom action instead...
Maybe would be worth making a custom action instead...
I think it definitely should be make a custom action, though it also has to cooperate with cross
.
@passcod IMO the best way to implement this is to build an image on cross
:
ARG CROSS_BASE_IMAGE
FROM $CROSS_BASE_IMAGE
RUN ...
This means that we only need to support for linux, specifically the images provided by cross
and still can cross compile to any target (with zig-cc
, we can truly build artifacts for any target llvm
and rustc
supports).
Edit: Checking the Dockerfile
s of cross, they are all ubuntu:20.04
images so it should be pretty easy to support all of them.
The drawback is that now use-cross
has to be set to true
for all builds, but I don't think that is a big problem.
Besides, building on windows and mac is so slow that I think using cross
might actually speed them up.
I think building in debug mode with cg_clif in CI is enough for testing, we don't need to build in release mode just for testing.
We don't need to, but given the timing it's essentially free.
That sounds good, but then we have to manually enable these profile options:
debug = true
debug-assertions = true
overflow-checks = true
panic = 'unwind'
Oh, that's true. Might as well just stick with debug then.
@passcod I think we can create a repository, which periodically builds cg_clif
on zig
image and publishes it.
The Dockerfile
for zig
is pretty straight forward and we can add a wrapper binary cargo
that switchs to $cg_clif_dir/build/cargo-clif
if an environment variable is present (and we need to configure it to pass through that env).
P.S. using build-std
feature on my MBA 2020 further trims 0.4M from release build:
cargo +nightly build -Z build-std=std,panic_abort -Z build-std-features=panic_immediate_abort --release --target aarch64-apple-darwin
This would be good for release build though it will spend more time building std without unwinding/backtracing.
I thought we didn't need this back when the caching was fixed and the compilation time was reduced to 2-4m, but now it is spiking again.
So looks like we still need this.
I wonder if it would make sense for us to build x86_64-unknown-windows-gnu
for debug build instead of the msvc build cause building on windows is so slow.
I don't think we should test/develop on one target and release on another, no. There's also other issues with windows-gnu/MinGW. However, we could probably cross-compile the -msvc target...
Given that after #1184, CI now takes 11m just to shallow clone without enabling any optimization, I suppose that we don't want to use cg_clif
for debug build in our CI anymore.