Investigate using cg_clif for debug builds in CI

[passcod]

Some benchmarking shows a significant speedup, from 30s on nightly to 23s on cg_clif. I'm doing some more benchmarking to check the numbers further.

[passcod]

$ hyperfine -p 'cargo clean' -p 'cargo +nightly-2022-07-18 clean' -p '/home/Downloads/cg_clif/cargo-clif clean' 'cargo build' 'cargo +nightly-2022-07-18 build' '/home/Downloads/cg_clif/cargo-clif build'
Benchmark 1: cargo build
  Time (mean ± σ):     34.141 s ±  1.219 s    [User: 220.332 s, System: 18.720 s]
  Range (min … max):   31.949 s … 36.452 s    10 runs

Benchmark 2: cargo +nightly-2022-07-18 build
  Time (mean ± σ):     30.792 s ±  1.525 s    [User: 202.360 s, System: 17.527 s]
  Range (min … max):   29.376 s … 33.309 s    10 runs

Benchmark 3: /home/Downloads/cg_clif/cargo-clif build
  Time (mean ± σ):     22.960 s ±  0.348 s    [User: 109.599 s, System: 12.014 s]
  Range (min … max):   22.302 s … 23.339 s    10 runs

Summary
  '/home/Downloads/cg_clif/cargo-clif build' ran
    1.34 ± 0.07 times faster than 'cargo +nightly-2022-07-18 build'
    1.49 ± 0.06 times faster than 'cargo build'

That particular nightly is the same nightly that the clif tooling uses.

[passcod]

Trying out release builds too (all on my 12-thread x86_64 Linux):

Toolchain	Profile	Time	Size
stable	debug	34s	173MB
stable	release	58s	4.7MB
nightly-2022-07-18	debug	31s	171MB
nightly-2022-07-18	release	62s	4.7MB
cg_clif-2022-07-18	debug	23s	76MB
cg_clif-2022-07-18	release	23s	30MB

cg_clif doesn't support LTO, either

So we could build in release mode with cg_clif in CI for testing, and do (llvm) nightly builds in release mode for the releases

[NobodyXu]

Amazing, I didn't think of using an alternative codegen.

So we could build in release mode with cg_clif in CI for testing, and do nightly builds in release mode for the releases

I think building in debug mode with cg_clif in CI is enough for testing, we don't need to build in release mode just for testing.

As for the actual release, IMO it is better to keep using the llvm codegen with release mode, as LTO can make the binary smaller.

[passcod]

I think building in debug mode with cg_clif in CI is enough for testing, we don't need to build in release mode just for testing.

We don't need to, but given the timing it's essentially free.

Agree on keeping LLVM for actual releases.

---

Now the second part of this investigation is how to use clif in CI in a reasonable manner...

Draft workflow would be:

• Query GHAs on https://github.com/bjorn3/rustc_codegen_cranelift/actions
• Find the latest successful CI run
• Download the linux artifact via https://docs.github.com/en/rest/actions/artifacts#download-an-artifact
• Extract the archives (a tar.xz inside a zip)
• Cache some of that somehow
• Run

Maybe would be worth making a custom action instead...

[NobodyXu]

Maybe would be worth making a custom action instead...

I think it definitely should be make a custom action, though it also has to cooperate with cross.

[NobodyXu]

@passcod IMO the best way to implement this is to build an image on cross:

ARG CROSS_BASE_IMAGE
FROM $CROSS_BASE_IMAGE

RUN ...

This means that we only need to support for linux, specifically the images provided by cross and still can cross compile to any target (with zig-cc, we can truly build artifacts for any target llvm and rustc supports).

Edit: Checking the Dockerfiles of cross, they are all ubuntu:20.04 images so it should be pretty easy to support all of them.

The drawback is that now use-cross has to be set to true for all builds, but I don't think that is a big problem. Besides, building on windows and mac is so slow that I think using cross might actually speed them up.

[NobodyXu]

I think building in debug mode with cg_clif in CI is enough for testing, we don't need to build in release mode just for testing.

We don't need to, but given the timing it's essentially free.

That sounds good, but then we have to manually enable these profile options:

debug = true
debug-assertions = true
overflow-checks = true
panic = 'unwind'

[passcod]

Oh, that's true. Might as well just stick with debug then.

[NobodyXu]

@passcod I think we can create a repository, which periodically builds cg_clif on zig image and publishes it.

The Dockerfile for zig is pretty straight forward and we can add a wrapper binary cargo that switchs to $cg_clif_dir/build/cargo-clif if an environment variable is present (and we need to configure it to pass through that env).

[NobodyXu]

P.S. using build-std feature on my MBA 2020 further trims 0.4M from release build:

cargo +nightly build -Z build-std=std,panic_abort -Z build-std-features=panic_immediate_abort --release --target aarch64-apple-darwin

This would be good for release build though it will spend more time building std without unwinding/backtracing.

[NobodyXu]

I thought we didn't need this back when the caching was fixed and the compilation time was reduced to 2-4m, but now it is spiking again.

So looks like we still need this.

[NobodyXu]

I wonder if it would make sense for us to build x86_64-unknown-windows-gnu for debug build instead of the msvc build cause building on windows is so slow.

[passcod]

I don't think we should test/develop on one target and release on another, no. There's also other issues with windows-gnu/MinGW. However, we could probably cross-compile the -msvc target...

[NobodyXu]

Progress report on rustc_codegen_cranelift (Okt 2022)

[NobodyXu]

Given that after #1184, CI now takes 11m just to shallow clone without enabling any optimization, I suppose that we don't want to use cg_clif for debug build in our CI anymore.