rustc_codegen_cranelift icon indicating copy to clipboard operation
rustc_codegen_cranelift copied to clipboard

Support sccache

Open mkpankov opened this issue 5 years ago • 2 comments

Logs are below

➜  rustc_codegen_cranelift git:(master) ./test.sh --release
    Updating git repository `https://github.com/bjorn3/rust-ar.git`
    Updating git repository `https://github.com/bytecodealliance/wasmtime/`
    Updating git submodule `https://github.com/WebAssembly/WASI`
    Updating git submodule `https://github.com/WebAssembly/wasm-c-api`
    Updating git submodule `https://github.com/WebAssembly/testsuite`
  Downloaded target-lexicon v0.11.0
  Downloaded region v2.2.0
  Downloaded regalloc v0.0.30
  Downloaded object v0.21.1
  Downloaded raw-cpuid v7.0.3
  Downloaded 5 crates (417.8 KB) in 1.00s
   Compiling autocfg v1.0.0
   Compiling proc-macro2 v1.0.19
   Compiling unicode-xid v0.2.1
   Compiling syn v1.0.38
   Compiling cfg-if v0.1.10
   Compiling cranelift-codegen-shared v0.66.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#1fabb051)
   Compiling log v0.4.11
   Compiling cranelift-entity v0.66.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#1fabb051)
   Compiling target-lexicon v0.11.0
   Compiling byteorder v1.3.4
   Compiling rustc-hash v1.1.0
   Compiling smallvec v1.4.1
   Compiling semver-parser v0.7.0
   Compiling bitflags v1.2.1
   Compiling cc v1.0.58
   Compiling anyhow v1.0.32
   Compiling libc v0.2.74
   Compiling crc32fast v1.2.0
   Compiling libloading v0.6.2
   Compiling ar v0.8.0 (https://github.com/bjorn3/rust-ar.git?branch=do_not_remove_cg_clif_ranlib#de9ab0e5)
   Compiling hashbrown v0.8.1
   Compiling indexmap v1.5.0
   Compiling cranelift-codegen-meta v0.66.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#1fabb051)
   Compiling semver v0.9.0
   Compiling cranelift-bforest v0.66.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#1fabb051)
   Compiling regalloc v0.0.30
   Compiling rustc_version v0.2.3
   Compiling quote v1.0.7
   Compiling raw-cpuid v7.0.3
   Compiling region v2.2.0
   Compiling errno v0.2.6
   Compiling gimli v0.21.0
   Compiling object v0.21.1
   Compiling thiserror-impl v1.0.20
   Compiling thiserror v1.0.20
   Compiling cranelift-codegen v0.66.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#1fabb051)
   Compiling cranelift-module v0.66.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#1fabb051)
   Compiling cranelift-native v0.66.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#1fabb051)
   Compiling cranelift-frontend v0.66.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#1fabb051)
   Compiling cranelift-object v0.66.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#1fabb051)
   Compiling cranelift-simplejit v0.66.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#1fabb051)
   Compiling rustc_codegen_cranelift v0.1.0 (/home/mkpankov/work/rustc_codegen_cranelift)
warning: unused variable: `clif_comments`
   --> src/optimize/stack2reg.rs:187:5
    |
187 |     clif_comments: &mut crate::pretty_clif::CommentWriter,
    |     ^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_clif_comments`
    |
    = note: `#[warn(unused_variables)]` on by default

warning: unused attribute
 --> src/target_features_whitelist.rs:1:1
  |
1 | #![rustfmt::skip]
  | ^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(unused_attributes)]` on by default

warning: 2 warnings emitted

    Finished release [optimized] target(s) in 1m 12s
rm: cannot remove 'target/out': No such file or directory
[BUILD] mini_core
[mini_core                     : codegen mono items] start
[mini_core                     : codegen mono items] end time: 2.791497ms
[BUILD] example
[example                       : codegen mono items] start
[example                       : codegen mono items] end time: 2.68377ms
[JIT] mini_core_hello_world
[mini_core_hello_world         : codegen mono items] start
[mini_core_hello_world         : codegen mono items] end time: 3.278508ms
Rustc codegen cranelift will JIT run the executable, because --jit was passed
mini_core_hello_world
abc
bcd
Hello printf
Hello
World!
abc
Boxed outer got dropped!
Inner got dropped!
Inner got dropped!
Inner got dropped!
[AOT] mini_core_hello_world
[mini_core_hello_world         : codegen mono items] start
[mini_core_hello_world         : codegen mono items] end time: 13.504106ms
./target/out/mini_core_hello_world
abc
bcd
Hello printf
Hello
World!
abc
Boxed outer got dropped!
Inner got dropped!
Inner got dropped!
Inner got dropped!
TLS works!

[AOT] arbitrary_self_types_pointers_and_wrappers
[arbitrary_self_types_pointers_and_wrappers: codegen mono items] start
[arbitrary_self_types_pointers_and_wrappers: codegen mono items] end time: 2.609972ms
[BUILD] sysroot
  Downloaded compiler_builtins v0.1.36
  Downloaded 1 crate (155.3 KB) in 0.66s
   Compiling compiler_builtins v0.1.36
   Compiling core v0.0.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/core)
   Compiling libc v0.2.77
   Compiling cc v1.0.60
   Compiling std v0.0.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/std)
   Compiling unwind v0.0.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/unwind)
   Compiling rustc-std-workspace-core v1.99.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/rustc-std-workspace-core)
[rustc_std_workspace_core      : codegen mono items] start
[rustc_std_workspace_core      : codegen mono items] end time: 170.003µs
[core                          : codegen mono items] start
   Compiling alloc v0.0.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/alloc)
   Compiling cfg-if v0.1.10
   Compiling adler v0.2.3
   Compiling rustc-demangle v0.1.16
[compiler_builtins             : codegen mono items] start
[cfg_if                        : codegen mono items] start
[cfg_if                        : codegen mono items] end time: 215.479µs
[adler                         : codegen mono items] start
[core                          : codegen mono items] end time: 450.069185ms
[adler                         : codegen mono items] end time: 21.694017ms
   Compiling panic_abort v0.0.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/panic_abort)
   Compiling alloc_system v0.0.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot/alloc_system)
[libc                          : codegen mono items] start
[libc                          : codegen mono items] end time: 14.713215ms
[compiler_builtins             : codegen mono items] end time: 139.730526ms
[unwind                        : codegen mono items] start
[unwind                        : codegen mono items] end time: 1.577243ms
[alloc_system                  : codegen mono items] start
[alloc_system                  : codegen mono items] end time: 2.227128ms
[panic_abort                   : codegen mono items] start
[panic_abort                   : codegen mono items] end time: 985.383µs
[rustc_demangle                : codegen mono items] start
[rustc_demangle                : codegen mono items] end time: 97.181898ms
   Compiling rustc-std-workspace-alloc v1.99.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/rustc-std-workspace-alloc)
   Compiling panic_unwind v0.0.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/panic_unwind)
[alloc                         : codegen mono items] start
   Compiling gimli v0.22.0
[rustc_std_workspace_alloc     : codegen mono items] start
   Compiling miniz_oxide v0.4.1
   Compiling hashbrown v0.9.0
   Compiling object v0.20.0
[rustc_std_workspace_alloc     : codegen mono items] end time: 173.654µs
[panic_unwind                  : codegen mono items] start
[panic_unwind                  : codegen mono items] end time: 6.666125ms
[alloc                         : codegen mono items] end time: 104.103099ms
[miniz_oxide                   : codegen mono items] start
[hashbrown                     : codegen mono items] start
[hashbrown                     : codegen mono items] end time: 1.491047ms
[miniz_oxide                   : codegen mono items] end time: 134.660778ms
   Compiling addr2line v0.13.0
[gimli                         : codegen mono items] start
[gimli                         : codegen mono items] end time: 128.485612ms
[addr2line                     : codegen mono items] start
[addr2line                     : codegen mono items] end time: 19.979057ms
[object                        : codegen mono items] start
[object                        : codegen mono items] end time: 422.485532ms
[std                           : codegen mono items] start
[std                           : codegen mono items] end time: 1.058374662s
   Compiling rustc-std-workspace-std v1.99.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/rustc-std-workspace-std)
   Compiling term v0.0.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/term)
   Compiling proc_macro v0.0.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/proc_macro)
[rustc_std_workspace_std       : codegen mono items] start
[rustc_std_workspace_std       : codegen mono items] end time: 165.193µs
   Compiling unicode-width v0.1.8
   Compiling getopts v0.2.21
[unicode_width                 : codegen mono items] start
[unicode_width                 : codegen mono items] end time: 207.121µs
[term                          : codegen mono items] start
[getopts                       : codegen mono items] start
[getopts                       : codegen mono items] end time: 130.32874ms
[term                          : codegen mono items] end time: 239.835485ms
[proc_macro                    : codegen mono items] start
warning: Argument of type `bridge::buffer::Slice<u8>` with pass mode `ByValPair(types::I64, types::I64)` is not yet supported for non-rust abi `"C"`. Calling this function may result in a crash.
   --> sysroot_src/library/proc_macro/src/bridge/buffer.rs:134:9
    |
134 |         extern "C" fn extend_from_slice<T: Copy>(b: Buffer<T>, xs: Slice<'_, T>) -> Buffer<T> {
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

warning: Argument of type `bridge::buffer::Slice<u8>` with pass mode `ByValPair(types::I64, types::I64)` is not yet supported for non-rust abi `"C"`. Calling this function may result in a crash.
  --> sysroot_src/library/proc_macro/src/bridge/buffer.rs:92:17
   |
92 |         *self = (b.extend_from_slice)(b, Slice::from(xs));
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[proc_macro                    : codegen mono items] end time: 335.946272ms
warning: 2 warnings emitted

   Compiling test v0.0.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/test)
[test                          : codegen mono items] start
[test                          : codegen mono items] end time: 509.866041ms
   Compiling sysroot v0.0.0 (/home/mkpankov/work/rustc_codegen_cranelift/build_sysroot)
[sysroot                       : codegen mono items] start
[sysroot                       : codegen mono items] end time: 147.123µs
    Finished release [optimized + debuginfo] target(s) in 25.80s

real	0m25.929s
user	0m32.438s
sys	0m1.699s
[AOT] alloc_example
[alloc_example                 : codegen mono items] start
[alloc_example                 : codegen mono items] end time: 4.396118ms
Hello World!
[JIT] std_example
[std_example                   : codegen mono items] start
[std_example                   : codegen mono items] end time: 89.669034ms
Rustc codegen cranelift will JIT run the executable, because --jit was passed
["/home/mkpankov/work/rustc_codegen_cranelift/target/release/cg_clif", "-L", "crate=target/out", "--out-dir", "target/out", "-Cdebuginfo=2", "--jit", "example/std_example.rs", "--target", "x86_64-unknown-linux-gnu"]
some <unknown> text
Hello from another thread!
cargo:rustc-link-lib=z
9.974182
4.9245777
2.3
1.5165751
2
3
1
2.3
5.29
1.2016338
5.29
Location { file: "example/std_example.rs", line: 98, col: 22 }
[example/std_example.rs:155] _mm_setr_epi8(255u8 as i8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = __m128i(
    255,
    0,
)
[AOT] dst_field_align
[dst_field_align               : codegen mono items] start
[dst_field_align               : codegen mono items] end time: 11.181634ms
[AOT] std_example
[std_example                   : codegen mono items] start
[std_example                   : codegen mono items] end time: 100.699522ms
["./target/out/std_example", "arg"]
some <unknown> text
Hello from another thread!
cargo:rustc-link-lib=z
9.974182
4.9245777
2.3
1.5165751
2
3
1
2.3
5.29
1.2016338
5.29
Location { file: "example/std_example.rs", line: 98, col: 22 }
[example/std_example.rs:155] _mm_setr_epi8(255u8 as i8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = __m128i(
    255,
    0,
)
[AOT] subslice-patterns-const-eval
[subslice_patterns_const_eval  : codegen mono items] start
[subslice_patterns_const_eval  : codegen mono items] end time: 24.559593ms
[AOT] track-caller-attribute
[track_caller_attribute        : codegen mono items] start
[track_caller_attribute        : codegen mono items] end time: 5.890734ms
[AOT] mod_bench
[mod_bench                     : codegen mono items] start
[mod_bench                     : codegen mono items] end time: 3.682622ms
~/work/rustc_codegen_cranelift/rand ~/work/rustc_codegen_cranelift
rm: cannot remove './target': No such file or directory
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/home/mkpankov/.cargo/bin/sccache /home/mkpankov/work/rustc_codegen_cranelift/target/release/cg_clif - --crate-name ___ --print=file-names --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (exit code: 2)
  --- stderr
  error: failed to execute compile
  caused by: Compiler not supported: "error: Unrecognized option: \'E\'\n\n"

Edit(bjorn3): Wrap logs in <details>.

mkpankov avatar Sep 30 '20 06:09 mkpankov

https://github.com/mozilla/sccache/blob/030c80a15e0b1aba0e61df264b320a21bb21fb15/src/compiler/compiler.rs#L875

sccache only recognizes rustc and clippy-driver as having a rustc like interface. In all other cases it will assume that it is a C compiler. I have pushed a commit to unset RUSTC_WRAPPER when sccache is detected. This should work around the problem.

bjorn3 avatar Sep 30 '20 07:09 bjorn3

I will repurpose this issue for finding a way to actually support sccache.

bjorn3 avatar Sep 30 '20 07:09 bjorn3

Partially fixed by https://github.com/bjorn3/rustc_codegen_cranelift/pull/1225. When using cargo-clif sccache is still not effective due to it not caching rustc invocations using --sysroot. This will not be an issue for a future rustup component. I don't think we need to keep tracking sccache support as at least it doesn't error anymore.

bjorn3 avatar Sep 01 '22 15:09 bjorn3