rustc_codegen_cranelift icon indicating copy to clipboard operation
rustc_codegen_cranelift copied to clipboard

Published 20 hours ago • verified publisher icon rust-lang

Test rustc bootstrapping on AArch64

Open bjorn3 opened this issue 4 years ago • 14 comments

bjorn3 avatar Jul 02 '21 13:07 bjorn3

Hey, I mentioned I had an issue natively running test_rustc_tests.sh but the same issue presents when running test_bootstrap.sh.

The issue shows up when compiling the byteorder-1.3.4 crate.

   Compiling proc-macro-error v1.0.4
error: failed to run custom build command for `byteorder v1.3.4`

Caused by:
  process didn't exit successfully: `/home/ubuntu/git/rustc_codegen_cranelift/rust/build/bootstrap/debug/build/byteorder-80e24b4ddeadaf84/build-script-build` (signal: 6, SIGABRT: process abort signal)
  --- stderr
  thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/byteorder-1.3.4/build.rs:54:14
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed
failed to run: /home/ubuntu/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/bin/cargo build --manifest-path /home/ubuntu/git/rustc_codegen_cranelift/rust/src/bootstrap/Cargo.toml
Build completed unsuccessfully in 0:00:33

Here's the offending line.

This is a bit weird because running --version on cg_clif works:

ubuntu@ubuntu:~/git/rustc_codegen_cranelift$ ./build/bin/cg_clif --version
rustc 1.55.0-nightly (6d820866a 2021-06-29)

Even if i set $RUSTC to cg_clif I get the same error:

ubuntu@ubuntu:~/git/rustc_codegen_cranelift$ RUSTC="/home/ubuntu/git/rustc_codegen_cranelift/build/bin/cg_clif" ./scripts/test_bootstrap.sh
...
same error as above

Am I missing anything?

afonso360 avatar Jul 02 '21 14:07 afonso360

Isn't this the offending line: https://github.com/BurntSushi/byteorder/blob/eaaad370715c04d78334d6a5b49c637549c84c71/build.rs#L54 (line 54, not 45) Doesn't look like a None unwrap should be possible at that point though. Can you get a backtrace?

bjorn3 avatar Jul 02 '21 16:07 bjorn3

I just finished compiling cg_clif for aarch64 using qemu. The panic happens in build_script_build::Version::parse. It doesn't like the cranelift version in the cg_clif -vV output I guess.

bjorn3 avatar Jul 02 '21 18:07 bjorn3

Oh, right, I think I got mixed up, because the issue I was having in test_rustc_tests.sh was about the --version flag not giving us anything.

It is indeed in line 54. Here's the backtrace:

RUST_BACKTRACE=1 ./scripts/test_bootstrap.sh
...
   Compiling crossbeam-utils v0.8.3
error: failed to run custom build command for `byteorder v1.3.4`

Caused by:
  process didn't exit successfully: `/home/ubuntu/git/rustc_codegen_cranelift/rust/build/bootstrap/debug/build/byteorder-80e24b4ddeadaf84/build-script-build` (signal: 6, SIGABRT: process abort signal)
  --- stderr
  thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/byteorder-1.3.4/build.rs:54:14
  stack backtrace:
     0: rust_begin_unwind
               at /home/ubuntu/git/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/std/src/panicking.rs:515:5
     1: core::panicking::panic_fmt
               at /home/ubuntu/git/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/core/src/panicking.rs:92:14
     2: core::panicking::panic
               at /home/ubuntu/git/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/core/src/panicking.rs:50:5
     3: core::str::slice_error_fail
               at /home/ubuntu/git/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/core/src/option.rs:388:21
     4: core::str::traits::<impl core::slice::index::SliceIndex<str> for core::ops::range::RangeFrom<usize>>::index
               at /home/ubuntu/git/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/core/src/str/traits.rs:365:21
     5: core::str::traits::<impl core::ops::index::Index<I> for str>::index
               at /home/ubuntu/git/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/core/src/str/traits.rs:64:9
     6: build_script_build::Version::parse
               at ./build.rs:54:14
     7: build_script_build::Version::read
               at ./build.rs:47:9
     8: build_script_build::main
               at ./build.rs:10:25
     9: core::ops::function::FnOnce::call_once
               at /home/ubuntu/git/rustc_codegen_cranelift/build_sysroot/sysroot_src/library/core/src/ops/function.rs:227:5
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
warning: build failed, waiting for other jobs to finish...
error: build failed
failed to run: /home/ubuntu/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/bin/cargo build --manifest-path /home/ubuntu/git/rustc_codegen_cranelift/rust/src/bootstrap/Cargo.toml
Build completed unsuccessfully in 0:00:07

afonso360 avatar Jul 02 '21 18:07 afonso360

It doesn't like the cranelift version in the cg_clif -vV output I guess.

Never mind. That only happens when you explicitly pass -Zcodegen-backend together with the -vV.

bjorn3 avatar Jul 02 '21 18:07 bjorn3

The error seems to happen at https://github.com/rust-lang/rust/blob/851c82e88ade86bfe3b4ee785d5e5ab1d954b61c/library/core/src/str/mod.rs#L112 suggesting a non-utf8 string.

bjorn3 avatar Jul 02 '21 19:07 bjorn3

With a debug sysroot:

   Compiling byteorder v1.3.4 (/home/bjorn/Projects/cg_clif2/byteorder)
error: failed to run custom build command for `byteorder v1.3.4 (/home/bjorn/Projects/cg_clif2/byteorder)`

Caused by:
  process didn't exit successfully: `/home/bjorn/Projects/cg_clif2/byteorder/target/debug/build/byteorder-25926a0b98921ea4/build-script-build` (signal: 15, SIGTERM: termination signal)
  --- stderr
  thread 'main' panicked at 'byte index 6 is not a char boundary; it is inside 'r' (bytes 0..1) of `rustc 1.55.0-nightly (6d820866a 2021-06-29)
  `', build.rs:54:14
  stack backtrace:
  thread 'main' panicked at 'byte index 7 is not a char boundary; it is inside '.' (bytes 0..1) of `.debug_addr`', sysroot_src/library/std/src/../../backtrace/src/symbolize/gimli/elf.rs:121:26

and then it hangs. Couldn't even see this until I killed qemu.

bjorn3 avatar Jul 02 '21 19:07 bjorn3

Hmm, I modified the build.rs file to include a dbg!(s). It seems to be correct:

[build.rs:51] s = "rustc 1.55.0-nightly (6d820866a 2021-06-29)\n"

afonso360 avatar Jul 02 '21 19:07 afonso360

Here's a minimal repro case:

fn main() {
    let s = "rustc 1.55.0-nightly (6d820866a 2021-06-29)\n";
    &s[6..];
}

This is enough to crash:

ubuntu@ubuntu:~/git/rustc_codegen_cranelift$ ./build/bin/cg_clif ./test.rs
ubuntu@ubuntu:~/git/rustc_codegen_cranelift$ ./test
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', ./test.rs:3:6
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Aborted

Edit: We are misscompiling str::is_char_boundary

fn main() {
    let s = "rustc 1.55.0-nightly (6d820866a 2021-06-29)\n";
    println!("{}", s.is_char_boundary(6));
}

ubuntu@ubuntu:~/git/rustc_codegen_cranelift$ rustc ./test.rs
ubuntu@ubuntu:~/git/rustc_codegen_cranelift$ ./test
true
ubuntu@ubuntu:~/git/rustc_codegen_cranelift$ ./build/bin/cg_clif ./test.rs
ubuntu@ubuntu:~/git/rustc_codegen_cranelift$ ./test
false

Edit 2:

Narrowing it down, it seems that we are miss compiling a comparison after a dereference. I'm going to try to get a clif test file out of this.

fn main() {
        dbg!({
                let a: i32 = 49;
                let a_ref = &a;
                (*a_ref as i8) >= -0x40
        });
        dbg!((49 as i8) >= -0x40);
}

ubuntu@ubuntu:~/git/rustc_codegen_cranelift$ ./build/bin/cg_clif ./test.rs
ubuntu@ubuntu:~/git/rustc_codegen_cranelift$ ./test
[./test.rs:2] { let a: i32 = 49; let a_ref = &a; (*a_ref as i8) >= -0x40 } = false
[./test.rs:7] (49 as i8) >= -0x40 = true
ubuntu@ubuntu:~/git/rustc_codegen_cranelift$ rustc ./test.rs
ubuntu@ubuntu:~/git/rustc_codegen_cranelift$ ./test
[./test.rs:2] { let a: i32 = 49; let a_ref = &a; (*a_ref as i8) >= -0x40 } = true
[./test.rs:7] (49 as i8) >= -0x40 = true

Edit 3: Here's the clif test file:

test run
target aarch64
target x86_64 machinst

function %test() -> b1 {
    ss0 = explicit_slot 16

block0:
    v0 = iconst.i32 49
    stack_store v0, ss0
    v1 = stack_addr.i64 ss0
    v2 = load.i32 notrap v1
    v3 = ireduce.i8 v2
    v4 = iconst.i8 192
    v5 = icmp sge v3, v4
    return v5
}
; run: %test() == true

This passes on x64 and fails on aarch64. I'll keep working on this tommorow.

afonso360 avatar Jul 02 '21 19:07 afonso360

Thanks for the debugging!

bjorn3 avatar Jul 03 '21 09:07 bjorn3

Now getting

   Compiling merge_derive v0.1.0
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
error: could not compile `merge_derive`

Caused by:
  process didn't exit successfully: `/home/bjorn/Projects/cg_clif2/rust/../build/bin/cg_clif --crate-name merge_derive --edition=2018 /home/bjorn/.cargo/registry/src/github.com-1ecc6299db9ec823/merge_derive-0.1.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type proc-macro --emit=dep-info,link -C prefer-dynamic -C embed-bitcode=no -C debuginfo=2 -C metadata=9ead013e1ae016e2 -C extra-filename=-9ead013e1ae016e2 --out-dir /home/bjorn/Projects/cg_clif2/rust/build/bootstrap/debug/deps -C linker=aarch64-linux-gnu-gcc -L dependency=/home/bjorn/Projects/cg_clif2/rust/build/bootstrap/debug/deps --extern proc_macro_error=/home/bjorn/Projects/cg_clif2/rust/build/bootstrap/debug/deps/libproc_macro_error-d02fc3c73f17a371.rlib --extern proc_macro2=/home/bjorn/Projects/cg_clif2/rust/build/bootstrap/debug/deps/libproc_macro2-de20e217372061ee.rlib --extern quote=/home/bjorn/Projects/cg_clif2/rust/build/bootstrap/debug/deps/libquote-51534bb9a71bfbd9.rlib --extern syn=/home/bjorn/Projects/cg_clif2/rust/build/bootstrap/debug/deps/libsyn-f967af3d96e2e23c.rlib --extern proc_macro --cap-lints allow -Cdebuginfo=2 -Wrust_2018_idioms -Wunused_lifetimes -Wsemicolon_in_expressions_from_macros` (signal: 11, SIGSEGV: invalid memory reference)

bjorn3 avatar Jul 04 '21 16:07 bjorn3

This seems to be an abi incompatibility with cg_llvm:

Backtrace 
#0  0x000000400827a204 in ?? () from /usr/aarch64-linux-gnu/lib/libc.so.6
#1  0x0000004019b256e4 in core::slice::<impl [T]>::copy_from_slice () at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/core/src/intrinsics.rs:1861
#2  0x0000004019b1d6ec in <u32 as proc_macro::bridge::rpc::DecodeMut<S>>::decode () at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/proc_macro/src/bridge/rpc.rs:39
#3  0x0000004019b17b20 in <core::num::nonzero::NonZeroU32 as proc_macro::bridge::rpc::DecodeMut<S>>::decode ()
    at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/proc_macro/src/bridge/rpc.rs:168
#4  0x0000004019b159fc in <proc_macro::bridge::client::TokenStream as proc_macro::bridge::rpc::DecodeMut<S>>::decode ()
    at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/proc_macro/src/bridge/client.rs:116
#5  0x0000004019b16bf8 in <(A,B) as proc_macro::bridge::rpc::DecodeMut<S>>::decode () at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/proc_macro/src/bridge/rpc.rs:183
#6  0x0000004019b15d64 in proc_macro::bridge::client::run_client::{{closure}}::{{closure}} ()
    at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/proc_macro/src/bridge/client.rs:372
#7  0x0000004019b1c100 in proc_macro::bridge::scoped_cell::ScopedCell<T>::set::{{closure}} ()
    at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/proc_macro/src/bridge/scoped_cell.rs:80
#8  0x0000004019b1c41c in proc_macro::bridge::scoped_cell::ScopedCell<T>::replace ()
    at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/proc_macro/src/bridge/scoped_cell.rs:75
#9  0x0000004019b1c088 in proc_macro::bridge::scoped_cell::ScopedCell<T>::set () at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/proc_macro/src/bridge/scoped_cell.rs:80
#10 0x0000004019b16914 in proc_macro::bridge::client::<impl proc_macro::bridge::Bridge>::enter::{{closure}} ()
    at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/proc_macro/src/bridge/client.rs:325
#11 0x0000004019b1bec4 in std::thread::local::LocalKey<T>::try_with () at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/std/src/thread/local.rs:399
#12 0x0000004019b1b3e0 in std::thread::local::LocalKey<T>::with () at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/std/src/thread/local.rs:375
#13 0x0000004019b19580 in proc_macro::bridge::client::<impl proc_macro::bridge::Bridge>::enter ()
    at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/proc_macro/src/bridge/client.rs:325
#14 0x0000004019b15c24 in proc_macro::bridge::client::run_client::{{closure}} () at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/proc_macro/src/bridge/client.rs:370
#15 0x0000004019b2386c in <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once ()
    at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/std/src/panic.rs:341
#16 0x0000004019b21364 in std::panicking::try::do_call () at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/std/src/panicking.rs:401
#17 0x0000004019b21170 in std::panicking::try () at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/std/src/panicking.rs:365
#18 0x0000004019b15b18 in proc_macro::bridge::client::run_client () at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/std/src/panic.rs:428
#19 0x0000004019b16048 in proc_macro::bridge::client::Client<fn(proc_macro::TokenStream,proc_macro::TokenStream) -> proc_macro::TokenStream>::expand2::run ()
    at /home/bjorn/Projects/cg_clif2/build_sysroot/sysroot_src/library/proc_macro/src/bridge/client.rs:425
#20 0x00000040054f48f0 in proc_macro::bridge::server::run_server () from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#21 0x00000040054d6c74 in <rustc_expand::proc_macro::AttrProcMacro as rustc_expand::base::AttrProcMacro>::expand ()
   from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#22 0x00000040054c4a5c in rustc_expand::expand::MacroExpander::fully_expand_fragment ()
   from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#23 0x00000040054c2630 in rustc_expand::expand::MacroExpander::expand_crate ()
   from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#24 0x0000004001b79d90 in rustc_session::utils::<impl rustc_session::session::Session>::time ()
   from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#25 0x0000004001b9d9b8 in rustc_interface::passes::configure_and_expand_inner ()
   from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#26 0x0000004001b77e44 in rustc_interface::passes::boxed_resolver::BoxedResolver::new ()
   from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#27 0x0000004001b83f60 in rustc_interface::queries::Queries::expansion ()
   from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#28 0x0000004001abba3c in rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter ()
   from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#29 0x0000004001a892ac in rustc_span::with_source_map () from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#30 0x0000004001abcc04 in rustc_interface::interface::create_compiler_and_run () from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#31 0x0000004001a8e3ec in scoped_tls::ScopedKey<T>::set () from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#32 0x0000004001abd740 in std::sys_common::backtrace::__rust_begin_short_backtrace () from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#33 0x0000004001a70fcc in core::ops::function::FnOnce::call_once{{vtable-shim}} () from /home/bjorn/.rustup/toolchains/nightly-2021-06-30-aarch64-unknown-linux-gnu/lib/librustc_driver-5839d4fe983c4c9c.so
#34 0x0000004008100674 in <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once () at /rustc/6d820866a27b1949e237be79b9c8c0145fe728b7/library/alloc/src/boxed.rs:1575
#35 <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once () at /rustc/6d820866a27b1949e237be79b9c8c0145fe728b7/library/alloc/src/boxed.rs:1575
#36 std::sys::unix::thread::Thread::new::thread_start () at library/std/src/sys/unix/thread.rs:72
#37 0x00000040083867e4 in start_thread () from /usr/aarch64-linux-gnu/lib/libpthread.so.0
#38 0x00000040082c698c in ?? () from /usr/aarch64-linux-gnu/lib/libc.so.6

bjorn3 avatar Jul 04 '21 16:07 bjorn3

At the moment of the crash the register state is:

x0             0x4008d61820        275026155552
x1             0x910003fda9bf7bfd  -7998388550200493059
x2             0x4                 4

For a memcpy that represents the dst, src and count arguments. They may have already been clobbered by this point though. I can't seem to get attaching a debugger to qemu emulated programs working natively though, so I can't really tell if it was clobbered or not.

bjorn3 avatar Jul 04 '21 17:07 bjorn3

With de7342b4446476a2d9375a7f93be54dc9f8707d1 and 244455d8e3137f3bd3e2ee6a49ebb787e7d7f792, all rustc tests now pass on AArch64 with the Cranelift main branch. Bootstrapping rustc also works.

bjorn3 avatar Sep 01 '22 16:09 bjorn3