Undefined reference to `getauxval` in function `init_have_lse_atomics` when compiling to nightly `aarch64-unknown-linux-musl`

[XAMPPRocky]

Code

I received this pull request, which caught a regression for aarch64-unknown-linux-musl when compiling for nightly. The same process works for stable and beta. I don't have a machine that can easily compile to this platform, so I can't reduce it at the moment: https://github.com/XAMPPRocky/rasn/pull/49

Version it worked on

It most recently worked on: Works on stable and beta 1.56.0-beta.4 (e6e620e1c 2021-10-04)

• https://github.com/XAMPPRocky/rasn/pull/49/checks?check_run_id=3827011175

Version with regression

rustc --version --verbose: 1.57.0-nightly (0eabf25b9 2021-10-06)

• https://github.com/XAMPPRocky/rasn/pull/49/checks?check_run_id=3827012571

Backtrace

= note: /rust/lib/rustlib/aarch64-unknown-linux-musl/lib/libcompiler_builtins-cbf6a032efba8222.rlib(cpu_model.o): In function `init_have_lse_atomics':
          /cargo/registry/src/github.com-1ecc6299db9ec823/compiler_builtins-0.1.49/./lib/builtins/cpu_model.c:786: undefined reference to `getauxval'
          collect2: error: ld returned 1 exit status
          
= help: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
= note: use the `-l` flag to specify native libraries to link
= note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

[nbdd0121]

The reference is introduced in https://github.com/llvm/llvm-project/commit/a4ac434c47434d80bca54bab96f295ed4e972cc6#diff-a380831627112e2516d39595d94e711db7ac63316b9c411aa77af4a561d15ab8

[nbdd0121]

@rustbot label E-needs-bisection

[XAMPPRocky]

Tagging https://github.com/rust-lang/rust/issues/76992 and https://github.com/rust-lang/rust/pull/85805 since they seem relevant and maybe a potential fix.

[apiraino]

forcing notification for prioritization

@rustbot label -I-prioritize +T-compiler +I-prioritize

[tonyg]

See also https://github.com/rust-embedded/cross/issues/598.

(NB. that issue has a bit of waffley analysis, but also a workaround.)

[jbg]

Any ideas for the equivalent workaround when not using cross? aarch64-unknown-linux-musl is the host as well as the target here (CI build in an Alpine container) and I'm hitting this issue. I tried setting either RUSTFLAGS or CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_RUSTFLAGS to -C link-arg=/usr/lib/libc.a but it causes a ton of duplicate symbol errors.

[joseluisq]

@jbg I know that this issue is all about nightly Rust but we faced this via Cross (which BTW is not what you are using) but we switched back to stable 1.55 and our CI built again for now. I don't know if that helps you.

[jbg]

Thanks! this particular codebase has to use a gated feature so going back to stable isn't an option for now.

[tonyg]

@jbg Ah, that's interesting. For me, it looked like there was no libc being linked at all, so adding it to the end worked out well. For you, I'm guessing there's some reference to libc earlier in the linker command line. Unfortunately if I remember rightly the linker command-line is order-sensitive, with later mentions of object/library files supplying symbols needed by earlier files. But I thought also that mentioning a library was idempotent!

What are the duplicate symbols you see? Can you run your build with -vv to see what command-line rustc invocation you have? Finally, /usr/lib/libc.a isn't present on any of my systems: instead I need to supply /usr/lib/[SOMEARCH]/libc.a. Could varying the libc.a you supply work in your context?

[jbg]

I just noticed that the libc crate is in the dependency tree (transitively), and it seems to be the source of the duplicate symbols when I try to use the RUSTFLAGS="-C link-arg=/usr/lib/libc.a" workaround:

error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crt1.o" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crti.o" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtbegin.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.0.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.1.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.10.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.11.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.12.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.13.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.14.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.15.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.2.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.3.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.4.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.5.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.6.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.7.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.8.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.build_script_build.8cb445ef-cgu.9.rcgu.o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26.32sykodykbcu9la2.rcgu.o" "-Wl,--as-needed" "-L" "/target/debug/deps" "-L" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libstd-c44788b657ddddd5.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libpanic_unwind-2becdccda730ce42.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libminiz_oxide-3a7898da562cf313.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libadler-fc70ddb33855e2e8.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libobject-5fb8217618711e88.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libmemchr-c061be432db3e13d.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libaddr2line-5a6d99ea33f6ceae.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libgimli-6db2c8017781e41d.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libstd_detect-7d0d2f13c64acb61.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/librustc_demangle-77c2cc6c5af8536c.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libhashbrown-8ca99a806b6c7500.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/librustc_std_workspace_alloc-eced44336e84fd45.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libunwind-ec4fe46000190b79.rlib" "-lunwind" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libcfg_if-692ed2e2b4273428.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liblibc-72b57ab90668a509.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liballoc-7aed2b0a9d39bb09.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/librustc_std_workspace_core-e0db88e40d9c7e0b.rlib" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libcore-fcedc0d4b8cb02ca.rlib" "-Wl,--end-group" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/libcompiler_builtins-cbf6a032efba8222.rlib" "-Wl,-Bdynamic" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-nostartfiles" "-L" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib" "-L" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained" "-o" "/target/debug/build/libc-98210b609d8e5a26/build_script_build-98210b609d8e5a26" "-Wl,--gc-sections" "-static" "-no-pie" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "/usr/lib/libc.a" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtend.o" "/usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/self-contained/crtn.o"
  = note: /usr/lib/gcc/aarch64-alpine-linux-musl/10.3.1/../../../../aarch64-alpine-linux-musl/bin/ld: /usr/lib/libc.a(__init_tls.lo): in function `__init_tp':
          /home/buildozer/aports/main/musl/src/v1.2.2/src/env/__init_tls.c:15: multiple definition of `__init_tp'; /usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liblibc-72b57ab90668a509.rlib(__init_tls.lo):__init_tls.c:(.text.__init_tp+0x0): first defined here
          /usr/lib/gcc/aarch64-alpine-linux-musl/10.3.1/../../../../aarch64-alpine-linux-musl/bin/ld: /usr/lib/libc.a(__init_tls.lo): in function `__copy_tls':
          /home/buildozer/aports/main/musl/src/v1.2.2/src/env/__init_tls.c:40: multiple definition of `__copy_tls'; /usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liblibc-72b57ab90668a509.rlib(__init_tls.lo):__init_tls.c:(.text.__copy_tls+0x0): first defined here
          /usr/lib/gcc/aarch64-alpine-linux-musl/10.3.1/../../../../aarch64-alpine-linux-musl/bin/ld: /usr/lib/libc.a(libc.lo):/home/buildozer/aports/main/musl/src/v1.2.2/src/internal/libc.c:6: multiple definition of `__progname_full'; /usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liblibc-72b57ab90668a509.rlib(libc.lo):(.bss.__progname_full+0x0): first defined here
          /usr/lib/gcc/aarch64-alpine-linux-musl/10.3.1/../../../../aarch64-alpine-linux-musl/bin/ld: /usr/lib/libc.a(libc.lo):/home/buildozer/aports/main/musl/src/v1.2.2/src/internal/libc.c:6: multiple definition of `__progname'; /usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liblibc-72b57ab90668a509.rlib(libc.lo):(.bss.__progname+0x0): first defined here
          /usr/lib/gcc/aarch64-alpine-linux-musl/10.3.1/../../../../aarch64-alpine-linux-musl/bin/ld: warning: size of symbol `__libc' changed from 112 in /usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liblibc-72b57ab90668a509.rlib(libc.lo) to 104 in /usr/lib/libc.a(libc.lo)
          /usr/lib/gcc/aarch64-alpine-linux-musl/10.3.1/../../../../aarch64-alpine-linux-musl/bin/ld: /usr/lib/libc.a(sigaction.lo): in function `__get_handler_set':
          /home/buildozer/aports/main/musl/src/v1.2.2/src/signal/sigaction.c:15: multiple definition of `__get_handler_set'; /usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liblibc-72b57ab90668a509.rlib(sigaction.lo):sigaction.c:(.text.__get_handler_set+0x0): first defined here
          /usr/lib/gcc/aarch64-alpine-linux-musl/10.3.1/../../../../aarch64-alpine-linux-musl/bin/ld: /usr/lib/libc.a(sigaction.lo): in function `__libc_sigaction':
          /home/buildozer/aports/main/musl/src/v1.2.2/src/signal/sigaction.c:21: multiple definition of `__libc_sigaction'; /usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liblibc-72b57ab90668a509.rlib(sigaction.lo):sigaction.c:(.text.__libc_sigaction+0x0): first defined here
          /usr/lib/gcc/aarch64-alpine-linux-musl/10.3.1/../../../../aarch64-alpine-linux-musl/bin/ld: /usr/lib/libc.a(sigaction.lo): in function `__sigaction':
          /home/buildozer/aports/main/musl/src/v1.2.2/src/signal/sigaction.c:61: multiple definition of `__sigaction'; /usr/local/rustup/toolchains/nightly-aarch64-unknown-linux-musl/lib/rustlib/aarch64-unknown-linux-musl/lib/liblibc-72b57ab90668a509.rlib(sigaction.lo):sigaction.c:(.text.__sigaction+0x0): first defined here
          collect2: error: ld returned 1 exit status
          
error: could not compile `libc` due to previous error

/usr/lib/libc.a is from the musl-dev Alpine package. Since this is Alpine, musl is the system libc. I don't have any other libc.a and the only other libc.a in the Alpine repositories are for other targets. Note that I'm not cross-compiling.

[hkratz]

Probably related to #83655.

[hkratz]

Actually even helloworld fails to compile with --target aarch64-unknown-linux-musl right now due to this issue while it compiles and runs fine with stable.

[tonyg]

Probably related to #83655.

Great catch! The timing looks just about right - it must have been about 12-13 days ago I noticed the problem that I subsequently reported in https://github.com/rust-embedded/cross/issues/598, and it looks like #83655 was merged into master about 14 days ago.

even helloworld fails to compile with --target aarch64-unknown-linux-musl

This matches what I saw with https://github.com/rust-embedded/cross/issues/598, where a cargo init foo fails to compile with nightly.

[jbg]

Yes, very likely. I've hit similar issues with outline atomics with C/C++ code on aarch64-unknown-linux-musl, which are fixed with CFLAGS=-mno-outline-atomics. See for example rust-lang/git2-rs#706

[jbg]

I tried to use this workaround: RUSTFLAGS="-C target-feature=-outline-atomics"

But it doesn't change anything. Is it because +outline-atomics is added after -C target-feature is processed here?

[hkratz]

Hmm. getauxval is aliased to __getauxval in liblibc-[...].rlib. The problem likely manifests because of #76992 (see above).

What works for me as a workaround with helloworld is using lld for linking like this:

RUSTFLAGS="-Zgcc-ld=lld" cargo +nightly build --target aarch64-unknown-linux-musl

cc @joshtriplett

[hkratz]

@rustbot label -E-needs-bisection

[apiraino]

Assigning priority as discussed in the Zulip thread of the Prioritization Working Group.

@rustbot label -I-prioritize +P-high

[BlackDex]

Rust stable is also having issues now.

cargo 1.56.0 (4ed5d137b 2021-10-04)
release: 1.56.0
commit-hash: 4ed5d137baff5eccf1bae5a7b2ae4b57efad4a7d
commit-date: 2021-10-04

rustc 1.56.1 (59eed8a2a 2021-11-01)
binary: rustc
commit-hash: 59eed8a2aac0230a8b53e89d4e99d55912ba6b35
commit-date: 2021-11-01
host: x86_64-unknown-linux-gnu
release: 1.56.1
LLVM version: 13.0.0

I was able to fix this on stable by adding this to the RUSTFLAGS -Clinker=rust-lld

This works for both stable and nightly btw.

[hkratz]

Hmm, can't reproduce the issue with current stable (1.56.1). However the problem is present in the current beta (1.57) which will become stable soon.

[BlackDex]

If i use blackdex/rust-musl:aarch64-musl-stable i get the same errors. Though for some reason if i use messense/rust-musl-cross:aarch64-musl it doesn't, even if i force use 1.56.1 because that image is still on 1.56.0. But if i use nightly on the messense image it does produce that same error.

The other way around is also a bit strange, if i force 1.56.0 (or even 1.55.0) on the blackdex image it still generates the error for some reason. Only difference i see that quickly is that messense is compiled with GCC 9.2.0, and the blackdex image with GCC 10.3.0.

[pnkfelix]

@XAMPPRocky can you confirm if this is fixed in 1.57 (and also current nightly)?

[BlackDex]

@pnkfelix for me it does not seem to work on both new stable and latest nightly. I still need the -Clinker=rust-lld RUSTFLAG.

[hkratz]

@BlackDex But you also had a problem with 1.56.1, so it does not seem like that is the same issue.

[BlackDex]

I get exactly the same error message and the exact same solution solves it, so not sure why it is that different.

[XAMPPRocky]

I can confirm that this is now fixed for me, https://github.com/XAMPPRocky/rasn/runs/4397275912?check_suite_focus=true

[hkratz]

@BlackDex The outline atomics feature was not enabled by default in Rust 1.55 and 1.56. So the root cause of the problem you are seeing must be different. I tried building a simple hello world project with your docker images and that worked fine for me. Can you file a new bug report with steps on how to reproduce the issue?

[BlackDex]

@hkratz i'm currently testing some different options on building the musl toolchain. I am using gcc 10.3.0 and both rust-embedded and messense are using 9.2.0. ill check if that is causing the issue. If that is the case, would that still be valid for a bug-report, if it is the gcc 10.3.0 which is causing this?

Update: Yes, that looks like what is the issue. outline-atomics was only added since GCC 9.4, and enabled by default in GCC 10.1 it looks like. I can probably disable this on the toolchain build. But then still, would this be worth an issue in combination with GCC 10.3 and the default of -moutline-atomics?

Update2: I just looked at https://github.com/rust-lang/rust/pull/91220 and i see that it is disabled there for aarch64, if i do the same when building the C/C++ libraries my self it looks like that solves the issue, i just need to adjust my build scripts for this.

[pnkfelix]

I can confirm that this is now fixed for me, https://github.com/XAMPPRocky/rasn/runs/4397275912?check_suite_focus=true

closing as fixed.

From my reading of the comment thread, it is not clear to me if the issues encountered by @BlackDex are outright bugs or not. but even if they are bugs, they need to be filed as fresh issues.

[Techcable]

I still have this exact same problem on stable, 1.55 and latest nightly 😦

I am building on alpine in a VM.