rust [WIP] Linker flavors: next steps

TODO: detailed description.

Features that may affect the naming scheme for linker flavors (at least those I'm aware of):

1. Clang linker flavors - allow supporting cross-linking automatically by passing --target, something that is not supported by gcc-style compilers.
1. Generic linker flavors like in https://github.com/rust-lang/rust/commit/364f36d9450ab3a1c16883c0df34f25f4cfd6a78, portable between targets.
1. A flavor for linking LLVM bitcode through rustc. It can be supported for all targets, I think - https://github.com/rust-lang/rust/pull/117458#issuecomment-1889150208.
1. Linker flavors that deviate from the default target's flavor in the main component, like LinkerFlavor::Gnu for a LinkerFlavor::Msvc-based target - https://github.com/rust-lang/rust/pull/110869. Even if it requires some special logic, that would still be less heavy weight than providing whole separate targets with their own CI and distributed binaries.

r? @lqd cc @jschwe

Jan 12 '24 22:01 petrochenkov

These commits modify compiler targets. (See the Target Tier Policy.)

Jan 12 '24 22:01 rustbot

The job x86_64-gnu-llvm-16 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

Built container sha256:9c3c93a371e5aed5c18185b24f130d95d5140dbd72a9b325e7b6b49e521a4faa
Looks like docker image is the same as before, not uploading
https://ci-caches.rust-lang.org/docker/7ebc15c01a233894034d277c8cce4e949f4e7791f66b4727c8fb6e058a0b8171d6152e1441d677cef0653843ceeee469c097b8699b2bb74249e674f6aa1a8813
sha256:9c3c93a371e5aed5c18185b24f130d95d5140dbd72a9b325e7b6b49e521a4faa
Setting extra environment values for docker:  --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-16]
##[group]Clock drift check
  local time: Fri Jan 12 22:13:33 UTC 2024
  network time: Fri, 12 Jan 2024 22:13:33 GMT
  network time: Fri, 12 Jan 2024 22:13:33 GMT
##[endgroup]
sccache: Starting the server...
##[group]Configure the build
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-16', '--enable-llvm-link-shared', '--set', 'rust.thin-lto-import-instr-limit=10', '--set', 'change-id=99999999', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'build.optimized-compiler-builtins', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-16/bin/llvm-config
configure: llvm.link-shared     := True
configure: rust.thin-lto-import-instr-limit := 10
configure: change-id            := 99999999
---
##[endgroup]
Testing GCC stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
   Compiling y v0.1.0 (/checkout/compiler/rustc_codegen_gcc/build_system)
    Finished release [optimized] target(s) in 1.37s
     Running `/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-codegen/x86_64-unknown-linux-gnu/release/y test --use-system-gcc --use-backend gcc --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc --release --no-default-features --mini-tests --std-tests`
Using system GCC
Using system GCC
[BUILD] example
[AOT] mini_core_hello_world
/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc/mini_core_hello_world
abc
---
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 12.03µs


running 239 tests
......FFFFF....F...F..FFF...F........F.F..FF................FFF..............F..FF.FF..F  88/239
F....FFFFFFF.FFF.F....F.....F.F..................FFFFFFFFFFFFF.F...F.................... 176/239
.FF.FF.............F.FFFFFFFF.F.FFFF.FFFF.F..FFFF.FFF.F.F.F.F..
failures:

---- spec::tests::aarch64_apple_ios_sim stdout ----
---- spec::tests::aarch64_apple_ios_sim stdout ----
thread 'spec::tests::aarch64_apple_ios_sim' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::aarch64_apple_darwin stdout ----
thread 'spec::tests::aarch64_apple_darwin' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes

---- spec::tests::aarch64_apple_ios_macabi stdout ----
---- spec::tests::aarch64_apple_ios_macabi stdout ----
thread 'spec::tests::aarch64_apple_ios_macabi' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::aarch64_apple_tvos stdout ----
thread 'spec::tests::aarch64_apple_tvos' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::aarch64_apple_ios stdout ----
thread 'spec::tests::aarch64_apple_ios' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::aarch64_apple_watchos_sim stdout ----
---- spec::tests::aarch64_apple_watchos_sim stdout ----
thread 'spec::tests::aarch64_apple_watchos_sim' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::aarch64_unknown_illumos stdout ----
thread 'spec::tests::aarch64_unknown_illumos' panicked at compiler/rustc_target/src/spec/mod.rs:2245:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::aarch64_apple_watchos stdout ----
thread 'spec::tests::aarch64_apple_watchos' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::aarch64_apple_tvos_sim stdout ----
---- spec::tests::aarch64_apple_tvos_sim stdout ----
thread 'spec::tests::aarch64_apple_tvos_sim' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::aarch64_pc_windows_gnullvm stdout ----
thread 'spec::tests::aarch64_pc_windows_gnullvm' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes

---- spec::tests::aarch64_unknown_nto_qnx_710 stdout ----
thread 'spec::tests::aarch64_unknown_nto_qnx_710' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::aarch64_unknown_teeos stdout ----
thread 'spec::tests::aarch64_unknown_teeos' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::arm64e_apple_ios stdout ----
thread 'spec::tests::arm64e_apple_ios' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::arm64e_apple_darwin stdout ----
thread 'spec::tests::arm64e_apple_darwin' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::arm64_32_apple_watchos stdout ----
thread 'spec::tests::arm64_32_apple_watchos' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::armv7_sony_vita_newlibeabihf stdout ----
thread 'spec::tests::armv7_sony_vita_newlibeabihf' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::armv7_linux_androideabi stdout ----
thread 'spec::tests::armv7_linux_androideabi' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::armv6k_nintendo_3ds stdout ----
thread 'spec::tests::armv6k_nintendo_3ds' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::armv7k_apple_watchos stdout ----
thread 'spec::tests::armv7k_apple_watchos' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::avr_unknown_gnu_atmega328 stdout ----
---- spec::tests::avr_unknown_gnu_atmega328 stdout ----
thread 'spec::tests::avr_unknown_gnu_atmega328' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::csky_unknown_linux_gnuabiv2 stdout ----
---- spec::tests::csky_unknown_linux_gnuabiv2 stdout ----
thread 'spec::tests::csky_unknown_linux_gnuabiv2' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::armv7s_apple_ios stdout ----
thread 'spec::tests::armv7s_apple_ios' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::csky_unknown_linux_gnuabiv2hf stdout ----
---- spec::tests::csky_unknown_linux_gnuabiv2hf stdout ----
thread 'spec::tests::csky_unknown_linux_gnuabiv2hf' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i586_pc_nto_qnx700 stdout ----
thread 'spec::tests::i586_pc_nto_qnx700' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i386_apple_ios stdout ----
thread 'spec::tests::i386_apple_ios' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i586_unknown_linux_musl stdout ----
thread 'spec::tests::i586_unknown_linux_musl' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
error: test failed, to rerun pass `-p rustc_target --lib`
Build completed unsuccessfully in 0:26:52
 right: Cc::No | Cc::Yes
---- spec::tests::i686_unknown_freebsd stdout ----
thread 'spec::tests::i686_unknown_freebsd' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i686_pc_windows_gnullvm stdout ----
thread 'spec::tests::i686_pc_windows_gnullvm' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i686_unknown_haiku stdout ----
---- spec::tests::i686_unknown_haiku stdout ----
thread 'spec::tests::i686_unknown_haiku' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i586_unknown_linux_gnu stdout ----
---- spec::tests::i586_unknown_linux_gnu stdout ----
thread 'spec::tests::i586_unknown_linux_gnu' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i686_unknown_hurd_gnu stdout ----
---- spec::tests::i686_unknown_hurd_gnu stdout ----
thread 'spec::tests::i686_unknown_hurd_gnu' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i686_apple_darwin stdout ----
thread 'spec::tests::i686_apple_darwin' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i686_unknown_linux_gnu stdout ----
---- spec::tests::i686_unknown_linux_gnu stdout ----
thread 'spec::tests::i686_unknown_linux_gnu' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i686_unknown_netbsd stdout ----
thread 'spec::tests::i686_unknown_netbsd' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i686_unknown_openbsd stdout ----
thread 'spec::tests::i686_unknown_openbsd' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i686_unknown_linux_musl stdout ----
thread 'spec::tests::i686_unknown_linux_musl' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i686_wrs_vxworks stdout ----
thread 'spec::tests::i686_wrs_vxworks' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i686_pc_windows_gnu stdout ----
---- spec::tests::i686_pc_windows_gnu stdout ----
thread 'spec::tests::i686_pc_windows_gnu' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::i686_uwp_windows_gnu stdout ----
---- spec::tests::i686_uwp_windows_gnu stdout ----
thread 'spec::tests::i686_uwp_windows_gnu' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc64_unknown_linux_gnu stdout ----
---- spec::tests::powerpc64_unknown_linux_gnu stdout ----
thread 'spec::tests::powerpc64_unknown_linux_gnu' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc64_unknown_freebsd stdout ----
thread 'spec::tests::powerpc64_unknown_freebsd' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc64_unknown_linux_musl stdout ----
thread 'spec::tests::powerpc64_unknown_linux_musl' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc64_unknown_openbsd stdout ----
thread 'spec::tests::powerpc64_unknown_openbsd' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc64_wrs_vxworks stdout ----
thread 'spec::tests::powerpc64_wrs_vxworks' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc64le_unknown_freebsd stdout ----
thread 'spec::tests::powerpc64le_unknown_freebsd' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc64le_unknown_linux_gnu stdout ----
---- spec::tests::powerpc64le_unknown_linux_gnu stdout ----
thread 'spec::tests::powerpc64le_unknown_linux_gnu' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc_unknown_freebsd stdout ----
thread 'spec::tests::powerpc_unknown_freebsd' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc64le_unknown_linux_musl stdout ----
thread 'spec::tests::powerpc64le_unknown_linux_musl' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc_unknown_linux_gnu stdout ----
---- spec::tests::powerpc_unknown_linux_gnu stdout ----
thread 'spec::tests::powerpc_unknown_linux_gnu' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc_unknown_linux_gnuspe stdout ----
thread 'spec::tests::powerpc_unknown_linux_gnuspe' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc_unknown_linux_musl stdout ----
thread 'spec::tests::powerpc_unknown_linux_musl' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc_wrs_vxworks stdout ----
thread 'spec::tests::powerpc_wrs_vxworks' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc_wrs_vxworks_spe stdout ----
thread 'spec::tests::powerpc_wrs_vxworks_spe' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::powerpc_unknown_netbsd stdout ----
thread 'spec::tests::powerpc_unknown_netbsd' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::sparc64_unknown_netbsd stdout ----
thread 'spec::tests::sparc64_unknown_netbsd' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::sparc64_unknown_openbsd stdout ----
thread 'spec::tests::sparc64_unknown_openbsd' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::sparc_unknown_linux_gnu stdout ----
---- spec::tests::sparc_unknown_linux_gnu stdout ----
thread 'spec::tests::sparc_unknown_linux_gnu' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::sparcv9_sun_solaris stdout ----
thread 'spec::tests::sparcv9_sun_solaris' panicked at compiler/rustc_target/src/spec/mod.rs:2245:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::thumbv7neon_linux_androideabi stdout ----
thread 'spec::tests::thumbv7neon_linux_androideabi' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::wasm32_unknown_unknown stdout ----
thread 'spec::tests::wasm32_unknown_unknown' panicked at compiler/rustc_target/src/spec/mod.rs:2245:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::wasm32_wasi stdout ----
thread 'spec::tests::wasm32_wasi' panicked at compiler/rustc_target/src/spec/mod.rs:2245:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::wasm32_wasi_preview1_threads stdout ----
thread 'spec::tests::wasm32_wasi_preview1_threads' panicked at compiler/rustc_target/src/spec/mod.rs:2245:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::wasm64_unknown_unknown stdout ----
thread 'spec::tests::wasm64_unknown_unknown' panicked at compiler/rustc_target/src/spec/mod.rs:2245:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_apple_darwin stdout ----
thread 'spec::tests::x86_64_apple_darwin' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_apple_ios stdout ----
thread 'spec::tests::x86_64_apple_ios' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_apple_ios_macabi stdout ----
---- spec::tests::x86_64_apple_ios_macabi stdout ----
thread 'spec::tests::x86_64_apple_ios_macabi' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_apple_tvos stdout ----
thread 'spec::tests::x86_64_apple_tvos' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_apple_watchos_sim stdout ----
---- spec::tests::x86_64_apple_watchos_sim stdout ----
thread 'spec::tests::x86_64_apple_watchos_sim' panicked at compiler/rustc_target/src/spec/mod.rs:2233:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_linux_android stdout ----
thread 'spec::tests::x86_64_linux_android' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_pc_nto_qnx710 stdout ----
thread 'spec::tests::x86_64_pc_nto_qnx710' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_pc_solaris stdout ----
thread 'spec::tests::x86_64_pc_solaris' panicked at compiler/rustc_target/src/spec/mod.rs:2245:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_pc_windows_gnullvm stdout ----
thread 'spec::tests::x86_64_pc_windows_gnullvm' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_unknown_dragonfly stdout ----
---- spec::tests::x86_64_unknown_dragonfly stdout ----
thread 'spec::tests::x86_64_unknown_dragonfly' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_unikraft_linux_musl stdout ----
thread 'spec::tests::x86_64_unikraft_linux_musl' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_pc_windows_gnu stdout ----
---- spec::tests::x86_64_pc_windows_gnu stdout ----
thread 'spec::tests::x86_64_pc_windows_gnu' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_unknown_freebsd stdout ----
thread 'spec::tests::x86_64_unknown_freebsd' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_unknown_haiku stdout ----
---- spec::tests::x86_64_unknown_haiku stdout ----
thread 'spec::tests::x86_64_unknown_haiku' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_unknown_illumos stdout ----
thread 'spec::tests::x86_64_unknown_illumos' panicked at compiler/rustc_target/src/spec/mod.rs:2245:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_unknown_linux_gnu stdout ----
---- spec::tests::x86_64_unknown_linux_gnu stdout ----
thread 'spec::tests::x86_64_unknown_linux_gnu' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_unknown_linux_gnux32 stdout ----
thread 'spec::tests::x86_64_unknown_linux_gnux32' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_unknown_linux_musl stdout ----
thread 'spec::tests::x86_64_unknown_linux_musl' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:
assertion `left matches right` failed
  left: Clang
  left: Clang
 right: Cc::No | Cc::Yes
---- spec::tests::x86_64_unknown_netbsd stdout ----
thread 'spec::tests::x86_64_unknown_netbsd' panicked at compiler/rustc_target/src/spec/mod.rs:2224:13:

Jan 12 '24 22:01 rust-log-analyzer

Let's talk about one of the hard problems in CS.

Linker flavor naming

The base component is unfortunately needed

It would be nice to just drop the base flavor components (like gnu/darwin/wasm/etc) from user-facing flavors and leave only generic ld, lld, (ld,lld)-(cc,clang), because the base flavor it tied to the target. This is not compatible with items 3 and 4 though - some targets like UEFI need linkers outside of their base flavor, and the hypothetical llvm-bc flavor should also be compatible with a wide range of targets. So we should have the full flavors in some form, but typically using the generic flavors seem preferable.

Generic flavors

I think usually the user intent behind generic flavors can be worded as "enable lld", or "enable clang", or "disable C compiler and use bare linker". So I'm tempted to use the +/-flag syntax for it (again), similarly to target features, native linking modifiers, link-self-contained components, etc. I wonder whether I'm over-using it for everything, and how much people hate it?

So, with this +/- generic flavors would look like this.

+/-lld - take the default flavor and add/remove lld
+/-cc - take the default flavor and add/remove cc
+/-clang - take the default flavor and add clang / remove clang (but add cc instead if the default is bare linker? probably not)
combinations of the above +/-lld,+/-cc, +/-lld,+/-clang with the obvious meaning, we don't even need to parse arbitrary comma-separated lists, just these specific strings.

The alternatives to that are names like ld or lld-cc (the base component is just skipped), but the "negative" scenarios (not lld, not cc) look worse in this case. The ld naming (opposite of lld) is not very appropriate for targets like msvc, and for lld flavor it's not immediately obvious that it disables cc.

Naming full flavors after generic flavors

What if we apply the +/- idea to full flavors?

Existing flavors

// Are `-`s dashes or minuses like in generic flavors?
"gnu"
"gnu-lld"
"gnu-cc"
"gnu-lld-cc"
"gnu-clang"
"gnu-lld-clang"

will turn into

// I dont' like this, TBH
"gnu-lld,-cc"
"gnu+lld,-cc"
"gnu-lld,+cc"
"gnu+lld,+cc"
"gnu-lld,+clang"
"gnu+lld,+clang"

or into

// More or less fine
"gnu"
"gnu+lld"
"gnu+cc"
"gnu+lld,+cc"
"gnu+clang"
"gnu+lld,+clang"

if we drop negatives, or into

"gnu"
"gnu+lld"
"gnu+cc"
"gnu+lld+cc"
"gnu+clang"
"gnu+lld+clang"

if we drop commas as well (looks farther from generic flavors).

llvm-bc flavor would bring some confusion though, what does it mean - "llvm minus bc"? But that's probably fine.

Conclusions

I'm not sure, basically. Let's maybe keep the existing names, add the generic flavors with +/-, and tolerate the confusion between dashes in the former and minuses in the latter.

Or change full flavors to the gnu+lld,+cc scheme without negatives, but with commas, as a second alternative.

Jan 16 '24 16:01 petrochenkov

@rustbot ready (on the design, not implementation)

Jan 16 '24 16:01 petrochenkov

a possibly nice thing I wasn't anticipating with the clang flavor is they'd simplify our wasm targets specs (it's maybe one of the WIP bits of the PR, as I'm not yet sure it's fully implemented just by reading the code, since these targets -- or the apple ones -- shouldn't have a flavor where uses_clang returns true in add_lld_args?)

Yes, the current state of the PR is a sketch, apple targets should switch to clang as the default flavor, and add_lld_args is buggy.

since the clang-specific features won't be used or referenced very often in the compiler or specs, do you think Cc as today's Yes/No with the Yes variant itself gaining the two cc kinds would be better or worse than the 3-variant Cc as shown here? My thoughts are we should be using/checking for Cc::Yes basically everywhere and otherwise will now have to check if it's yes or clang. Reading these made me wonder whether Cc::Yes(_) would nicely encode this property of "de-emphasizing" the clang variant in general, and look for it in Cc::Yes(compiler_flavor) only when needed.

Maybe, but it's a big churn, probably not worth doing until everything stabilizes for sure. The effect can be observed in the @jschwe's attempt at the same change - https://github.com/rust-lang/rust/commit/f4a39514a9d4eb6a6aad0fb85fac04afbf3fdf4e

Jan 16 '24 22:01 petrochenkov

Maybe, but it's a big churn, probably not worth doing until everything stabilizes for sure.

Looking at the diff, I agree. Let's keep the 3-variant enum for now and once we're a bit surer about the overall structure (components, order, naming, etc) we'll clean these up easily.

Jan 17 '24 08:01 lqd

I find the idea of generic flavors ...

So, I thought maybe we should do a 180 degree turn and shift all the logic except principal flavors to -Clinker-features? Then, for example, instead of the clang flavors we'd basically have -Clinker-features=+target.

Feature flags are a more flexible mechanism than current linker flavors, in general.

You can have default feature sets attached to principal flavors, or to specific targets.
You can have umbrella features like e.g. +clang if clang accumulates more features than just +target in the future.
You can have niche features for resolving specific issues, e.g. on Apple targets the linker flag implementing https://github.com/rust-lang/rust/issues/99424 is only implementable on sufficiently new linker versions, IIRC. rustc could allow enabling/disabling passing it with a feature.
Build systems that are more similar to CMake or automake, than to cargo, could run feature detection at configure step, put the set of supported features into a cache, and then use them during builds.

Jan 17 '24 12:01 petrochenkov

So, I thought maybe we should do a 180 degree turn and shift all the logic except principal flavors to -Clinker-features? Then, for example, instead of the clang flavors we'd basically have -Clinker-features=+target.

That's a very interesting twist! That could unify a bunch of disparate code-paths and flags/options (and should also work for self-contained linking features), as well as simplify use with a single entry-point, --print support and documentation, and keep flexibility/evolvability on rustc's side.

It feels like it should simplify our current setup, and this PR's flavors of course, but were you also imagining using this for the generic flavors? (I don't think so?)

Jan 23 '24 15:01 lqd

It feels like it should simplify our current setup, and this PR's flavors of course, but were you also imagining using this for the generic flavors? (I don't think so?)

Well, -Clinker-features=+lld or -Clinker-features=+cc could work as generic flavors.

Jan 23 '24 15:01 petrochenkov

Sure, like I described in my earlier comment.

Overall, I personally find this direction promising. A bunch of the support code could also be based on renaming the self-contained linking components to "linker features" (w/ a prefix), adding cc and lld, and simplifying the existing flavors ^^

Jan 23 '24 16:01 lqd

renaming the self-contained linking components to "linker features" (w/ a prefix)

Could you elaborate, I don't understand this part.

Jan 23 '24 16:01 petrochenkov

I meant you could imagine the possible linker features as a generalization of self-contained linking components. And get an approximation of how that'd look like by renaming LinkSelfContainedComponents to LinkerFeatures, and adding a prefix to the existing variants there to mark their relationship to self-contained linking: à la -Clinker-features=+self-contained-crto and LinkerFeatures::SELF_CONTAINED_CRT_OBJECTS, as a generalization of the current LinkSelfContainedComponents::CRT_OBJECTS for -Clink-self-contained=+crto. Then add +cc, +lld, +target, to simplify the modern unstable linker flavors.

Jan 23 '24 17:01 lqd

I see, I suspected that, but it seems far enough from linker features, if they are understood as something representing linker interface.

Jan 23 '24 20:01 petrochenkov

Ok, then let’s not merge linker-features and self-contained linking components.

Jan 23 '24 21:01 lqd

:umbrella: The latest upstream changes (presumably #120468) made this pull request unmergeable. Please resolve the merge conflicts.

Mar 04 '24 21:03 bors

To summarize, the use-cases were:

automatic cross-linking with clang-specific functionality: can be done with e.g. -Clinker-features=+target (or +clang if it grows more similar features)
turning parts of the flavor on or off, AKA "generic linker flavors": e.g. -Clinker-features=+/-lld and -Clinker-features=+/-cc
linking LLVM bitcode: this looks like a new self-contained linker so that's likely to involve at the very least its own dedicated flavor subpart, à la -Clink-self-contained=+linker -Clinker-flavor=gnu-llbc-cc, or full unique flavor -Clinker-flavor=llbc. Since this can most likely work everywhere, it could also be toggled via the generic linker flavor mechanism, -Clinker-features=+llbc
targets with "interesting" linking requirements: I don't know the MSVC UEFI use-case but it looks like they can use either MSVC+lld or another unix. I was expecting that overriding the default with -Clinker-flavor=unix-cc would match that use-case, but I guess not. Maybe the idea is that this would be a rustc default rather than users doing it? I'm not sure what you had in mind, but that seems frameable as linker features as well: either via toggling the principal flavor (e.g. -Clinker-features=-msvc,+unix,+cc, or -Clinker-features=+unix,+cc ), or having an "override" linker feature (e.g. -Clinker-features=+override:unix-cc). The former could open the erroneous case of turning off the principal flavor without picking another. However, if a CLI argument is acceptable for this use-case, I must be missing the reason why -Clinker-flavor is not. (It could be that it's more cumbersome for us to mark msvc-lld and unix-cc acceptable for this target only and do CLI validation, but if it's that, that sounds fixable)

The other arguments for linker features:

defaults per principal flavor, or target
flexibility and evolvability: if needed, features can encompass others. New features can be added over time without impacting existing ones, or require name changes and so on.
precision: in theory even niche issues can use this mechanism.
integration in exotic environments: it's not unique to this feature, but it allows discoverability and automation, and can fit well with feature autodetection

In a future where x86_64-unknown-linux-gnu uses lld by default, we could imagine the default features -Clinker-features=+cc,+lld.

So now we can also wonder if -Clink-self-contained=+linker is worthwhile on its own? Versus e.g. -Clinker-features=+self-contained and keeping -Clink-self-contained as mostly related to libraries and native objects. Here, in a future where x86_64-unknown-linux-gnu uses rust-lld by default, we could imagine the default features -Clinker-features=+self-contained,+cc,+lld. And similarly, nvptx64-nvidia-cuda could be -Clinker-features=+self-contained,+llbc.

What do you think @petrochenkov?

(It seems we already had shared ideas about the design, so I'm not sure if this was really waiting-on-review, @rustbot author)

Mar 06 '24 12:03 lqd

linking LLVM bitcode ... -Clinker-features=+llbc

I just r+'d the LLVM bitcode linker today - https://github.com/rust-lang/rust/pull/117458, it should land soon. It has its own interface and no cc wrapper, so it has its own principal flavor. So it's just going to be -Clinker-flavor=llbc, not a feature.

Mar 07 '24 16:03 petrochenkov

automatic cross-linking with clang-specific ...

turning parts of the flavor on or off ...

+

Mar 07 '24 16:03 petrochenkov

Sorry for the delay.

targets with "interesting" linking requirements: I don't know the MSVC UEFI use-case but it looks like they can use either MSVC+lld or another unix.

My point in general was that we should not make the "single principal flavor for a target" an axiom. This likely affects only places like LinkerFlavor::infer_cli_hints and LinkerFlavor::check_compatibility.

"Windows GNU LLD" is a thin wrapper over "Windows MSVC LLD" that only translates command line options from one format to another. In the UEFI case they wanted to use the Unix-style command line interface specifically for some build system reason. I think specifying the full flavor would be fine in that case, -Clinker-flavor=gnu-lld or just -Clinker-flavor=gnu depending on our choice regarding lld and cc modifiers.

Mar 11 '24 15:03 petrochenkov

All the remaining part makes sense to me, except I'm still skeptical about moving the "self-contained" bit to the linker flavor/features.

("Self-contained" and other features will even be processed in different parts of the compiler because the latter is about linker command line interface, and the former is about directories included into PATH and similar things.)

Mar 11 '24 15:03 petrochenkov

I'll close this PR because it's only used for discussion, and the changes themselves are going to be implemented in a different way.

Mar 13 '24 15:03 petrochenkov

rust rust copied to clipboard

[WIP] Linker flavors: next steps

Linker flavor naming

The base component is unfortunately needed

Generic flavors

Naming full flavors after generic flavors

Conclusions

rust
rust copied to clipboard