rust Introduce -Zsplit-metadata option

This will split the crate metadata out of library files. Instead only the svh and a bit of extra metadata is preserved to allow for loading the right rmeta file. This significantly reduces library size. In addition it allows for cheaper checks if different library files are the same crate.

A fair amount of the complexity in this PR is to work around the fact that cargo doesn't directly support this option yet.

Fixes https://github.com/rust-lang/rust/issues/23366 Closes https://github.com/rust-lang/rust/issues/29511 Fixes https://github.com/rust-lang/rust/issues/57076

Revives https://github.com/rust-lang/rust/pull/93945

Feb 09 '24 18:02 bjorn3

r? @cjgillot

rustbot has assigned @cjgillot. They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Feb 09 '24 18:02 rustbot

@bors try @rust-timer queue

Feb 09 '24 18:02 bjorn3

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

Feb 09 '24 18:02 rust-timer

:hourglass: Trying commit 77700020d3bb323331cc669cdb22f921b5423b9c with merge 8fb65acf1a11f547302c62c93bf83cca3ff4cd9d...

Feb 09 '24 18:02 bors

What would be the best way to handle the necessary cargo changes? We can't use them until the bootstrap compiler includes the cargo changes, but cargo would depend on -Zsplit-metadata which doesn't exist yet. Would unconditionally creating a .rmeta file for dylibs like we do for rlibs be fine? Or should I add a -Zsplit-metadata option to cargo without it getting tested and then once the bootstrap bump happens, change this PR to use this option? Or land -Zsplit-metadata first without actually making any use of it, then update cargo and finally make use of it in the standard library?

Feb 09 '24 18:02 bjorn3

The job x86_64-gnu-llvm-16 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

#12 writing image sha256:b3d79a1e42161fa05d38c810ae59bf072eb63e27d678d56e6ed282304ec40428 done
#12 naming to docker.io/library/rust-ci done
#12 DONE 10.0s
##[endgroup]
Setting extra environment values for docker:  --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-16]
##[group]Clock drift check
  local time: Fri Feb  9 18:50:07 UTC 2024
  network time: Fri, 09 Feb 2024 18:50:07 GMT
  network time: Fri, 09 Feb 2024 18:50:07 GMT
##[endgroup]
sccache: Starting the server...
##[group]Configure the build
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-16', '--enable-llvm-link-shared', '--set', 'rust.thin-lto-import-instr-limit=10', '--set', 'change-id=99999999', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'build.optimized-compiler-builtins', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-16/bin/llvm-config
configure: llvm.link-shared     := True
configure: rust.thin-lto-import-instr-limit := 10
configure: change-id            := 99999999
---
##[endgroup]
Testing GCC stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
   Compiling y v0.1.0 (/checkout/compiler/rustc_codegen_gcc/build_system)
    Finished release [optimized] target(s) in 1.44s
     Running `/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-codegen/x86_64-unknown-linux-gnu/release/y test --use-system-gcc --use-backend gcc --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc --release --no-default-features --mini-tests --std-tests`
Using system GCC
Using system GCC
[BUILD] example
[AOT] mini_core_hello_world
/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc/mini_core_hello_world
abc

Feb 09 '24 19:02 rust-log-analyzer

:sunny: Try build successful - checks-actions Build commit: 8fb65acf1a11f547302c62c93bf83cca3ff4cd9d (8fb65acf1a11f547302c62c93bf83cca3ff4cd9d)

Feb 09 '24 20:02 bors

Queued 8fb65acf1a11f547302c62c93bf83cca3ff4cd9d with parent e28fae52d99d7c14bf0890d1f2b13c2c34fa8932, future comparison URL. There is currently 1 preceding artifact in the queue. It will probably take at least ~2.2 hours until the benchmark run finishes.

Feb 09 '24 20:02 rust-timer

The job x86_64-gnu-llvm-16 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

#12 writing image sha256:e4eee1e863d785eca5dc51b1983ba4234b8f3899fae31fb52d1799952d9d218b done
#12 naming to docker.io/library/rust-ci done
#12 DONE 10.1s
##[endgroup]
Setting extra environment values for docker:  --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-16]
##[group]Clock drift check
  local time: Fri Feb  9 20:24:52 UTC 2024
  network time: Fri, 09 Feb 2024 20:24:52 GMT
  network time: Fri, 09 Feb 2024 20:24:52 GMT
##[endgroup]
sccache: Starting the server...
##[group]Configure the build
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-16', '--enable-llvm-link-shared', '--set', 'rust.thin-lto-import-instr-limit=10', '--set', 'change-id=99999999', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'build.optimized-compiler-builtins', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-16/bin/llvm-config
configure: llvm.link-shared     := True
configure: rust.thin-lto-import-instr-limit := 10
configure: change-id            := 99999999
---
##[endgroup]
Testing GCC stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
   Compiling y v0.1.0 (/checkout/compiler/rustc_codegen_gcc/build_system)
    Finished release [optimized] target(s) in 1.45s
     Running `/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-codegen/x86_64-unknown-linux-gnu/release/y test --use-system-gcc --use-backend gcc --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc --release --no-default-features --mini-tests --std-tests`
Using system GCC
Using system GCC
[BUILD] example
[AOT] mini_core_hello_world
/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc/mini_core_hello_world
abc
---

error: make failed
status: exit status: 2
Build completed unsuccessfully in 0:39:34
command: cd "/checkout/tests/run-make/invalid-library" && env -u CARGO_MAKEFLAGS -u MAKEFLAGS -u MFLAGS -u RUSTFLAGS AR="ar" CC="cc -ffunction-sections -fdata-sections -fPIC -m64" CXX="c++ -ffunction-sections -fdata-sections -fPIC -m64" HOST_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" LD_LIB_PATH_ENVVAR="LD_LIBRARY_PATH" LLVM_BIN_DIR="/usr/lib/llvm-16/bin" LLVM_COMPONENTS="aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets amdgpu amdgpuasmparser amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgputargetmca amdgpuutils analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen core coroutines coverage debuginfocodeview debuginfodwarf debuginfogsym debuginfologicalview debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker dwarflinkerparallel dwp engine executionengine extensions filecheck frontendhlsl frontendopenacc frontendopenmp fuzzercli fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interfacestub interpreter ipo irprinter irreader jitlink lanai lanaiasmparser lanaicodegen lanaidesc lanaidisassembler lanaiinfo libdriver lineeditor linker loongarch loongarchasmparser loongarchcodegen loongarchdesc loongarchdisassembler loongarchinfo lto m68k m68kasmparser m68kcodegen m68kdesc m68kdisassembler m68kinfo mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts objcopy object objectyaml option orcjit orcshared orctargetprocess passes perfjitevents powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvtargetmca runtimedyld scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target targetparser textapi transformutils ve veasmparser vecodegen vectorize vedesc vedisassembler veinfo webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo webassemblyutils windowsdriver windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86targetmca xcore xcorecodegen xcoredesc xcoredisassembler xcoreinfo xray" LLVM_FILECHECK="/usr/lib/llvm-16/bin/FileCheck" NODE="/usr/bin/node" PYTHON="/usr/bin/python3" RUSTC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" RUSTDOC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" RUST_BUILD_STAGE="stage2-x86_64-unknown-linux-gnu" S="/checkout" TARGET="x86_64-unknown-linux-gnu" TARGET_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" TMPDIR="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library" "make"
touch /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library/lib.rmeta
touch /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library/lib.rmeta
ar crus /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library/libfoo-ffffffff-1.0.rlib /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library/lib.rmeta
LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library  -Ainternal_features foo.rs 2>&1 | "/checkout/src/etc/cat-and-grep.sh" "found invalid metadata"
[[[ begin stdout ]]]
error[E0514]: found crate `foo` compiled by an incompatible version of rustc
  |
1 | extern crate foo;
  | ^^^^^^^^^^^^^^^^^
  |
  |
  = note: the following crate versions were found:
          crate `foo` compiled by <invalid metadata header>: /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library/libfoo-ffffffff-1.0.rlib
  = help: please recompile that crate using this compiler (rustc 1.78.0-nightly (2e4c4cfaf 2024-02-09)) (consider running `cargo clean` first)
error: aborting due to 1 previous error

For more information about this error, try `rustc --explain E0514`.


[[[ end stdout ]]]
Error: cannot match: found invalid metadata
--- stderr -------------------------------
ar: `u' modifier ignored since `D' is the default (see `U')
make: *** [Makefile:6: all] Error 1
------------------------------------------

Feb 09 '24 21:02 rust-log-analyzer

Finished benchmarking commit (8fb65acf1a11f547302c62c93bf83cca3ff4cd9d): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never @rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.4%	[0.3%, 2.7%]	7
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.4%	[-0.5%, -0.4%]	3
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	7.1%	[5.4%, 8.4%]	4
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-5.2%	[-5.2%, -5.2%]	1
All ❌✅ (primary)	-	-	0

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 666.509s -> 669.311s (0.42%) Artifact size: 307.99 MiB -> 301.34 MiB (-2.16%)

Feb 09 '24 23:02 rust-timer

This should save about 4MB on the download size of a toolchain without any extra targets other than the host:

component	e28fae52d (no split metadata)	8fb65acf1 (split metadata)	diff
rustc	70.54MB	68.39MB	-2.15MB
rust-std	27.63MB	25.61MB	-2MB
rustc-dev	110.84MB	110.90MB	+0.06MB

For Bevy the size of the target dir shrinks a lot:

mode	normal build	`--emit metadata -Zsplit-metadata`	diff	relative diff
debug mode	4.6GiB	4.3GiB	-300MiB	-6.5%
debug mode (no incr comp)	2.8GiB	2.6GiB	-200MiB	-7%
release mode	843.6MiB	614.3MiB	-230MiB	-27%

Feb 10 '24 14:02 bjorn3

:umbrella: The latest upstream changes (presumably #120486) made this pull request unmergeable. Please resolve the merge conflicts.

Feb 16 '24 09:02 bors

@bjorn3 Could you

Split https://github.com/rust-lang/rust/pull/120855/commits/2eb24d9d154eda645bf25fdef2107d2dad215cf5 into a separate PR in case it causes regressions
Split all preparatory refactorings like https://github.com/rust-lang/rust/pull/120855/commits/1065559b2e07d540b7752f7cfa7b2ed1c43177a1 into a separate PR
Rebase the remaining changes, make CI green, cleanup history, and add FIXME comments to logic that only exists due to lacking cargo support

(I'll be able to review the first two PRs quickly.) @rustbot author

Mar 07 '24 15:03 petrochenkov

The job x86_64-gnu-llvm-16 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

#12 writing image sha256:0f06771cd097a20d7c5a8d90707cd1f8d89d350627250128dafe80d7eb71ef3e done
#12 naming to docker.io/library/rust-ci done
#12 DONE 10.1s
##[endgroup]
Setting extra environment values for docker:  --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-16]
##[group]Clock drift check
  local time: Fri Mar  8 11:30:24 UTC 2024
  network time: Fri, 08 Mar 2024 11:30:24 GMT
  network time: Fri, 08 Mar 2024 11:30:24 GMT
##[endgroup]
sccache: Starting the server...
##[group]Configure the build
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-16', '--enable-llvm-link-shared', '--set', 'rust.thin-lto-import-instr-limit=10', '--set', 'change-id=99999999', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'build.optimized-compiler-builtins', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-16/bin/llvm-config
configure: llvm.link-shared     := True
configure: rust.thin-lto-import-instr-limit := 10
configure: change-id            := 99999999
---
##[endgroup]
Testing GCC stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
   Compiling y v0.1.0 (/checkout/compiler/rustc_codegen_gcc/build_system)
    Finished release [optimized] target(s) in 1.26s
     Running `/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-codegen/x86_64-unknown-linux-gnu/release/y test --use-system-gcc --use-backend gcc --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc --release --no-default-features --mini-tests --std-tests`
Using system GCC
Using system GCC
[BUILD] example
[AOT] mini_core_hello_world
/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc/mini_core_hello_world
abc
---
failures:

---- [run-make] tests/run-make/a-b-a-linker-guard stdout ----

error: run-make test failed: could not build `rmake.rs` recipe
status: exit status: 1
command: env -u CARGO_MAKEFLAGS -u MAKEFLAGS -u MFLAGS HOST_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" LD_LIB_PATH_ENVVAR="LD_LIBRARY_PATH" LLVM_COMPONENTS="aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets amdgpu amdgpuasmparser amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgputargetmca amdgpuutils analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen core coroutines coverage debuginfocodeview debuginfodwarf debuginfogsym debuginfologicalview debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker dwarflinkerparallel dwp engine executionengine extensions filecheck frontendhlsl frontendopenacc frontendopenmp fuzzercli fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interfacestub interpreter ipo irprinter irreader jitlink lanai lanaiasmparser lanaicodegen lanaidesc lanaidisassembler lanaiinfo libdriver lineeditor linker loongarch loongarchasmparser loongarchcodegen loongarchdesc loongarchdisassembler loongarchinfo lto m68k m68kasmparser m68kcodegen m68kdesc m68kdisassembler m68kinfo mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts objcopy object objectyaml option orcjit orcshared orctargetprocess passes perfjitevents powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvtargetmca runtimedyld scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target targetparser textapi transformutils ve veasmparser vecodegen vectorize vedesc vedisassembler veinfo webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo webassemblyutils windowsdriver windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86targetmca xcore xcorecodegen xcoredesc xcoredisassembler xcoreinfo xray" PYTHON="/usr/bin/python3" RUSTC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" RUST_BUILD_STAGE="stage2-x86_64-unknown-linux-gnu" S="/checkout" TARGET="x86_64-unknown-linux-gnu" TARGET_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" TMPDIR="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/a-b-a-linker-guard/a-b-a-linker-guard" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/a-b-a-linker-guard/a-b-a-linker-guard/rmake" "-Ldependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools-bin" "-Ldependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/deps" "-Ldependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/release/deps" "--extern" "run_make_support=/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools-bin/librun_make_support.rlib" "/checkout/tests/run-make/a-b-a-linker-guard/rmake.rs"
stdout: none
--- stderr -------------------------------
error[E0463]: can't find crate for `run_make_support`
##[error] --> /checkout/tests/run-make/a-b-a-linker-guard/rmake.rs:3:1
---


---- [run-make] tests/run-make/CURRENT_RUSTC_VERSION stdout ----

error: run-make test failed: could not build `rmake.rs` recipe
status: exit status: 1
command: env -u CARGO_MAKEFLAGS -u MAKEFLAGS -u MFLAGS HOST_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" LD_LIB_PATH_ENVVAR="LD_LIBRARY_PATH" LLVM_COMPONENTS="aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets amdgpu amdgpuasmparser amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgputargetmca amdgpuutils analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen core coroutines coverage debuginfocodeview debuginfodwarf debuginfogsym debuginfologicalview debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker dwarflinkerparallel dwp engine executionengine extensions filecheck frontendhlsl frontendopenacc frontendopenmp fuzzercli fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interfacestub interpreter ipo irprinter irreader jitlink lanai lanaiasmparser lanaicodegen lanaidesc lanaidisassembler lanaiinfo libdriver lineeditor linker loongarch loongarchasmparser loongarchcodegen loongarchdesc loongarchdisassembler loongarchinfo lto m68k m68kasmparser m68kcodegen m68kdesc m68kdisassembler m68kinfo mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts objcopy object objectyaml option orcjit orcshared orctargetprocess passes perfjitevents powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvtargetmca runtimedyld scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target targetparser textapi transformutils ve veasmparser vecodegen vectorize vedesc vedisassembler veinfo webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo webassemblyutils windowsdriver windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86targetmca xcore xcorecodegen xcoredesc xcoredisassembler xcoreinfo xray" PYTHON="/usr/bin/python3" RUSTC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" RUST_BUILD_STAGE="stage2-x86_64-unknown-linux-gnu" S="/checkout" TARGET="x86_64-unknown-linux-gnu" TARGET_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" TMPDIR="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/CURRENT_RUSTC_VERSION/CURRENT_RUSTC_VERSION" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/CURRENT_RUSTC_VERSION/CURRENT_RUSTC_VERSION/rmake" "-Ldependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools-bin" "-Ldependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/deps" "-Ldependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/release/deps" "--extern" "run_make_support=/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools-bin/librun_make_support.rlib" "/checkout/tests/run-make/CURRENT_RUSTC_VERSION/rmake.rs"
--- stderr -------------------------------
error[E0463]: can't find crate for `run_make_support`
##[error] --> /checkout/tests/run-make/CURRENT_RUSTC_VERSION/rmake.rs:3:1
  |
---
---- [run-make] tests/run-make/invalid-library stdout ----

error: make failed
status: exit status: 2
command: cd "/checkout/tests/run-make/invalid-library" && env -u CARGO_MAKEFLAGS -u MAKEFLAGS -u MFLAGS -u RUSTFLAGS AR="ar" CC="cc -ffunction-sections -fdata-sections -fPIC -m64" CXX="c++ -ffunction-sections -fdata-sections -fPIC -m64" HOST_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" LD_LIB_PATH_ENVVAR="LD_LIBRARY_PATH" LLVM_BIN_DIR="/usr/lib/llvm-16/bin" LLVM_COMPONENTS="aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets amdgpu amdgpuasmparser amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgputargetmca amdgpuutils analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen core coroutines coverage debuginfocodeview debuginfodwarf debuginfogsym debuginfologicalview debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker dwarflinkerparallel dwp engine executionengine extensions filecheck frontendhlsl frontendopenacc frontendopenmp fuzzercli fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interfacestub interpreter ipo irprinter irreader jitlink lanai lanaiasmparser lanaicodegen lanaidesc lanaidisassembler lanaiinfo libdriver lineeditor linker loongarch loongarchasmparser loongarchcodegen loongarchdesc loongarchdisassembler loongarchinfo lto m68k m68kasmparser m68kcodegen m68kdesc m68kdisassembler m68kinfo mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts objcopy object objectyaml option orcjit orcshared orctargetprocess passes perfjitevents powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvtargetmca runtimedyld scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target targetparser textapi transformutils ve veasmparser vecodegen vectorize vedesc vedisassembler veinfo webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo webassemblyutils windowsdriver windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86targetmca xcore xcorecodegen xcoredesc xcoredisassembler xcoreinfo xray" LLVM_FILECHECK="/usr/lib/llvm-16/bin/FileCheck" NODE="/usr/bin/node" PYTHON="/usr/bin/python3" RUSTC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" RUSTDOC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" RUST_BUILD_STAGE="stage2-x86_64-unknown-linux-gnu" S="/checkout" TARGET="x86_64-unknown-linux-gnu" TARGET_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" TMPDIR="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library" "make"
touch /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library/lib.rmeta
touch /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library/lib.rmeta
ar crus /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library/libfoo-ffffffff-1.0.rlib /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library/lib.rmeta
LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library  -Ainternal_features foo.rs 2>&1 | "/checkout/src/etc/cat-and-grep.sh" "found invalid metadata"
[[[ begin stdout ]]]
error[E0514]: found crate `foo` compiled by an incompatible version of rustc
  |
1 | extern crate foo;
  | ^^^^^^^^^^^^^^^^^
  |
  |
  = note: the following crate versions were found:
          crate `foo` compiled by <invalid metadata header>: /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/invalid-library/invalid-library/libfoo-ffffffff-1.0.rlib
  = help: please recompile that crate using this compiler (rustc 1.78.0-nightly (ad5760a10 2024-03-08)) (consider running `cargo clean` first)
error: aborting due to 1 previous error

For more information about this error, try `rustc --explain E0514`.


[[[ end stdout ]]]
Error: cannot match: found invalid metadata
--- stderr -------------------------------
ar: `u' modifier ignored since `D' is the default (see `U')
make: *** [Makefile:6: all] Error 1
------------------------------------------

Mar 08 '24 12:03 rust-log-analyzer

:umbrella: The latest upstream changes (presumably #122241) made this pull request unmergeable. Please resolve the merge conflicts.

Mar 09 '24 18:03 bors

The job x86_64-gnu-llvm-17 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

#16 exporting to docker image format
#16 sending tarball 45.4s done
#16 DONE 55.2s
##[endgroup]
Setting extra environment values for docker:  --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-17]
---
sccache: Starting the server...
##[group]Configure the build
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-17', '--enable-llvm-link-shared', '--set', 'rust.thin-lto-import-instr-limit=10', '--set', 'change-id=99999999', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-17/bin/llvm-config
configure: llvm.link-shared     := True
configure: rust.thin-lto-import-instr-limit := 10
configure: change-id            := 99999999
---
   Compiling rustc_smir v0.0.0 (/checkout/compiler/rustc_smir)
error[E0382]: borrow of moved value: `metadata`
   --> compiler/rustc_metadata/src/locator.rs:657:17
    |
572 |             let (hash, metadata) = match get_metadata_section(
    |                        -------- move occurs because `metadata` has type `MetadataBlob`, which does not implement the `Copy` trait
...
655 |             *slot = Some((hash, metadata, lib.clone()));
    |                                 -------- value moved here
656 |
657 |             if !metadata.get_header().is_reference {
    |
help: consider cloning the value if the performance cost is acceptable
    |
    |
655 |             *slot = Some((hash, metadata.clone(), lib.clone()));

For more information about this error, try `rustc --explain E0382`.
error: could not compile `rustc_metadata` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...

Mar 23 '24 16:03 rust-log-analyzer

The job mingw-check failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

    Checking xz2 v0.1.7
error: this `if` statement can be collapsed
   --> src/bin/rustc.rs:115:5
    |
115 | /     if orig_args.iter().any(|arg| arg == "-Zsplit-metadata") {
116 | |         if orig_args.windows(2).any(|args| args[0] == "--crate-type" && args[1] == "dylib") {
117 | |             cmd.arg("--emit").arg("metadata");
119 | |     }
    | |_____^
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#collapsible_if
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#collapsible_if
    = note: `-D clippy::collapsible-if` implied by `-D warnings`
    = help: to override `-D warnings` add `#[allow(clippy::collapsible_if)]`
help: collapse nested if block
    |
115 ~     if orig_args.iter().any(|arg| arg == "-Zsplit-metadata") && orig_args.windows(2).any(|args| args[0] == "--crate-type" && args[1] == "dylib") {
116 +         cmd.arg("--emit").arg("metadata");
    |

error: could not compile `bootstrap` (bin "rustc") due to 1 previous error
Build completed unsuccessfully in 0:01:24

Jun 30 '24 14:06 rust-log-analyzer

:umbrella: The latest upstream changes (presumably #126777) made this pull request unmergeable. Please resolve the merge conflicts.

Jul 11 '24 11:07 bors

rust rust copied to clipboard

Introduce -Zsplit-metadata option

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Instruction count

Max RSS (memory usage)

Cycles

Binary size

rust
rust copied to clipboard