rust
rust copied to clipboard
rust-lang
Set mmapped files as readonly to prevent other processes from modifying it by accident
Unfortunately this only is a guarantee on windows. Double-unfortunately I don't know what's gonna happen when rustc segfaults on windows and a file is locked. Possibly the file is now deadlocked.
Linux just locks file as a hint, so only tools that know about file locking will actually respect it, others will just access it.
I guess if we had some sort of story for volatile memory we could use that as we're only reading bytes out of it and not actually mapping data structures to that memory.
r? @fee1-dead
rustbot has assigned @fee1-dead. They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.
Use r? to explicitly pick a reviewer
:hourglass: Trying commit 57ff16e7d7bef425304d54f55bbe7198145476b5 with merge 393b4c0828ae2fbf04bf4613c02e0c412db53144...
The job x86_64-gnu-llvm-18 failed! Check out the build log: (web) (plain)
Click to see the possible cause of the failure (guessed by this bot)
#21 exporting to docker image format
#21 sending tarball 28.3s done
#21 DONE 41.3s
##[endgroup]
Setting extra environment values for docker: --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-18]
debug: `DISABLE_CI_RUSTC_IF_INCOMPATIBLE` configured.
---
sccache: Starting the server...
##[group]Configure the build
configure: processing command line
configure:
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-18', '--enable-llvm-link-shared', '--set', 'rust.randomize-layout=true', '--set', 'rust.thin-lto-import-instr-limit=10', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'rust.lld=false', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-18/bin/llvm-config
configure: llvm.link-shared := True
configure: rust.randomize-layout := True
configure: rust.thin-lto-import-instr-limit := 10
---
Compiling rustc_codegen_gcc v0.1.0 (/checkout/compiler/rustc_codegen_gcc)
error[E0277]: the trait bound `std::fs::File: AsRef<std::path::Path>` is not satisfied
--> compiler/rustc_codegen_gcc/src/back/lto.rs:130:27
|
130 | Mmap::map(File::open(path).expect("couldn't open rlib")).expect("couldn't map rlib")
| --------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `AsRef<std::path::Path>` is not implemented for `std::fs::File`
| required by a bound introduced by this call
|
note: required by a bound in `Mmap::map`
--> /checkout/compiler/rustc_data_structures/src/memmap.rs:28:34
--> /checkout/compiler/rustc_data_structures/src/memmap.rs:28:34
|
28 | pub unsafe fn map(path: impl AsRef<Path>) -> io::Result<Self> {
| ^^^^^^^^^^^ required by this bound in `Mmap::map`
For more information about this error, try `rustc --explain E0277`.
error: could not compile `rustc_codegen_gcc` (lib) due to 1 previous error
Build completed unsuccessfully in 0:04:04
local time: Fri Feb 14 12:08:49 UTC 2025
If a process terminates with a portion of a file locked or closes a file that has outstanding locks, the locks are unlocked by the operating system. However, the time it takes for the operating system to unlock these locks depends upon available system resources. Therefore, it is recommended that your process explicitly unlock all files it has locked when it terminates. If this is not done, access to these files may be denied if the operating system has not yet unlocked them.
Though I would also add:
Locking a region of a file does not prevent reading or writing from a mapped file view.
The job dist-x86_64-linux failed! Check out the build log: (web) (plain)
Click to see the possible cause of the failure (guessed by this bot)
file:.git/config remote.origin.url=https://github.com/rust-lang-ci/rust
file:.git/config remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
file:.git/config gc.auto=0
file:.git/config http.https://github.com/.extraheader=AUTHORIZATION: basic ***
file:.git/config branch.try.remote=origin
file:.git/config branch.try.merge=refs/heads/try
file:.git/config remote.upstream.fetch=+refs/heads/*:refs/remotes/upstream/*
file:.git/config submodule.library/backtrace.active=true
file:.git/config submodule.library/backtrace.url=https://github.com/rust-lang/backtrace-rs.git
file:.git/config submodule.library/stdarch.active=true
---
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
error: failed to add native library /rustroot/bin/../lib/gcc/x86_64-pc-linux-gnu/9.5.0/../../../../lib64/libstdc++.a: Permission denied (os error 13)
[RUSTC-TIMING] rustc_llvm test:false 0.079
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
---
Caused by:
Command RUST_BACKTRACE=full python3 /checkout/x.py build --target x86_64-unknown-linux-gnu --host x86_64-unknown-linux-gnu --stage 2 library/std --rust-profile-generate /tmp/tmp-multistage/opt-artifacts/rustc-pgo --set llvm.thin-lto=false --set llvm.link-shared=true [at /checkout/obj] has failed with exit code Some(1)
Stack backtrace:
0: <anyhow::Error>::msg::<alloc::string::String>
at /rust/deps/anyhow-1.0.95/src/backtrace.rs:27:14
1: <opt_dist::exec::CmdBuilder>::run
at /rustc/393b4c0828ae2fbf04bf4613c02e0c412db53144/src/tools/opt-dist/src/exec.rs:80:17
2: <opt_dist::exec::Bootstrap>::run
at /rustc/393b4c0828ae2fbf04bf4613c02e0c412db53144/src/tools/opt-dist/src/exec.rs:181:9
at /rustc/393b4c0828ae2fbf04bf4613c02e0c412db53144/src/tools/opt-dist/src/main.rs:222:13
at /rustc/393b4c0828ae2fbf04bf4613c02e0c412db53144/src/tools/opt-dist/src/main.rs:222:13
4: <opt_dist::timer::TimerSection>::section::<opt_dist::execute_pipeline::{closure#1}::{closure#0}, ()>
at /rustc/393b4c0828ae2fbf04bf4613c02e0c412db53144/src/tools/opt-dist/src/timer.rs:111:22
at /rustc/393b4c0828ae2fbf04bf4613c02e0c412db53144/src/tools/opt-dist/src/main.rs:211:9
at /rustc/393b4c0828ae2fbf04bf4613c02e0c412db53144/src/tools/opt-dist/src/main.rs:211:9
6: <opt_dist::timer::TimerSection>::section::<opt_dist::execute_pipeline::{closure#1}, opt_dist::training::RustcPGOProfile>
at /rustc/393b4c0828ae2fbf04bf4613c02e0c412db53144/src/tools/opt-dist/src/timer.rs:111:22
at /rustc/393b4c0828ae2fbf04bf4613c02e0c412db53144/src/tools/opt-dist/src/main.rs:208:29
8: opt_dist::main
at /rustc/393b4c0828ae2fbf04bf4613c02e0c412db53144/src/tools/opt-dist/src/main.rs:399:18
9: <fn() -> core::result::Result<(), anyhow::Error> as core::ops::function::FnOnce<()>>::call_once
9: <fn() -> core::result::Result<(), anyhow::Error> as core::ops::function::FnOnce<()>>::call_once
at /rustc/38213856a8a3a6d49a234e0d95a722a4f28a2b18/library/core/src/ops/function.rs:250:5
10: std::sys::backtrace::__rust_begin_short_backtrace::<fn() -> core::result::Result<(), anyhow::Error>, core::result::Result<(), anyhow::Error>>
at /rustc/38213856a8a3a6d49a234e0d95a722a4f28a2b18/library/std/src/sys/backtrace.rs:152:18
11: std::rt::lang_start::<core::result::Result<(), anyhow::Error>>::{closure#0}
at /rustc/38213856a8a3a6d49a234e0d95a722a4f28a2b18/library/std/src/rt.rs:195:18
12: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
13: std::panicking::try::do_call
at /rustc/38213856a8a3a6d49a234e0d95a722a4f28a2b18/library/std/src/panicking.rs:584:40
14: std::panicking::try
at /rustc/38213856a8a3a6d49a234e0d95a722a4f28a2b18/library/std/src/panicking.rs:547:19
Some changes occurred in compiler/rustc_codegen_gcc
cc @antoyo, @GuillaumeGomez
:hourglass: Trying commit 0db5406049a4d5be243780d4dd411b0d6d438b5d with merge 91c58543b58b1d3bf3345c863f8b728ffd0b8f23...
The job x86_64-gnu-llvm-18 failed! Check out the build log: (web) (plain)
Click to see the possible cause of the failure (guessed by this bot)
#22 exporting to docker image format
#22 sending tarball 28.8s done
#22 DONE 34.8s
##[endgroup]
Setting extra environment values for docker: --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-18]
debug: `DISABLE_CI_RUSTC_IF_INCOMPATIBLE` configured.
---
sccache: Starting the server...
##[group]Configure the build
configure: processing command line
configure:
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-18', '--enable-llvm-link-shared', '--set', 'rust.randomize-layout=true', '--set', 'rust.thin-lto-import-instr-limit=10', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'rust.lld=false', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-18/bin/llvm-config
configure: llvm.link-shared := True
configure: rust.randomize-layout := True
configure: rust.thin-lto-import-instr-limit := 10
---
Compiling unicase v2.8.1
Compiling tracing-log v0.2.0
Compiling nu-ansi-term v0.50.1
Compiling powerfmt v0.2.0
error: failed to add native library /usr/lib/gcc/x86_64-linux-gnu/13/libstdc++.a: Permission denied (os error 13)
error: could not compile `rustc_llvm` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
Build completed unsuccessfully in 0:06:45
local time: Fri Feb 14 15:09:34 UTC 2025
The job dist-x86_64-linux failed! Check out the build log: (web) (plain)
Click to see the possible cause of the failure (guessed by this bot)
file:.git/config remote.origin.url=https://github.com/rust-lang-ci/rust
file:.git/config remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
file:.git/config gc.auto=0
file:.git/config http.https://github.com/.extraheader=AUTHORIZATION: basic ***
file:.git/config branch.try.remote=origin
file:.git/config branch.try.merge=refs/heads/try
file:.git/config remote.upstream.fetch=+refs/heads/*:refs/remotes/upstream/*
file:.git/config submodule.library/backtrace.active=true
file:.git/config submodule.library/backtrace.url=https://github.com/rust-lang/backtrace-rs.git
file:.git/config submodule.library/stdarch.active=true
---
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
error: failed to add native library /rustroot/bin/../lib/gcc/x86_64-pc-linux-gnu/9.5.0/../../../../lib64/libstdc++.a: Permission denied (os error 13)
[RUSTC-TIMING] rustc_llvm test:false 0.078
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
warning: [email protected]: Inherited flag "-fembed-bitcode=all" is not supported by the currently used CC
---
Caused by:
Command RUST_BACKTRACE=full python3 /checkout/x.py build --target x86_64-unknown-linux-gnu --host x86_64-unknown-linux-gnu --stage 2 library/std --rust-profile-generate /tmp/tmp-multistage/opt-artifacts/rustc-pgo --set llvm.thin-lto=false --set llvm.link-shared=true [at /checkout/obj] has failed with exit code Some(1)
Stack backtrace:
0: <anyhow::Error>::msg::<alloc::string::String>
at /rust/deps/anyhow-1.0.95/src/backtrace.rs:27:14
1: <opt_dist::exec::CmdBuilder>::run
at /rustc/91c58543b58b1d3bf3345c863f8b728ffd0b8f23/src/tools/opt-dist/src/exec.rs:80:17
2: <opt_dist::exec::Bootstrap>::run
at /rustc/91c58543b58b1d3bf3345c863f8b728ffd0b8f23/src/tools/opt-dist/src/exec.rs:181:9
at /rustc/91c58543b58b1d3bf3345c863f8b728ffd0b8f23/src/tools/opt-dist/src/main.rs:222:13
at /rustc/91c58543b58b1d3bf3345c863f8b728ffd0b8f23/src/tools/opt-dist/src/main.rs:222:13
4: <opt_dist::timer::TimerSection>::section::<opt_dist::execute_pipeline::{closure#1}::{closure#0}, ()>
at /rustc/91c58543b58b1d3bf3345c863f8b728ffd0b8f23/src/tools/opt-dist/src/timer.rs:111:22
at /rustc/91c58543b58b1d3bf3345c863f8b728ffd0b8f23/src/tools/opt-dist/src/main.rs:211:9
at /rustc/91c58543b58b1d3bf3345c863f8b728ffd0b8f23/src/tools/opt-dist/src/main.rs:211:9
6: <opt_dist::timer::TimerSection>::section::<opt_dist::execute_pipeline::{closure#1}, opt_dist::training::RustcPGOProfile>
at /rustc/91c58543b58b1d3bf3345c863f8b728ffd0b8f23/src/tools/opt-dist/src/timer.rs:111:22
at /rustc/91c58543b58b1d3bf3345c863f8b728ffd0b8f23/src/tools/opt-dist/src/main.rs:208:29
8: opt_dist::main
at /rustc/91c58543b58b1d3bf3345c863f8b728ffd0b8f23/src/tools/opt-dist/src/main.rs:399:18
9: <fn() -> core::result::Result<(), anyhow::Error> as core::ops::function::FnOnce<()>>::call_once
9: <fn() -> core::result::Result<(), anyhow::Error> as core::ops::function::FnOnce<()>>::call_once
at /rustc/38213856a8a3a6d49a234e0d95a722a4f28a2b18/library/core/src/ops/function.rs:250:5
10: std::sys::backtrace::__rust_begin_short_backtrace::<fn() -> core::result::Result<(), anyhow::Error>, core::result::Result<(), anyhow::Error>>
at /rustc/38213856a8a3a6d49a234e0d95a722a4f28a2b18/library/std/src/sys/backtrace.rs:152:18
11: std::rt::lang_start::<core::result::Result<(), anyhow::Error>>::{closure#0}
at /rustc/38213856a8a3a6d49a234e0d95a722a4f28a2b18/library/std/src/rt.rs:195:18
12: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
13: std::panicking::try::do_call
at /rustc/38213856a8a3a6d49a234e0d95a722a4f28a2b18/library/std/src/panicking.rs:584:40
14: std::panicking::try
at /rustc/38213856a8a3a6d49a234e0d95a722a4f28a2b18/library/std/src/panicking.rs:547:19
The files we mmap are write-once. So couldn't we set them to readonly after creating them? That should be at least as good as locking since making them writable again would require another program going out of its way to bypass this, which isn't the case with advisory locks.
:umbrella: The latest upstream changes (presumably #137046) made this pull request unmergeable. Please resolve the merge conflicts.
The job x86_64-gnu-llvm-18 failed! Check out the build log: (web) (plain)
Click to see the possible cause of the failure (guessed by this bot)
#21 exporting to docker image format
#21 sending tarball 32.9s done
#21 DONE 46.8s
##[endgroup]
Setting extra environment values for docker: --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-18]
debug: `DISABLE_CI_RUSTC_IF_INCOMPATIBLE` configured.
---
sccache: Listening on address 127.0.0.1:4226
##[group]Configure the build
configure: processing command line
configure:
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-18', '--enable-llvm-link-shared', '--set', 'rust.randomize-layout=true', '--set', 'rust.thin-lto-import-instr-limit=10', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'rust.lld=false', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-18/bin/llvm-config
configure: llvm.link-shared := True
configure: rust.randomize-layout := True
configure: rust.thin-lto-import-instr-limit := 10
---
Compiling pulldown-cmark-escape v0.11.0
Compiling unicase v2.8.1
Compiling pathdiff v0.2.3
Compiling tracing-log v0.2.0
error: failed to add native library /usr/lib/gcc/x86_64-linux-gnu/13/libstdc++.a: Permission denied (os error 13)
error: could not compile `rustc_llvm` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
Build completed unsuccessfully in 0:06:47
local time: Thu Feb 20 13:52:21 UTC 2025
The job x86_64-gnu-llvm-18 failed! Check out the build log: (web) (plain)
Click to see the possible cause of the failure (guessed by this bot)
#19 exporting to docker image format
#19 sending tarball 20.1s done
#19 DONE 23.9s
##[endgroup]
Setting extra environment values for docker: --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-18]
[CI_JOB_NAME=x86_64-gnu-llvm-18]
debug: `DISABLE_CI_RUSTC_IF_INCOMPATIBLE` configured.
---
sccache: Listening on address 127.0.0.1:4226
##[group]Configure the build
configure: processing command line
configure:
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-18', '--enable-llvm-link-shared', '--set', 'rust.randomize-layout=true', '--set', 'rust.thin-lto-import-instr-limit=10', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'rust.lld=false', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: build.build := x86_64-unknown-linux-gnu
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-18/bin/llvm-config
configure: llvm.link-shared := True
configure: rust.randomize-layout := True
configure: rust.thin-lto-import-instr-limit := 10
---
Compiling rustc_builtin_macros v0.0.0 (/checkout/compiler/rustc_builtin_macros)
error: unnecessary `unsafe` block
--> compiler/rustc_metadata/src/rmeta/encoder.rs:2251:20
|
2251 | let mmap = unsafe { Some(Mmap::map(path)?) };
| ^^^^^^ unnecessary `unsafe` block
|
= note: `-D unused-unsafe` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(unused_unsafe)]`
error: unnecessary `unsafe` block
--> compiler/rustc_metadata/src/locator.rs:825:24
|
825 | let mmap = unsafe { Mmap::map(filename) };
| ^^^^^^ unnecessary `unsafe` block
error: could not compile `rustc_metadata` (lib) due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
Build completed unsuccessfully in 0:02:15
Btw, memmap2 currently used is super outdated (0.2.3), https://github.com/RazrFalcon/memmap2-rs/blob/v0.9.5/CHANGELOG.md, maybe bump it? Need to review changelog, as there was a lot of changes.
I think the linux CI containers run as root, so we probably should drop CAP_DAC_OVERRIDE to make sure that the permissions don't get bypassed.
OTOH macos and windows builds aren't affected by this so they would catch writes that shouldn't happen.
:umbrella: The latest upstream changes (presumably #137535) made this pull request unmergeable. Please resolve the merge conflicts.