[musl] use posix_spawn if a directory change was requested

[sunshowers]

Currently, not all libcs have the posix_spawn_file_actions_addchdir_np symbol available to them. So we attempt to do a weak symbol lookup for that function. But that only works if libc is a dynamic library -- with statically linked musl binaries the symbol lookup would never work, so we would never be able to use it even if the musl in use supported the symbol.

Now that Rust has a minimum musl version of 1.2.3, all supported musl versions now include this symbol, so we can unconditionally expect it to be there. This symbol was added to libc in https://github.com/rust-lang/libc/pull/3949 -- use it here.

I couldn't find any tests for whether the posix_spawn path is used, but I've verified with cargo-nextest that this change works. This is a substantial improvement to nextest's performance with musl. On my workstation with a Ryzen 7950x, against https://github.com/clap-rs/clap at 61f5ee514f8f60ed8f04c6494bdf36c19e7a8126:

Before:

     Summary [   1.071s] 879 tests run: 879 passed, 0 skipped

After:

     Summary [   0.392s] 879 tests run: 879 passed, 0 skipped

Fixes #99740.

[rustbot]

r? @workingjubilee

rustbot has assigned @workingjubilee. They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

These commits modify the library/Cargo.lock file. Unintentional changes to library/Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged. Otherwise, you can ignore this comment.

[rust-log-analyzer]

The job mingw-check-tidy failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
COPY host-x86_64/mingw-check/validate-error-codes.sh /scripts/

# NOTE: intentionally uses python2 for x.py so we can test it still works.
# validate-toolstate only runs in our CI, so it's ok for it to only support python3.
ENV SCRIPT TIDY_PRINT_DIFF=1 python2.7 ../x.py test \
           --stage 0 src/tools/tidy tidyselftest --extra-checks=py:lint,cpp:fmt
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --allow-unsafe --generate-hashes reuse-requirements.in
---
#13 2.891 Building wheels for collected packages: reuse
#13 2.892   Building wheel for reuse (pyproject.toml): started
#13 3.143   Building wheel for reuse (pyproject.toml): finished with status 'done'
#13 3.144   Created wheel for reuse: filename=reuse-4.0.3-cp310-cp310-manylinux_2_35_x86_64.whl size=132720 sha256=026f3bb0f1aa8090b861fd0a0939cb1a782396d84c8aab7875096557d637a0f6
#13 3.145   Stored in directory: /tmp/pip-ephem-wheel-cache-sl5utpmp/wheels/3d/8d/0a/e0fc6aba4494b28a967ab5eaf951c121d9c677958714e34532
#13 3.147 Installing collected packages: boolean-py, binaryornot, tomlkit, reuse, python-debian, markupsafe, license-expression, jinja2, chardet, attrs
#13 3.549 Successfully installed attrs-23.2.0 binaryornot-0.4.4 boolean-py-4.0 chardet-5.2.0 jinja2-3.1.4 license-expression-30.3.0 markupsafe-2.1.5 python-debian-0.1.49 reuse-4.0.3 tomlkit-0.13.0
#13 3.549 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#13 4.098 Collecting virtualenv
#13 4.098 Collecting virtualenv
#13 4.147   Downloading virtualenv-20.26.6-py3-none-any.whl (6.0 MB)
#13 4.375      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.0/6.0 MB 26.6 MB/s eta 0:00:00
#13 4.438 Collecting filelock<4,>=3.12.2
#13 4.445   Downloading filelock-3.16.1-py3-none-any.whl (16 kB)
#13 4.468 Collecting distlib<1,>=0.3.7
#13 4.477   Downloading distlib-0.3.9-py2.py3-none-any.whl (468 kB)
#13 4.488      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 469.0/469.0 KB 50.6 MB/s eta 0:00:00
#13 4.524 Collecting platformdirs<5,>=3.9.1
#13 4.532   Downloading platformdirs-4.3.6-py3-none-any.whl (18 kB)
#13 4.612 Installing collected packages: distlib, platformdirs, filelock, virtualenv
#13 4.803 Successfully installed distlib-0.3.9 filelock-3.16.1 platformdirs-4.3.6 virtualenv-20.26.6
#13 DONE 4.9s

#14 [7/8] COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
#14 DONE 0.0s
---
DirectMap4k:      202688 kB
DirectMap2M:    10283008 kB
DirectMap1G:     8388608 kB
##[endgroup]
Executing TIDY_PRINT_DIFF=1 python2.7 ../x.py test            --stage 0 src/tools/tidy tidyselftest --extra-checks=py:lint,cpp:fmt
+ TIDY_PRINT_DIFF=1 python2.7 ../x.py test --stage 0 src/tools/tidy tidyselftest --extra-checks=py:lint,cpp:fmt
    Finished `dev` profile [unoptimized] target(s) in 0.04s
##[endgroup]
downloading https://static.rust-lang.org/dist/2024-10-16/rustfmt-nightly-x86_64-unknown-linux-gnu.tar.xz
extracting /checkout/obj/build/cache/2024-10-16/rustfmt-nightly-x86_64-unknown-linux-gnu.tar.xz to /checkout/obj/build/x86_64-unknown-linux-gnu/rustfmt
---
+    use crate::sys::pal::unix::linux::pidfd as imp;
     use crate::sys_common::FromInner;
     use crate::{io, mem};
 
fmt error: Running `"/checkout/obj/build/x86_64-unknown-linux-gnu/rustfmt/bin/rustfmt" "--config-path" "/checkout" "--edition" "2021" "--unstable-features" "--skip-children" "--check" "/checkout/library/std/src/sys/pal/itron/abi.rs" "/checkout/library/std/src/sys/pal/itron/task.rs" "/checkout/library/std/src/sys/pal/itron/thread_parking.rs" "/checkout/library/std/src/sys/pal/itron/thread.rs" "/checkout/library/std/src/sys/pal/itron/spin.rs" "/checkout/library/std/src/sys/pal/itron/error.rs" "/checkout/library/std/src/sys/pal/itron/time.rs" "/checkout/library/std/src/sys/pal/mod.rs" "/checkout/compiler/rustc_data_structures/src/stable_hasher.rs" "/checkout/compiler/rustc_data_structures/src/sorted_map/tests.rs" "/checkout/compiler/rustc_data_structures/src/sorted_map/index_map.rs" "/checkout/compiler/rustc_data_structures/src/svh.rs" "/checkout/compiler/rustc_data_structures/src/intern.rs" "/checkout/compiler/rustc_data_structures/src/flock.rs" "/checkout/library/std/src/sys/pal/wasm/atomics/thread.rs" "/checkout/library/std/src/sys/pal/wasm/atomics/futex.rs" "/checkout/library/std/src/sys/pal/wasm/env.rs" "/checkout/library/std/src/sys/pal/wasm/mod.rs" "/checkout/compiler/rustc_data_structures/src/binary_search_util/tests.rs" "/checkout/compiler/rustc_data_structures/src/binary_search_util/mod.rs" "/checkout/compiler/rustc_data_structures/src/stable_hasher/tests.rs" "/checkout/compiler/rustc_data_structures/src/snapshot_map/tests.rs" "/checkout/compiler/rustc_data_structures/src/snapshot_map/mod.rs" "/checkout/compiler/rustc_data_structures/src/sharded.rs" "/checkout/compiler/rustc_data_structures/src/small_c_str.rs" "/checkout/compiler/rustc_data_structures/src/tagged_ptr/drop.rs" "/checkout/compiler/rustc_data_structures/src/tagged_ptr/copy.rs" "/checkout/compiler/rustc_data_structures/src/tagged_ptr/impl_tag/tests.rs" "/checkout/compiler/rustc_data_structures/src/tagged_ptr/impl_tag.rs" "/checkout/library/std/src/sys/pal/unix/stack_overflow.rs" "/checkout/library/std/src/sys/pal/unix/os.rs" "/checkout/library/std/src/sys/pal/unix/l4re.rs" "/checkout/compiler/rustc_data_structures/src/tagged_ptr/drop/tests.rs" "/checkout/library/std/src/sys/pal/unix/os/tests.rs" "/checkout/library/std/src/sys/pal/unix/fs.rs" "/checkout/library/std/src/sys/pal/unix/stdio.rs" "/checkout/library/std/src/sys/pal/unix/net.rs" "/checkout/compiler/rustc_data_structures/src/tagged_ptr/copy/tests.rs" "/checkout/compiler/rustc_data_structures/src/temp_dir.rs" "/checkout/library/std/src/sys/pal/unix/fd/tests.rs" "/checkout/compiler/rustc_data_structures/src/graph/reference.rs" "/checkout/library/std/src/sys/pal/unix/process/process_fuchsia.rs" "/checkout/library/std/src/sys/pal/unix/process/process_unsupported.rs" "/checkout/compiler/rustc_data_structures/src/graph/iterate/tests.rs" "/checkout/compiler/rustc_data_structures/src/graph/iterate/mod.rs" "/checkout/compiler/rustc_data_structures/src/graph/vec_graph/tests.rs" "/checkout/compiler/rustc_data_structures/src/graph/vec_graph/mod.rs" "/checkout/library/std/src/sys/pal/unix/process/process_unsupported/wait_status/tests.rs" "/checkout/library/std/src/sys/pal/unix/process/process_unsupported/wait_status.rs" "/checkout/library/std/src/sys/pal/unix/process/process_unix.rs" "/checkout/compiler/rustc_data_structures/src/graph/dominators/tests.rs" "/checkout/compiler/rustc_data_structures/src/graph/dominators/mod.rs" "/checkout/compiler/rustc_data_structures/src/graph/tests.rs" "/checkout/library/std/src/sys/pal/unix/process/process_common/tests.rs" "/checkout/library/std/src/sys/pal/unix/process/process_common.rs" "/checkout/compiler/rustc_data_structures/src/graph/implementation/tests.rs" "/checkout/compiler/rustc_data_structures/src/graph/implementation/mod.rs" "/checkout/library/std/src/sys/pal/unix/process/process_unix/tests.rs" "/checkout/library/std/src/sys/pal/unix/process/process_vxworks.rs" "/checkout/library/std/src/sys/pal/unix/process/zircon.rs" "/checkout/library/std/src/sys/pal/unix/process/mod.rs" "/checkout/compiler/rustc_data_structures/src/graph/scc/tests.rs" "/checkout/compiler/rustc_data_structures/src/graph/scc/mod.rs" "/checkout/library/std/src/sys/pal/itron/time/tests.rs"` failed.
If you're running `tidy`, try again with `--bless`. Or, if you just want to format code, run `./x.py fmt` instead.
  local time: Thu Oct 17 20:26:17 UTC 2024
  network time: Thu, 17 Oct 2024 20:26:17 GMT
##[error]Process completed with exit code 1.
Post job cleanup.

[the8472]

I couldn't find any tests for whether the posix_spawn path is used

Well, posix_spawn is an implementation detail, so there's no API that would let you distinguish that.

But you can do the opposite: Have a test that covers command.current_dir() (this test already exists) and hope that it uses posix_spawn (which can be manually verified via strace) and then write a second test that combines current_dir and pre_exec to make sure the fork path gets exercised too.

https://github.com/rust-lang/rust/blob/86bd45979a964678b40b79156744f0057759d840/library/std/src/sys/pal/unix/linux/pidfd/tests.rs#L47-L49

[workingjubilee]

Hm. I guess one can write internal unit-tests where the APIs are exposed? But doing that with multiprocess situations sounds... treacherous.

[sunshowers]

Yeah, a test attempting to exercise both paths sounds good. I'll work on that once #131823 lands.

Hm. I guess one can write internal unit-tests where the APIs are exposed? But doing that with multiprocess situations sounds... treacherous.

What I would personally do is abstract the posix_spawn/fork decision into a function which returns an enum, and test that. I don't know how feasible that is, and I'd rather not block this relatively straightforward PR on that refactor :)

[workingjubilee]

Yeah, I don't wanna scope creep ya.

[sunshowers]

All right, rebased and added a test to exercise both paths.

[rust-log-analyzer]

The job mingw-check failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

#16 2.621 Building wheels for collected packages: reuse
#16 2.622   Building wheel for reuse (pyproject.toml): started
#16 2.859   Building wheel for reuse (pyproject.toml): finished with status 'done'
#16 2.860   Created wheel for reuse: filename=reuse-4.0.3-cp310-cp310-manylinux_2_35_x86_64.whl size=132720 sha256=026f3bb0f1aa8090b861fd0a0939cb1a782396d84c8aab7875096557d637a0f6
#16 2.861   Stored in directory: /tmp/pip-ephem-wheel-cache-l194t5jp/wheels/3d/8d/0a/e0fc6aba4494b28a967ab5eaf951c121d9c677958714e34532
#16 2.863 Installing collected packages: boolean-py, binaryornot, tomlkit, reuse, python-debian, markupsafe, license-expression, jinja2, chardet, attrs
#16 3.228 Successfully installed attrs-23.2.0 binaryornot-0.4.4 boolean-py-4.0 chardet-5.2.0 jinja2-3.1.4 license-expression-30.3.0 markupsafe-2.1.5 python-debian-0.1.49 reuse-4.0.3 tomlkit-0.13.0
#16 3.228 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#16 DONE 3.3s

[workingjubilee]

@bors try

[bors]

:hourglass: Trying commit a9016c8c2228211b38175cc834048969df38db74 with merge b6417643b9f46a4173307d09f52d0efea5915433...

[rust-log-analyzer]

The job dist-various-2 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

file:.git/config remote.origin.url=https://github.com/rust-lang-ci/rust
file:.git/config remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
file:.git/config gc.auto=0
file:.git/config http.https://github.com/.extraheader=AUTHORIZATION: basic ***
file:.git/config branch.try.remote=origin
file:.git/config branch.try.merge=refs/heads/try
file:.git/config remote.upstream.fetch=+refs/heads/*:refs/remotes/upstream/*
file:.git/config submodule.library/backtrace.active=true
file:.git/config submodule.library/backtrace.url=https://github.com/rust-lang/backtrace-rs.git
file:.git/config submodule.library/stdarch.active=true

[bors]

:broken_heart: Test failed - checks-actions

[workingjubilee]

yep, goes kaput on the solaris build.

you may want to cross-compile for the illumos stdlib while testing this.

[sunshowers]

@rustbot review

[workingjubilee]

nice!

[bors]

:pushpin: Commit 7f74c894b0e31f370b5321d94f2ca2830e1d30fd has been approved by workingjubilee

It is now in the queue for this repository.