macOS: `cargo-zigbuild` reports `ProcessFdQuotaExceeded` with zig 0.14.0+

[xobs]

Zig Version

0.14.0

Steps to Reproduce and Observed Behavior

1. Install Rust -- curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
2. Get probe-rs -- git clone https://github.com/probe-rs/probe-rs.git
3. Install cargo-zigbuild: cargo install cargo-zigbuild
4. Install the Windows toolchain: rustup target add x86_64-pc-windows-gnu
5. Attempt to build probe-rs: rm -rf target; cargo zigbuild --target x86_64-pc-windows-gnu -p probe-rs-tools

With zig 0.14.0 or newer (including the latest nightly), this fails with an error:

  = note: error: failed to open object /Users/seancross/Code/probe-rs/target/x86_64-pc-windows-gnu/debug/deps/probe_rs-6084bb00951e46ab.e5cjc0ec27acx9jogd9t0pslj.rcgu.o: ProcessFdQuotaExceeded

Note that the build fails with the same error for both Linux and Windows, but the Linux build requires some patches which is why I'm building the Windows version in this example.

Expected Behavior

The binary should build. If using zig 0.13.0, then the binaries build.

[alexrp]

This is probably because MinGW-w64 consists of more C source files in 0.14.0 than it does in 0.13.0, and we compile all of them in parallel. We've seen similar issues with musl in the past.

Try increasing your system's file descriptor limit in small increments and see if that helps.

[xobs]

Yes, by default ulimit -n shows a limit of 256 files. If I set ulimit -n 1024 then it does build when targeting Windows. When targeting x86_64-unknown-linux-musl, I have to set the limit to 4096.

Annoyingly, it seems like I can only set the limit once per session. That is, I can run ulimit -n 1024, but then I can't subsequently run ulimit -n 4096 without closing the shell and opening a new one. It gives an error ulimit: Permission denied when changing resource of type 'Maximum number of open file descriptors'

What would be the correct solution here? Should the number of open file descriptors be limited to the ulimit? Or should the instruction to change ulimit -n be communicated to the user? I can open an issue in cargo-zigbuild, but given that I'm running into it here I wonder if there isn't something zig could do to improve the situation.

[alexrp]

I suppose it might be possible to limit concurrency based on the FD limit in some fashion. But given @andrewrk's comment here, I'll defer to him on this.

[henderkes]

Also running into this when attempting to build php with all somewhat common extensions linked in statically. Upping ulimit -n to 4096 works around it.

[enricoschaaf]

I still get this despite this

Tried multiple versions of zig including 0.16.0-dev.1366+4ea472808

Any recommendations on what could be the cause?