jobserver-rs icon indicating copy to clipboard operation
jobserver-rs copied to clipboard

New jobserver type in GNU Make 4.4

Open bjorn3 opened this issue 2 years ago • 5 comments

Docs: https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver Release announcement: https://lwn.net/Articles/913253/

On systems that support it GNU Make 4.4 will now use --jobserver-auth=fifo:PATH where PATH points to a named pipe rather than --jobserver-auth=R,W. I haven't checked if this breaks anything, but based on looking at the code it seems like it will cause the jobserver to be ignored entirely.

bjorn3 avatar Oct 31 '22 20:10 bjorn3

I can confirm that the new (default since GNU make 4.4) fifo method makes this crate ignore the jobserver. That unfortunately includes Cargo.

Here's a reproducer: https://github.com/kaspar030/jobserver-rs-fifo-test

as a workaround, use --jobserver-style=pipe as top-level make option.

Here's the output from above's test application:

❯ make -j2
echo $MAKEFLAGS
-j2 --jobserver-auth=fifo:/tmp/GMfifo1775897
cargo build
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
target/debug/jobserver-rs-fifo-test
MAKEFLAGS= -j2 --jobserver-auth=fifo:/tmp/GMfifo1775897
MFLAGS=-j2 --jobserver-auth=fifo:/tmp/GMfifo1775897
None
jobserver-rs-fifo-test on  main 
❯ make -j2 --jobserver-style=pipe
echo $MAKEFLAGS
-j2 --jobserver-auth=3,4 --jobserver-auth=-2,-2
cargo build
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
target/debug/jobserver-rs-fifo-test
MAKEFLAGS= -j2 --jobserver-auth=3,4 --jobserver-auth=-2,-2
MFLAGS=-j2 --jobserver-auth=3,4 --jobserver-auth=-2,-2
Some(Client { inner: Client { read: File { fd: 3, path: "pipe:[31836913]", read: true, write: false }, write: File { fd: 4, path: "pipe:[31836913]", read: false, write: true } } })

kaspar030 avatar Jan 26 '23 13:01 kaspar030

#49 probably resolves the compatibility of new GNU Make 4.4. For creating a new client in fifo style, the idea is still brewing. If anyone has a great design of that, I am willing to listen and help :)

weihanglo avatar Feb 27 '23 10:02 weihanglo

For safety/reliability reasons this should become the preferred jobserver protocol in the future since it avoids mucking with file descriptors based on environment variables where inherited environment variables and inherited descriptors can get out of sync.

the8472 avatar Sep 08 '23 13:09 the8472

Could you be more specific about the workaround? How do I pass --jobserver-style=pipe? I have tried MAKEFLAGS=--jobserver-style=pipe cargo build but it didn't fix the problem for me.

anacrolix avatar Jan 24 '24 06:01 anacrolix

as a workaround, use --jobserver-style=pipe as top-level make option.

See --jobserver-type in GNU make manual.

weihanglo avatar Jan 24 '24 07:01 weihanglo