zig icon indicating copy to clipboard operation
zig copied to clipboard

fix deadlock by making the child process API support passing a File as stdin and use it in std.Build.Step.Run

Open andrewrk opened this issue 2 years ago • 2 comments

Extracted from #16358, follow-up from #16051.

Problem 1: The value discarded here...

https://github.com/ziglang/zig/blob/a7553107345bab99b0f5318e0fd3efae84f56b56/lib/std/Build/Step/Run.zig#L479

...could have been used to obtain the file contents since the cache system already needed to hash the file.

Problem 2: Instead of writing the file via a pipe to stdin, the child process API should allow merely opening the file handle and passing that in as the stdin file descriptor directly. No pipe.

https://github.com/ziglang/zig/blob/a7553107345bab99b0f5318e0fd3efae84f56b56/lib/std/Build/Step/Run.zig#L1192-L1203

andrewrk avatar Jul 09 '23 19:07 andrewrk

Marking as "bug" and bumping priority because the status quo implementation deadlocks if the child process writes enough data to stdout and then tries to read from stdin.

andrewrk avatar Jul 20 '23 23:07 andrewrk

Marking as "bug" and bumping priority because the status quo implementation deadlocks if the child process writes enough data to stdout and then tries to read from stdin.

I discovered this issue because I was encountering deadlocks when spawning zig ast-check over stdin on multiple threads. Writing into stdout wasn't even necessary. The child process just needed to be spawned multiple times and read from stdin.

Reproduction using build system:

const std = @import("std");
pub fn build(b: *std.Build) void {
    for (0..8) |_| {
        const run = std.Build.Step.Run.create(b, "run ast-check");
        run.addArgs(&.{ b.graph.zig_exe, "ast-check" });
        run.setStdIn(.{ .bytes = "test {}" });
        b.getInstallStep().dependOn(&run.step);
    }
}

Reproduction without build system:

const std = @import("std");

pub fn main() !void {
    var threads: [8]std.Thread = undefined;
    for (&threads) |*thread| thread.* = try std.Thread.spawn(.{}, worker, .{});
    for (threads) |thread| thread.join();
}

fn worker() void {
    var process = std.process.Child.init(&[_][]const u8{ "zig", "ast-check" }, std.heap.page_allocator);
    process.stdin_behavior = .Pipe;
    process.stdout_behavior = .Ignore;
    process.stderr_behavior = .Ignore;

    process.spawn() catch unreachable;
    process.stdin.?.writeAll("test {}") catch unreachable;
    process.stdin.?.close();
    process.stdin = null;

    const term = process.wait() catch unreachable;
    if (term != .Exited or term.Exited != 0) unreachable;
}

Techatrix avatar Feb 20 '24 02:02 Techatrix