Open fifo blocks other fibers (macOS)

[jpcamara]

A core ruby test in make test-all is failing with MN threads on macOS, and so I tested it on async+io-event to see the behavior there. It gets stuck in a (seemingly) identical way. The following script hangs indefinitely, and is based on the test_open_fifo_does_not_block_other_threads thread test found in test/ruby/test_io.rb (https://github.com/ruby/ruby/blob/master/test/ruby/test_io.rb).

require "tempfile"
require "async"

def mkcdtmpdir
  Dir.mktmpdir {|d|
    Dir.chdir(d) {
      yield
    }
  }
end

mkcdtmpdir {
  File.mkfifo("fifo")

  Async do |task|
    t1 = task.async {
      open("fifo", "r") {|r|
        r.read
      }
    }
    t2 = task.async {
      open("fifo", "w") {|w|
        w.write "foo"
      }
    }
    # Hangs indefinitely
    puts t1.wait
  end
}

The "fix" i've found so far for MN threads is to change the rb_io_read_memory call:

// return (ssize_t)rb_thread_io_blocking_call(internal_read_func, &iis, fptr->fd, RB_WAITFD_IN);
return (ssize_t)rb_thread_io_blocking_call(internal_read_func, &iis, fptr->fd, RB_WAITFD_IN | RB_WAITFD_OUT);

Once it goes from RB_WAITFD_IN to RB_WAITFD_IN | RB_WAITFD_OUT it starts "working" correctly. I haven't had a chance to dig into it much yet, but since it has the same stuck issue in the fiber scheduler i'm also opening an issue here.

[ioquatix]

Okay I will take a look thanks for the detailed report and reproduction.

[jpcamara]

Yw! I'm noticing now that it never even makes it to t1.wait

mkcdtmpdir {
  File.mkfifo("fifo")

  puts "async"
  Async do |task|
    puts "t1.async"
    t1 = task.async {
      open("fifo", "r") {|r|
        r.read
      }
    }
    puts "t2.async"
    t2 = task.async {
      open("fifo", "w") {|w|
        w.write "foo"
      }
    }
    # Hangs indefinitely
    puts "time to wait"
    puts t1.wait
  end
}

All I see is

async
t1.async

and then it hangs - so it seems to get stuck just trying to even run t1 at all.

[ioquatix]

https://github.com/socketry/io-event/pull/91

Trying to add some tests that reproduce the issue.

[ioquatix]

I've merged the test.

FIFOs are a bit tricky, as they must be opened on both ends in order to work. It seems like opening it first with "w+" is good enough, otherwise I guess open itself might be blocking.

We don't support non-blocking open at this time, but it's something we could probably introduce with io_uring.

[ioquatix]

The following version of your code works:

File.mkfifo("fifo")

Async do |task|
  puts "t1.async"
  t1 = task.async {
    open("fifo", "r+") {|r|
      r.read(3)
    }
  }
  puts "t2.async"
  t2 = task.async {
    open("fifo", "w+") {|w|
      w.write "foo"
    }
  }
  # Hangs indefinitely
  puts "time to wait"
  puts t1.wait
end

I believe because r+ and w+ won't block.

[ioquatix]

As a follow up, we could consider introducing an io_open hook, but it would only work on the io_uring backend.