Simple threaded example causes valgrind to hang indefinitely

[cmazakas]

Given a toy C++ example:

#include <iostream>
#include <thread>

#include <liburing.h>

int main() {
  struct io_uring ring;
  io_uring_queue_init(16, &ring, 0);

  auto t = std::thread([&]{
    std::this_thread::sleep_for(std::chrono::seconds(1));
    std::cout << "foreign thread waking up the io_uring context now..." << std::endl;
    struct io_uring_sqe *sqe = io_uring_get_sqe(&ring);
    io_uring_prep_nop(sqe);
    io_uring_submit(&ring);
    std::cout << "nop submitted!" << std::endl;
  });

  std::cout << "Going to block on the io_uring reactor now..." << std::endl;

  struct io_uring_cqe *cqe = nullptr;
  io_uring_wait_cqe(&ring, &cqe);
  io_uring_cqe_seen(&ring, cqe);

  t.join();

  std::cout << "nop was completed!" << std::endl;

  io_uring_queue_exit(&ring);
}

This will cause valgrind to hang seemingly indefinitely on the io_uring_wait_cqe(&ring, &cqe); call. This example seems to work fine otherwise. Is there something I'm missing? Is it UB to submit to the ring from a separate thread?

[CarterLi]

You must not share io_uring instances between threads. They are not thread safe,

[cmazakas]

I've actually run into this when refactoring to use a Unix domain socket to communicate with the io_uring context.

i.e. I'll call io_uring_prep_read(...); io_uring_submit(&ring); io_uring_wait_cqe(...); and then a separate thread calls ::write() on the client Unix domain socket.

I think what's happening is, io_uring _wait_cqe() is busy-waiting and because valgrind serializes all threads, that's what why it's blocking indefinitely.

[DylanZA]

You must not share io_uring instances between threads. They are not thread safe,

not true - that should work fine.

io_uring _wait_cqe() is busy-waiting and because valgrind serializes all threads, that's what why it's blocking indefinitely

yes - it seems to be a valgrind issue: https://github.com/tklengyel/valgrind/blob/xen-patches/coregrind/m_syswrap/syswrap-linux.c

I am only now looking at valgrind's code, but it seems sys_io_uring_enter should have

   *flags |= SfMayBlock;

I'll try find time to send a request, unless you wish to :)

[DylanZA]

someone already has submitted this: https://bugs.kde.org/show_bug.cgi?id=428364