[Feature] Support Idle Callback in TaskGroup for flexibility usage

Open royguo opened this issue 3 weeks ago • 7 comments

Implement Idle Hook: Added TaskGroup::SetWorkerIdleCallback to allow executing custom logic (e.g., IO polling) when a worker thread is idle.
Support Timeout Wait: Modified ParkingLot::wait to support an optional timeout, preventing workers from sleeping indefinitely when an idle callback is registered.
Enable Thread-per-Core IO: Enabled thread-local IO management (like io_uring ) by invoking the hook within the worker's thread context.
Add Unit Test: Added bthread_idle_unittest to verify worker isolation and idle callback execution.

The main reason that we need this is that:

We want to make sure all iouring cqe (or other similar async engine, including some network call results), can be signaled within its original task group (we don't want cross-thread signal, which is very slow under observation)
By using a user-defined callback, we can implement the following strategy:

bthread submit iouriing, and tries to reap cqe result
if no cqe found, wait() here and next bthread will wake it up.
But, if the current bthread is the last one, then we will rely on the Idle Callback in the task group to wake it up.

Then we will make the whole stack thread-per-core and iouring-per-thread, we don't need another polling thread to reap all the CQEs, which will not be easy to avoid cross-thread signaling.

Issue Number: none

Problem Summary:

Changed:

task_group.h/.cc
- Added a new function and a few related static member variables to handle idle callbacks
parking_lot.h
- Added a new timeout param to wait() function, with default NULL value, which will not break current implementation.

Side effects:

Dec 09 '25 04:12 royguo