Add cooperative task yielding to this runtime

[notgull]

tokio uses cooperative task yielding to improve tail latencies in the circumstance that a resource polled in a loop fails to yield, leading to starvation of other tasks. Something similar could be set up for async-io as well.

The Reactor already has a concept of "ticks", which are used to coordinate polling. My idea would be that each Source keeps track of the tick which it last returned Poll::Pending. If it compares tick and this theoretical last_pending_tick value and finds that the difference between the two is a large value (say, 10_000, but this could use some tweaking), pooling the Source automatically returns Pending.

Any thoughts on this? This could be a global counter as well, or maybe a thread-local one.

I would be willing to implement this if the functionality is desired.

[taiki-e]

async-io previously had something similar, but it has been removed: https://github.com/smol-rs/async-io/commit/4e7cdb6a48d94864d3106adb773634f747899f3f

[notgull]

Why was it removed? Perhaps the RNG added too much overhead, or it yielded too much?

[taiki-e]

I don't know the exact reason, but the tokio's cooperative scheduling had compatibility issues with various primitives in futures and tokio that do not participate in the cooperative scheduling. (maybe a popular one is this case, finally fixed by https://github.com/rust-lang/futures-rs/pull/2551) I guess there may have been a similar problem.

[notgull]

That makes sense. I'll close this issue.