pipeliner icon indicating copy to clipboard operation
pipeliner copied to clipboard

Add support for async pipelines

Open NfNitLoop opened this issue 2 years ago • 0 comments

Pipeliner was created before Rust's async/.await functionality became available in stable. It uses OS threads for both CPU-bound and I/O-bound computation. But async code is much more efficient for I/O-bound work, since you can efficiently wait for I/O without needing a whole thread.

I'd like to update Pipeliner to support async I/O. Some thoughts:

  • allow mixing async/sync code. This would work well for putting I/O-bound code in async executors, and CPU-bound code on threads.
  • Is there a way to make it pluggable so it works for tokio and async-std, or do I need to do that manually w/ feature flags?

If you'd like this feature, please leave a comment or👍.

NfNitLoop avatar Mar 06 '23 17:03 NfNitLoop