pipeliner
pipeliner copied to clipboard
Add support for async pipelines
Pipeliner was created before Rust's async/.await functionality became available in stable. It uses OS threads for both CPU-bound and I/O-bound computation. But async code is much more efficient for I/O-bound work, since you can efficiently wait for I/O without needing a whole thread.
I'd like to update Pipeliner to support async I/O. Some thoughts:
- allow mixing async/sync code. This would work well for putting I/O-bound code in async executors, and CPU-bound code on threads.
- Is there a way to make it pluggable so it works for tokio and async-std, or do I need to do that manually w/ feature flags?
If you'd like this feature, please leave a comment or👍.