Sink & Stream and Codecs in `capnp-futures`

[comath]

I think we can use tokio's codecs to implement configurable compression for async reads and writes, if we add 3 features to capnp-futures:

1. Support for writing messages to sinks and streams
2. Creating a codec that de-serializes, taking a AsyncRead to a stream making capnp::message (and vice a versa)
3. Creating a codec like the normal one that implements packing.

Then the current functions in the futures crate can be implemented using tokio's codec utilities. We can even add a 3rd codec that does zlib compression too. @dwrensha has mentioned that the RPC system would like this too. This would resolve https://github.com/capnproto/capnproto-rust/issues/228

There's a few interesting & cute options here too, the codecs could write the de-serialized messages to internal buffers and return a reader that refers to that buffer to reduce the number of allocations. But, this kind of optimization would be done later, if at all.

I am planning on doing the work, if it's welcome. It would be a large change to the futures crate, so I'm asking first.

[dwrensha]

Would this require making the capnp-futures crate depend on a tokio crate? I want to avoid that, to give users maximal flexibility on using whatever async runtime they prefer.

1. Support for writing messages to sinks and sources

What's a "source" in this context? Do you just mean futures::stream::Stream?

[comath]

Yea that's what I meant, sorry. Edited to fix this brain-typo.

It would mean a tokio dependency, but I really would love to be able to asynchronously read and write compressed messages. It could be another crate, capnp-tokio?

[AelMalinka]

there is also async_codec which does not depend on tokio