tokio icon indicating copy to clipboard operation
tokio copied to clipboard

Provide Sink to AsyncWrite adaptor

Open hutchisr opened this issue 3 years ago • 4 comments

tokio-util provides StreamReader and ReaderStream to convert a Stream to AsyncRead or AsyncRead to Stream, but there aren't corresponding tools for Sink and AsyncWrite. The codec module does appear to provide a way to go from an AsyncWrite to a Sink but nothing for going from Sink to AsyncWrite.

I've run into a situation where I want to generate an archive (which takes an AsyncWrite) and stream it somewhere else (as opposed to writing it to the filesystem), but without being able to create something like a channel with the send side being an AsyncWrite I don't see a way to do this.

hutchisr avatar May 05 '21 02:05 hutchisr

This is indeed somewhat tricky to do. It's rather easy to make something that cuts the byte stream into chunks arbitrarily, but if you want more structure in the messages given to the sink, it gets tricky. The logic would be similar to an FramedRead.

Darksonn avatar May 05 '21 09:05 Darksonn

I ran into a similar issue recently and I was able to use rw-stream-sink, although that comes with two noteworthy caveats: 1. it only accepts a type that implements both Sink and Stream and 2. you need to use the compatibility traits since it returns futures-io versions of the AsyncRead/AsyncWrite traits.

Example code (if you use their hidden Wrapper type):

RwStreamSink::new(Wrapper(read, write)).compat()

themaxdavitt avatar May 19 '21 17:05 themaxdavitt

Would it be possible to define something that looks like Framed where user specifies a codec on how to split byte stream into chunks?

minghuaw avatar Jul 29 '22 06:07 minghuaw

Yeah, that probably would not be impossible.

Darksonn avatar Jul 29 '22 07:07 Darksonn