binary icon indicating copy to clipboard operation
binary copied to clipboard

Provide buffer-agnostic interface for the Get monad.

Open schernichkin opened this issue 9 years ago • 1 comments

Despite the fact that binary was primary designed with lazy bytestrins in mind, it also very useful for generic serialization with various types of buffers. And it works quite well for Put monad: we have execPut :: PutM a -> Builder latter we can run this builder with runBuilder :: Builder -> BufferWriter from Data.ByteString.Builder.Extra on any type of buffer: the interface requires vanilla pointer of type Ptr Word8 and also provides information about minimal buffer size and number of bytes written to buffer. This function can be used to generate lazy bytestring, to write directly to device buffer, I use this function to write data to preallocated memory buffer in my application and then to generate strict bytestring from it.

Unfortunately similar functionality for Get monad is missed from the package. The closest function we have is runGetIncremental :: Get a -> Decoder a but it still operates on ByteStrings (not on pointers) and it even had serious performance issue until https://github.com/kolmodin/binary/commit/7532daa8789e5199109bb1fcde367d71effb07e2 Even been fixed, it still introduces some overhead by relatively complex handling logic, and this can be noticeable effect on very small Getters (in my application I have a getter which reads 6 bytes, even very small overhead on it affects performance).

I think Decoder should be redesigned to operate on pointers and buffer size information in similar way as Builder from the Data.ByteString does. Having such interface it would be easy to create ByteString version, as well as any custom buffer reader with minimal overhead.

schernichkin avatar Jun 10 '16 10:06 schernichkin

Yes, I'd like to see Binary redesigned to lower the overhead, especially for small Getters. I just haven't had time to do this yet.

kolmodin avatar Jun 13 '16 11:06 kolmodin