How to reuse context for stream decompression?

[rocallahan]

The stream Decoder can only decompress one stream. For each independent stream you decompress, you have to create a new zstd context.

The block API gives you a Decompressor that can decompress many blocks, but you have to know an upper bound on the decompressed data size in advance and pass it as capacity.

I guess one can use Decompressor and store the uncompressed size at the start of the block. But it's not ideal. I think Decoder needs a way to decode multiple zstd streams while reusing the context.

[rocallahan]

Actually I need Encoder to be able to reuse the context as well.

[rocallahan]

I figured out how to use the block API for my use case. If you don't know the decompressed data size you can call Decompressor::decompress_to_buffer in a loop, expanding the buffer by a constant factor each time it's found to be too small. If you share the buffer between decompressions this is relatively cheap.

However, if you want to decompress many standard zstd streams, this issue is still relevant.