zstd icon indicating copy to clipboard operation
zstd copied to clipboard

Shrink literal decoding buffer for streaming compressiong

Open terrelln opened this issue 2 years ago • 0 comments

In streaming compression, memory usage is very important. We recently added the capability to split the literals between the output buffer & the dctx. But the size of this buffer is fixed at compile time. We should either:

  1. Allow this to be configurable at runtime, so that users can choose the speed vs. memory usage tradeoff.
  2. Shrink the compile time constant down to be small enough to suit most use cases, like 4KB or maybe 16KB.

terrelln avatar Apr 11 '23 18:04 terrelln