zstd-ruby icon indicating copy to clipboard operation
zstd-ruby copied to clipboard

Streaming decompression questions

Open gucki opened this issue 1 year ago • 2 comments
trafficstars

I have a tcp connection that receives the zstd compressed data. Let's assume I do a data = socket.read(16*1024) and this fills my data with a zstd frame and another partial zstd frame. If I use decompressor.decompress(buf) (decompressor being an instance of Zstd::StreamingDecompress) I only get back the data of the first frame. How can I know how much data from data has not been processed and needs to be reused after receiving more data from the socket? Afaik the original zstd library returns this information, but the ruby abstraction discards it?

gucki avatar May 15 '24 17:05 gucki

Thank you for the issues. I would like to understand more about what you mean by "the original zstd library returns this information." Could you please provide more details or an example of how the original zstd library provides this information?

SpringMT avatar May 16 '24 04:05 SpringMT

Hey @SpringMT , I just implemented a new method that returns the needed details to the caller and opened a PR. If the example/ use case in the PR is unclear, please let me know and I'll explain further. I tested my implementation quite a bit and it works without any issues so far.

gucki avatar May 16 '24 11:05 gucki