csnappy icon indicating copy to clipboard operation
csnappy copied to clipboard

Maximum size for compressed/uncompressed data

Open avereha opened this issue 9 years ago • 0 comments

What is the maximum size for the compressed and/or uncompressed that this library is supporting? I think it would be nice if we document the desired limits and make the implementation match the documentation.

The snappy format description specifies that the maximum size of the uncompressed data is 2^32 - 1:

https://code.google.com/p/snappy/source/browse/trunk/format_description.txt

The compressed data can be > 2^32(if the uncompressed data is random enough).

In my tests csnappy seems to be able to compress data of 3G, but it is not able to decompress it back. I think the limitation is here:

https://github.com/zeevt/csnappy/blob/master/csnappy_decompress.c#L329

'available' is declared an an signed int and if it is > 2Gb it becomes negative. Is this the desired behaviour(csnappy supports input of maximum 2GB) or a bug? If you need more details please let me know.

avereha avatar Jul 31 '15 14:07 avereha