csnappy
csnappy copied to clipboard
Maximum size for compressed/uncompressed data
What is the maximum size for the compressed and/or uncompressed that this library is supporting? I think it would be nice if we document the desired limits and make the implementation match the documentation.
The snappy format description specifies that the maximum size of the uncompressed data is 2^32 - 1:
https://code.google.com/p/snappy/source/browse/trunk/format_description.txt
The compressed data can be > 2^32(if the uncompressed data is random enough).
In my tests csnappy seems to be able to compress data of 3G, but it is not able to decompress it back. I think the limitation is here:
https://github.com/zeevt/csnappy/blob/master/csnappy_decompress.c#L329
'available' is declared an an signed int and if it is > 2Gb it becomes negative. Is this the desired behaviour(csnappy supports input of maximum 2GB) or a bug? If you need more details please let me know.