python-snappy icon indicating copy to clipboard operation
python-snappy copied to clipboard

Example Mismatch

Open cneberg opened this issue 5 years ago • 2 comments

Python example to compress a string with snappy can't be decompressed with the command line tool example to decompress a file with the same string in it. The command line tool should be expanded to support this use case, or at the very least a second programmatic example explaining how to create a file which can be read with the tool.

import snappy
import os
fout = open('compressed_file.snappy', 'w')
buf ='hello world'
buf = snappy.compress(buf)
fout.write(buf)
fout.close()
os.system("python -m snappy -d compressed_file.snappy uncompressed_file");

UncompressError: chunk truncated

cneberg avatar Feb 28 '20 19:02 cneberg

Indeed, it is not obvious that files should be in the "framing format" rather than the simpler, implicit snappy.de/compress functions. This is the code that would do it

import snappy
import io
src = io.BytesIO(b'hello world')
with open('compressed_file.snappy', 'wb') as dst:
    snappy.stream_compress(src, dst)

(substitute an open input file for src instead of the in-memory buffer, if desired)

Please propose as a pull request!

martindurant avatar Feb 28 '20 19:02 martindurant

(by the way, the equivalent function call is snappy.StreamCompressor().compress(b'hello world'))

martindurant avatar Feb 28 '20 19:02 martindurant