snappy-java icon indicating copy to clipboard operation
snappy-java copied to clipboard

What type of inputstream does SnappyFramedInputStream work with better?

Open darouwan opened this issue 7 years ago • 1 comments

I need to decompress some large files generated by snappyframedoutputstream, which is always over 2GB size. Currently on some small testing files FileInputStream wrapped with SnappyFramedInputStream works fine for me, but I am not sure if it still has the similar performance with over 2GB files since I have no such large file on my computer for testing.

So I am wondering if I need to wrap the FileInputStream with BufferedInputStream or using FileInputStream directly for large file ?

darouwan avatar Jul 04 '17 03:07 darouwan

SnappyFramedInputStream is actually based on a ReadableByteChannel. A FileInputStream is converted to a FileChannel, which should be quite efficient. A BufferedInputStream would be wrapped with a ReadableByteChannel, which would effectively be buffering the data twice.

My suggestion would be to start with either a FileInputStream or a FileChannel. Only revisit if you are experiencing specific issues.

bokken avatar Jul 13 '17 01:07 bokken