node-lz4 icon indicating copy to clipboard operation
node-lz4 copied to clipboard

Invalid checksum failures

Open buu700 opened this issue 6 years ago • 1 comments

I'm not entirely sure why, but when testing this library I was almost consistently getting checksum failures during decompression, which I know for a fact were false positives because the data being compressed/decompressed was also being cryptographically signed/validated to ensure integrity.

Everything is working as expected with streamChecksum set to false (which is a good enough workaround for my use case, given the aforementioned signing), so it seems like this must be an issue with the checksumming specifically, and not just a general encoding/decoding failure.

I didn't test all that exhaustively, and can't say what about my test case was special enough to cause this, but a few guesses as to the root cause:

  • A bug in handling byte offsets of typed arrays.

  • Something to do with compressing from Node.js and decompressing in a browser.

  • Issues with certain data inputs, e.g. Protocol Buffers or ~50kb png and jpeg images.

(As far as the actual error message: I thought I'd saved it somewhere, but apparently not. That said, I recall something about magic numbers, if that helps.)

buu700 avatar Jul 18 '17 02:07 buu700

Thanks for the report. I think I know what that is but I would like to confirm. If you have a sample file that exhibits the failure, please provide it.

pierrec avatar Aug 27 '17 08:08 pierrec