lz4js icon indicating copy to clipboard operation
lz4js copied to clipboard

Output cannot be decompressed by other lz4 tools in some cases

Open JonnoFTW opened this issue 3 years ago • 1 comments

I'm using this to compress a large XML file (120mb) in the browser and trying to decompress server-side using python-lz4 (which can decompress every other file compressed with lz4js except this one). It also couldn't be decompressed using the standard lz4 tool, getting the error:

$ lz4 -d test.lz4 decompressed.xml
Error 66 : Decompression error : ERROR_decompressionFailed

My file is here: test.zip

Compressing/decompressing the same file with the standard tools succeeds.

My code to replicate this issue is here:

#!/usr/bin/env node
#compress.js
const lz4 = require("lz4js");
const fs = require("fs");

const data = fs.readFileSync(process.argv[2]);
var compressed = Buffer.from(lz4.compress(data));
fs.writeFileSync(process.argv[3], compressed);
#!/usr/bin/env python3
#decompress.py
import lz4.frame
from pathlib import Path
import sys

if __name__ == "__main__":
    try:
        p = Path(sys.argv[1])
        decomped = lz4.frame.decompress(p.read_bytes())
        print("Success")
    except RuntimeError as e:
        print("Error")

I've chopped down the file to a minimum working case, which I've attached. Removing the first 10 lines from the start or end will make it succeed, but I'm not sure why. Run with:

npm i lz4js && pip install lz4
./compress.js test.xml test.lz4 && ./decompress.py test.lz4

JonnoFTW avatar Aug 06 '20 07:08 JonnoFTW