Failed to unpack Invalid data block: 4 @38538

[aol-nnov]

Hi!

Here is the minimal reproducible example and the file in question Packages.lz4.txt (kindly remove .txt part from name - github forbids .lz4 upload...)

const fs = require('fs');
const lz4 = require('lz4');

const decoder = lz4.createDecoderStream();
const input = fs.createReadStream('./Packages.lz4');
input.pipe(decoder).pipe(process.stdout); // file is plaintext, so it's safe to pipe to stdout directly

I've managed to unpack it on linux with lz4cat (from liblz4-tools) but it fails with an error when I'm trying with node-lz4.

Any glue what's going wrong?

events.js:291
      throw er; // Unhandled 'error' event
      ^

Error: Invalid data block: 4 @38538
    at Decoder.emit_Error (/Users/aol/develop/lotes-local/node-lz4/lib/decoder_stream.js:64:22)
    at Decoder.uncompress_DataBlock (/Users/aol/develop/lotes-local/node-lz4/lib/decoder_stream.js:245:9)
    at Decoder._main (/Users/aol/develop/lotes-local/node-lz4/lib/decoder_stream.js:316:25)
    at Decoder._transform (/Users/aol/develop/lotes-local/node-lz4/lib/decoder_stream.js:60:7)
    at Decoder.Transform._read (_stream_transform.js:191:10)
    at Decoder.Transform._write (_stream_transform.js:179:12)
    at doWrite (_stream_writable.js:403:12)
    at writeOrBuffer (_stream_writable.js:387:5)
    at Decoder.Writable.write (_stream_writable.js:318:11)
    at ReadStream.ondata (_stream_readable.js:718:22)
Emitted 'error' event on Decoder instance at:
    at errorOrDestroy (internal/streams/destroy.js:108:12)
    at Decoder.onerror (_stream_readable.js:754:7)
    at Decoder.emit (events.js:314:20)
    at Decoder.emit_Error (/Users/aol/develop/lotes-local/node-lz4/lib/decoder_stream.js:64:7)
    at Decoder.uncompress_DataBlock (/Users/aol/develop/lotes-local/node-lz4/lib/decoder_stream.js:245:9)
    [... lines matching original stack trace ...]
    at doWrite (_stream_writable.js:403:12)

[email protected] on node v12.21.0 on Mac OS

Thanks in advance, Andrey

[thmang82]

I have the same issue with some files. Especially on bigger files encoded with the Rust lz4 library. lz4 on command line can decode the file. lz4 in javascript cannot. Really strange.