lzo.net icon indicating copy to clipboard operation
lzo.net copied to clipboard

First instruction has special handling when high nibble is set

Open MikeGratsas opened this issue 3 years ago • 3 comments

On start of LzoStream decoding first instruction should be analyzed in addition. If instruction > 17 then literal length is calculated as instruction - 17, a match should not be copied, since a ring buffer is empty. I found this issue trying to decompress .lzo file compressed by lzop with LZO1X_999 method.

MikeGratsas avatar Apr 10 '22 12:04 MikeGratsas

Can you share an example file to reproduce the issue?

zivillian avatar Apr 10 '22 13:04 zivillian

I believe I'm running into this issue as well. I have the following compressed data which I'm trying to decompress:

63 FF FF 4A B9 FF FF FF    4B EA FF FF FF 4C C1 FF 
FF FF 4D C5 FF FF FF 4E    CC FF FF FF 4F DF FF FF 
FF 50 DE FF FF FF 52 EB    FF FF FF 53 BA FF FF FF 
19 F1 FF FF FF 2B EC FF    FF FF 2D E8 FF FF FF 2E 
DC FF FF FF 2F D8 FF FF    FF 30 D1 FF FF FF 3A CB 
FF FF FF 11 00 00

Looking at minilzo.c, there's a block at the beginning of decompression here:

    NEED_IP(1);
    if (*ip > 17)
    {
        t = *ip++ - 17;
        if (t < 4)
            goto match_next;
        assert(t > 0); NEED_OP(t); NEED_IP(t+3);
        do *op++ = *ip++; while (--t > 0);
        goto first_literal_run;
    }

In my case the first byte of the block is 99, so 99 - 17 = 82 bytes of input data should be copied to the output. (There's similar blocks in a couple of the other LZO source files also, but I don't really know what each of the various files represents.)

I forked this repo to try and implement a fix myself, but I'm not familiar enough to do it cleanly. I'd be happy to help out if you can give me some pointers though. Attaching example files (same as above data) for reference.

data example.zip

chrishayesmu avatar Jun 25 '24 18:06 chrishayesmu

Opened PR #5 which handles my specific data, though it isn't complete. Hopefully this serves as a decent example.

chrishayesmu avatar Jun 25 '24 23:06 chrishayesmu