7-Zip-zstd icon indicating copy to clipboard operation
7-Zip-zstd copied to clipboard
verified publisher icon mcmilk

Lizard v2.1

Open sXNA opened this issue 10 months ago • 3 comments

https://github.com/inikep/lizard/releases/tag/v2.1

sXNA avatar Feb 25 '25 18:02 sXNA

From the github:

2025 Update Back in 2017, Lizard 1.0 filled a niche between LZ4 and zstd in terms of compression ratio, as well as compression and decompression speed. However, based on benchmarks conducted in 2025 (shared below), both LZ4 and zstd have made substantial advancements, particularly in compression and decompression speed, leaving Lizard outdated and effectively obsolete.

malvarenga123 avatar Mar 14 '25 19:03 malvarenga123

@malvarenga123 from issue

This test result on Intel is a bit different. In relation to zstd, for example, lizard is much faster.

Z:\> lzbench-1.9-u2_w64.exe -t10,10 -o1c4 -elz4/lz4hc/lizard/zstd_fast/zstd/zlib/brotli silesia.tar

lzbench 1.9 (64-bit Windows)  13th Gen Intel(R) Core(TM) i5-13600K @ 5.10GHz

| Compressor name         | Compression| Decompress.| Compr. size | Ratio | Filename   |
| ---------------         | -----------| -----------| ----------- | ----- | --------   |
| memcpy                  | 17550 MB/s | 17150 MB/s |   211947520 |100.00 | silesia.tar|

| lz4 1.10.0              |   942 MB/s |  5983 MB/s |   100880800 | 47.60 | silesia.tar|
| lz4hc 1.10.0 -1         |   437 MB/s |  5341 MB/s |    89135429 | 42.06 | silesia.tar|
| lz4hc 1.10.0 -6         |  92.6 MB/s |  5832 MB/s |    78385708 | 36.98 | silesia.tar|
| lz4hc 1.10.0 -12        |  18.0 MB/s |  6022 MB/s |    77262620 | 36.45 | silesia.tar|

| lizard 1.1 -10          |   787 MB/s |  4557 MB/s |   103402971 | 48.79 | silesia.tar|
| lizard 1.1 -16          |   105 MB/s |  4221 MB/s |    79372512 | 37.45 | silesia.tar|
| lizard 1.1 -19          |  8.42 MB/s |  4356 MB/s |    77416400 | 36.53 | silesia.tar|

| lizard 1.1 -20          |   638 MB/s |  3478 MB/s |    96924204 | 45.73 | silesia.tar|
| lizard 1.1 -26          |  10.2 MB/s |  3694 MB/s |    72459161 | 34.19 | silesia.tar|
| lizard 1.1 -29          |  3.54 MB/s |  3643 MB/s |    68694227 | 32.41 | silesia.tar|

| lizard 1.1 -30          |   608 MB/s |  2056 MB/s |    85727429 | 40.45 | silesia.tar|
| lizard 1.1 -36          |   138 MB/s |  2928 MB/s |    73850400 | 34.84 | silesia.tar|
| lizard 1.1 -39          |  8.17 MB/s |  3031 MB/s |    69807522 | 32.94 | silesia.tar|

| lizard 1.1 -40          |   487 MB/s |  2102 MB/s |    80843049 | 38.14 | silesia.tar|
| lizard 1.1 -46          |  19.9 MB/s |  2428 MB/s |    65413061 | 30.86 | silesia.tar|
| lizard 1.1 -49          |  3.36 MB/s |  2590 MB/s |    60679215 | 28.63 | silesia.tar|

| zstd_fast 1.5.6 --5     |   958 MB/s |  3167 MB/s |   103093752 | 48.64 | silesia.tar|
| zstd_fast 1.5.6 --1     |   770 MB/s |  2812 MB/s |    86984009 | 41.04 | silesia.tar|
| zstd 1.5.6 -1           |   703 MB/s |  2000 MB/s |    73421914 | 34.64 | silesia.tar|
| zstd 1.5.6 -12          |  52.9 MB/s |  2193 MB/s |    58206982 | 27.46 | silesia.tar|
| zstd 1.5.6 -18          |  5.44 MB/s |  1947 MB/s |    53329873 | 25.16 | silesia.tar|
| zstd 1.5.6 -22          |  3.21 MB/s |  1790 MB/s |    52333880 | 24.69 | silesia.tar|

| zlib 1.3.1 -1           |   123 MB/s |   461 MB/s |    77259029 | 36.45 | silesia.tar|
| zlib 1.3.1 -6           |  43.7 MB/s |   497 MB/s |    68228431 | 32.19 | silesia.tar|
| zlib 1.3.1 -9           |  17.8 MB/s |   501 MB/s |    67644548 | 31.92 | silesia.tar|

| brotli 1.1.0 -0         |   601 MB/s |   574 MB/s |    78433298 | 37.01 | silesia.tar|
| brotli 1.1.0 -4         |   130 MB/s |   742 MB/s |    64122989 | 30.25 | silesia.tar|
| brotli 1.1.0 -9         |  16.0 MB/s |   811 MB/s |    56703946 | 26.75 | silesia.tar|
| brotli 1.1.0 -11        |  0.93 MB/s |   670 MB/s |    50407795 | 23.78 | silesia.tar|

sXNA avatar Mar 16 '25 11:03 sXNA

In relation to zstd, for example, lizard is much faster.

I assume the statement meant that the niche between LZ4 and zstd, it filled previously, is not really there anymore.

And the given bench results also confirm it, particularly:

  • if one needs rather decompression speed, LZ4 is a lot faster (by almost the same ratio and compression speed):
  | Compressor name         | Compression| Decompress.| Compr. size | Ratio | Filename   |
  | ---------------         | -----------| -----------| ----------- | ----- | --------   |
- | lizard 1.1 -10          |   787 MB/s |  4557 MB/s |   103402971 | 48.79 | silesia.tar|
+ | lz4 1.10.0              |   942 MB/s |  5983 MB/s |   100880800 | 47.60 | silesia.tar|
- | lizard 1.1 -16          |   105 MB/s |  4221 MB/s |    79372512 | 37.45 | silesia.tar|
+ | lz4hc 1.10.0 -6         |  92.6 MB/s |  5832 MB/s |    78385708 | 36.98 | silesia.tar|
- | lizard 1.1 -19          |  8.42 MB/s |  4356 MB/s |    77416400 | 36.53 | silesia.tar|
+ | lz4hc 1.10.0 -12        |  18.0 MB/s |  6022 MB/s |    77262620 | 36.45 | silesia.tar|
  • if one needs rather compression ratio and/or compression speed, zstd is a lot better (by comparable decompression speed):
  | Compressor name         | Compression| Decompress.| Compr. size | Ratio | Filename   |
  | ---------------         | -----------| -----------| ----------- | ----- | --------   |
- | lizard 1.1 -10          |   787 MB/s |  4557 MB/s |   103402971 | 48.79 | silesia.tar|
+ | zstd_fast 1.5.6 --1     |   770 MB/s |  2812 MB/s |    86984009 | 41.04 | silesia.tar|
- | lizard 1.1 -16          |   105 MB/s |  4221 MB/s |    79372512 | 37.45 | silesia.tar|
- | lizard 1.1 -19          |  8.42 MB/s |  4356 MB/s |    77416400 | 36.53 | silesia.tar|
+ | zstd 1.5.6 -1           |   703 MB/s |  2000 MB/s |    73421914 | 34.64 | silesia.tar|

by the way, the table shows "negated" ratio (as compressed size in percent to original, so smaller is better) what a bit strange for comparison, normally one uses vice versa value for ratio (either relation between original and compressed or difference in percent to original, so larger would mean better compression), what illustrates it more evident. In this case the picture looks like:

  | Compressor name         | Compression| Decompress.| Compr. size | Ratio | Filename   |
  | ---------------         | -----------| -----------| ----------- | ----- | --------   |
- | lizard 1.1 -10          |   787 MB/s |  4557 MB/s |   103402971 | 2.050 | silesia.tar|
+ | lz4 1.10.0              |   942 MB/s |  5983 MB/s |   100880800 | 2.101 | silesia.tar|
- | lizard 1.1 -16          |   105 MB/s |  4221 MB/s |    79372512 | 2.670 | silesia.tar|
+ | lz4hc 1.10.0 -6         |  92.6 MB/s |  5832 MB/s |    78385708 | 2.704 | silesia.tar|
- | lizard 1.1 -19          |  8.42 MB/s |  4356 MB/s |    77416400 | 2.738 | silesia.tar|
+ | lz4hc 1.10.0 -12        |  18.0 MB/s |  6022 MB/s |    77262620 | 2.743 | silesia.tar|
  | ---------------         | -----------| -----------| ----------- | ----- | --------   |
  | Compressor name         | Compression| Decompress.| Compr. size | Ratio | Filename   |
  | ---------------         | -----------| -----------| ----------- | ----- | --------   |
- | lizard 1.1 -10          |   787 MB/s |  4557 MB/s |   103402971 | 2.050 | silesia.tar|
+ | zstd_fast 1.5.6 --1     |   770 MB/s |  2812 MB/s |    86984009 | 2.437 | silesia.tar|
- | lizard 1.1 -16          |   105 MB/s |  4221 MB/s |    79372512 | 2.670 | silesia.tar|
- | lizard 1.1 -19          |  8.42 MB/s |  4356 MB/s |    77416400 | 2.738 | silesia.tar|
+ | zstd 1.5.6 -1           |   703 MB/s |  2000 MB/s |    73421914 | 2.887 | silesia.tar|

Although the update is necessary anyway (for the sake of support)... just for the record.

sebres avatar Mar 16 '25 13:03 sebres

Update 2.1 with important bug fix to heap buffer flow and other improvements to actual compiler versions. Why not in 25.0x release? https://github.com/inikep/lizard/releases

stefano2734 avatar Aug 11 '25 11:08 stefano2734

Why not in 25.0x release?

Lack of time? Anyway, the last release is mostly an update to mainline version 25.01 + few stable enhancements. Nothing else. The strategy "close all open issues before new release" would cause, that other people (even not using lizard etc) could wait for the next release much longer.

Regarding mentioned heap-BO, it may be indeed an issue (despite ASLR etc), and since it is hardly possible to estimate exactly the potential to write useable exploit (probably much more complex than an update or a cherry-pick the fix), I'll try to update it soon.

sebres avatar Aug 11 '25 16:08 sebres