lz4 icon indicating copy to clipboard operation
lz4 copied to clipboard

Add interoperability tests

Open anatol opened this issue 4 years ago • 4 comments

It would be great to add tests to make sure this library is compatible with lz4 C library (that can be considered here as a reference implementation).

Add a test that

  • compresses some text with lz4 command-line tool and then decompresses with this golang library
  • compresses some text with this library and decompress it with the command-line tool

Verify different compression mode, legacy/modern formats, ....

anatol avatar Nov 11 '21 01:11 anatol

All the LZ4 compressed files in the testdata directory have been generated by the C reference implementation. I agree that not all combinations are currently tested though.

pierrec avatar Nov 11 '21 17:11 pierrec

It is interesting. lz4 tool complains about some of the files under testdata/ dir:

➜  testdata git:(v4) lz4 --version
*** LZ4 command line interface 64-bits v1.9.3, by Yann Collet ***
➜  testdata git:(v4) lz4 --test vmlinux_LZ4_19377.lz4
Stream followed by undecodable data at position 11398804 
vmlinux_LZ4_19377.lz : decoded 45037000 bytes                                  
➜  testdata git:(v4) lz4 --test bzImage_lz4_isolated.lz4 
Error 52 : Read error : cannot access compressed block !

anatol avatar Nov 11 '21 19:11 anatol

Here is an example of interoperability test for legacy writer

diff --git a/writer_test.go b/writer_test.go
index e43fd36..f0fee40 100644
--- a/writer_test.go
+++ b/writer_test.go
@@ -7,6 +7,7 @@ import (
        "io"
        "io/ioutil"
        "os"
+       "os/exec"
        "reflect"
        "strings"
        "testing"
@@ -247,6 +248,22 @@ func TestWriterLegacy(t *testing.T) {
                                t.Fatal(err)
                        }
 
+                       tmp, err := os.CreateTemp("", "")
+                       if err != nil {
+                               t.Fatal(err)
+                       }
+                       defer os.Remove(tmp.Name())
+                       if _, err := tmp.Write(out.Bytes()); err != nil {
+                               t.Fatal(err)
+                       }
+
+                       cmd := exec.Command("lz4", "--test", tmp.Name())
+                       cmd.Stdout = os.Stdout
+                       cmd.Stderr = os.Stderr
+                       if err := cmd.Run(); err != nil {
+                               t.Fatal(err)
+                       }
+
                        out2 := new(bytes.Buffer)

anatol avatar Dec 03 '21 03:12 anatol

Weird fact that lz4 dies not return error code on some invalid inputs, e.g. it prints

Stream followed by undecodable data at position 8 
/tmp/4278189638      : decoded 0 bytes 

but the error code is 0.

anatol avatar Dec 03 '21 03:12 anatol