trafficserver icon indicating copy to clipboard operation
trafficserver copied to clipboard

Add Zstandard compression support to the compress plugin

Open JakeChampion opened this issue 8 months ago • 2 comments

This is a backport of https://github.com/apache/trafficserver/pull/12201

  • Updated Makefile and configuration files to include Zstandard as a compression algorithm.
  • Enhanced compress.cc to handle Zstandard compression and added necessary initialization and cleanup functions.
  • Modified misc.cc to normalize accept-encoding headers for Zstandard.
  • Updated tests to verify Zstandard functionality alongside existing compression algorithms.

JakeChampion avatar Apr 23 '25 08:04 JakeChampion

@JakeChampion : Thank you for working on this. It looks like a few autests have output expectation issues.

The compress autest fails due to zstd beign in the output now. Should be a simple gold file update required (src/tests/gold_tests/pluginTest/compress/compress_userver.gold):

   Run: 31-tr: Failed
     file compress_userver.log : Checking that compress_userver.log matches compress_userver.gold - Failed
        Reason: File differences
         Gold File : /home/jenkins/workspace/Github_Builds/autest/src/tests/gold_tests/pluginTest/compress/compress_userver.gold
         Data File : /tmp/sandbox/compress/compress_userver.log
         - 0/gzip, deflate, sdch, br
         + 0/zstd, gzip, deflate, sdch, br
         ?   ++++++
         
           0/gzip
           0/br
           0/deflate
         + 0/zstd
         - 1/gzip, deflate, sdch, br
         + 1/zstd, gzip, deflate, sdch, br
         ?   ++++++
         
           1/gzip
           1/br
           1/deflate
         + 1/zstd
         - 2/gzip, deflate, sdch, br
         + 2/zstd, gzip, deflate, sdch, br
         ?   ++++++
         
           2/gzip
           2/br
           2/deflate
         + 2/zstd
         + 3/zstd, gzip, deflate, sdch, br
         + 3/gzip
         + 3/br
         + 3/deflate
         + 3/zstd
           0/gzip;q=0.666
           0/gzip;q=0.666x
           0/gzip;q=#0.666
           0/gzip; Q = 0.666
           0/gzip;q=0.0
           0/gzip;q=-0.1
           0/aaa, gzip;q=0.666, bbb
           0/ br ; q=0.666, bbb
           0/aaa, gzip;q=0.666 , 
           3/gzip

Also, the normalized_ae_match_vary_cache autest fails, it looks like also for output expecation update needs:

       file /tmp/sandbox/normalized_ae_match_vary_cache/_output/0-tr-Default/stream.stdout.txt : There should be no Proxy Verifier violation errors. - Failed
          Reason: Contents of /tmp/sandbox/normalized_ae_match_vary_cache/_output/0-tr-Default/stream.stdout.txt contains expression: "Violation|Invalid status"
             Details:
               [INFO]: Equals Violation: Different. Key: "34", Field Name: "x-response-identifier", Correct Value: "Zstd-Accept-Encoding", Actual Value: "Br-Accept-Encoding" : 80
               [INFO]: Equals Violation: Different. Key: "34", Field Name: "x-cache", Correct Value: "miss", Actual Value: "hit-fresh" : 81
               [INFO]: Equals Violation: Different. Key: "38", Field Name: "x-response-identifier", Correct Value: "Zstd-Accept-Encoding", Actual Value: "Br-Accept-Encoding" : 98

bneradt avatar Apr 28 '25 22:04 bneradt

We don't add new features to patch releases and we are not planning on having another minor release of 9.x. We are happy for you to use CI to verity the change and we can have someone look over the code for you. 9.2.x is currently our LTS release and we are doing new development on ATS 10.x.

For example on out release versioning: major.minor.patch

bryancall avatar Apr 28 '25 22:04 bryancall