Test native (de)compression [Snappy, Zstd, Lz4]

[wendigo]

trafficstars

Nothing to see here yet

[starburstdata-automation]

Started benchmark workflow for this PR with test type = iceberg/sf1000_parquet_part.

Building Trino finished with status: success Benchmark finished with status: success Status message: NO Regression found. Benchmark Comparison Report

[starburstdata-automation]

Started benchmark workflow for this PR with test type = iceberg/sf1000_parquet_unpart.

Building Trino finished with status: success Benchmark finished with status: success Status message: NO Regression found. Benchmark Comparison Report

[github-actions[bot]]

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

[wendigo]

Updated to released version of aircompressor

[starburstdata-automation]

Started benchmark workflow for this PR with test type = iceberg/sf1000_parquet_part.

Building Trino finished with status: success Benchmark finished with status: success Status message: NO Regression found. Benchmark Comparison Report

[sopel39]

@wendigo does it use native decompressors?

Could you share the numbers for both Intel and ARM?

[wendigo]

@sopel39 it does

[wendigo]

@sopel39 you mean micro (JMH) or macro benchmarks (benchto)?

[wendigo]

Some JMH results were posted in the aircompressor PR:

  compress    airlift_lz4             calgary/book2                  333,498   375.5MB/s ±    29.4MB/s ( 7.82%) (N = 3, α = 99.9%)
  compress    airlift_lz4_native      calgary/book2                  333,498   461.9MB/s ±    47.4MB/s (10.27%) (N = 3, α = 99.9%)
  compress    airlift_snappy          calgary/book2                  334,111   357.4MB/s ±    34.5MB/s ( 9.64%) (N = 3, α = 99.9%)
  compress    airlift_snappy_native   calgary/book2                  334,941   529.0MB/s ±   139.7MB/s (26.41%) (N = 3, α = 99.9%)
  compress    airlift_zstd            calgary/book2                  205,814   149.4MB/s ±    49.8MB/s (33.30%) (N = 3, α = 99.9%)
  compress    airlift_zstd_native     calgary/book2                  203,941   236.8MB/s ±    63.9MB/s (26.98%) (N = 3, α = 99.9%)
  decompress  airlift_lz4             calgary/book2                  333,498  2713.4MB/s ±   616.6MB/s (22.73%) (N = 3, α = 99.9%)
  decompress  airlift_lz4_native      calgary/book2                  333,498  3553.0MB/s ±   959.0MB/s (26.99%) (N = 3, α = 99.9%)
  decompress  airlift_snappy          calgary/book2                  334,111   735.0MB/s ±    26.7MB/s ( 3.64%) (N = 3, α = 99.9%)
  decompress  airlift_snappy_native   calgary/book2                  334,941  2225.0MB/s ±   105.1MB/s ( 4.72%) (N = 3, α = 99.9%)
  decompress  airlift_zstd            calgary/book2                  205,814   817.0MB/s ±    16.6MB/s ( 2.04%) (N = 3, α = 99.9%)
  decompress  airlift_zstd_native     calgary/book2                  203,941  1115.3MB/s ±   169.5MB/s (15.19%) (N = 3, α = 99.9%)

[sopel39]

@wendigo

Are these JMH results for ARM or Intel? We should test both TBH in case there is regression in one arch.

benchmarks (benchto)?

Bechto TPC-DS would be awesome! Arm/Intel

[wendigo]

@sopel39 this is ARM, I'm not sure whether we ran these for Intel but I don't expect regression there as well

[wendigo]

@sopel39 last time I've ran benchto benchmarks on Iceberg/SF1000 it showed ~10% performance improvement on read and ~10% on write.

[starburstdata-automation]

Started benchmark workflow for this PR with test type = iceberg/sf1000_parquet_part.

Building Trino finished with status: success Benchmark finished with status: success Status message: NO Regression found. Benchmark Comparison Report

[wendigo]

All tests are passing. This is ready to be reviewed @dain @electrum

[wendigo]

@dain I've updated this PR to use v3. Please review