blazingmq icon indicating copy to clipboard operation
blazingmq copied to clipboard

Consider adding Zstandard compression library as an option for message compression

Open 678098 opened this issue 8 months ago • 2 comments

https://github.com/facebook/zstd

678098 avatar Jun 26 '25 14:06 678098

TL;DR On our benchmarks:

  • zstd compression performance is up to 55 times faster compared to zlib. On the blocks staring form 1024 bytes zstd starts to outperform zlib while on smaller buffer they are on par.
  • Compression ratio seems to be on par for zlib vs zstd (~34% decrease in size for buffers greater than 128 bytes)

What's done: I implemented compression via zstd in the similar way to how it's done for zlib. Added a few unit tests to test correctness. Also added performance and compression ratio benchmarks for zstd.

Here are some performance and compression ratio results comparing zlib vs zstd: https://github.com/bbpetukhov/blazingmq/pull/5

bbpetukhov avatar Sep 29 '25 14:09 bbpetukhov

@bbpetukhov very nice results!

678098 avatar Sep 29 '25 18:09 678098