blazingmq
blazingmq copied to clipboard
Consider adding Zstandard compression library as an option for message compression
https://github.com/facebook/zstd
TL;DR On our benchmarks:
- zstd compression performance is up to 55 times faster compared to zlib. On the blocks staring form 1024 bytes zstd starts to outperform zlib while on smaller buffer they are on par.
- Compression ratio seems to be on par for zlib vs zstd (~34% decrease in size for buffers greater than 128 bytes)
What's done: I implemented compression via zstd in the similar way to how it's done for zlib. Added a few unit tests to test correctness. Also added performance and compression ratio benchmarks for zstd.
Here are some performance and compression ratio results comparing zlib vs zstd: https://github.com/bbpetukhov/blazingmq/pull/5
@bbpetukhov very nice results!