KIP-780: Support fine-grained compression options
This PR implements KIP-780: Per-codec compression configuration. In addition to compression.type and compression.level (KIP-390), Kafka now exposes tunable codec-specific parameters for Producer, Broker, and Topic configs.
Supported options:
- GZIP: buffer, strategy
- Snappy: block
- LZ4: block
- Zstd: level, window, workers
All defaults match existing behaviour → no migration required. These options allow users to trade off throughput, compression ratio, CPU, and memory depending on the workload (e.g., Snappy/LZ4 for speed, Zstd/Gzip for compression ratio). You can check TestCompression and see results or directly see my observations KIP-780.
Benchmarks demonstrate codec-dependent trade-offs across synthetic and real-world (Debezium) datasets.
A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.
This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please leave a comment asking for a review. If the PR has merge conflicts, update it with the latest from the base branch.
If you are having difficulty finding a reviewer, please reach out on the [mailing list](https://kafka.apache.org/contact).
If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed.