kafka icon indicating copy to clipboard operation
kafka copied to clipboard

KIP-780: Support fine-grained compression options

Open see-quick opened this issue 5 months ago • 2 comments

This PR implements KIP-780: Per-codec compression configuration. In addition to compression.type and compression.level (KIP-390), Kafka now exposes tunable codec-specific parameters for Producer, Broker, and Topic configs.

Supported options:

  • GZIP: buffer, strategy
  • Snappy: block
  • LZ4: block
  • Zstd: level, window, workers

All defaults match existing behaviour → no migration required. These options allow users to trade off throughput, compression ratio, CPU, and memory depending on the workload (e.g., Snappy/LZ4 for speed, Zstd/Gzip for compression ratio). You can check TestCompression and see results or directly see my observations KIP-780.

Benchmarks demonstrate codec-dependent trade-offs across synthetic and real-world (Debezium) datasets.

see-quick avatar Aug 18 '25 12:08 see-quick

A label of 'needs-attention' was automatically added to this PR in order to raise the attention of the committers. Once this issue has been triaged, the triage label should be removed to prevent this automation from happening again.

github-actions[bot] avatar Aug 26 '25 03:08 github-actions[bot]

This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please leave a comment asking for a review. If the PR has merge conflicts, update it with the latest from the base branch.

If you are having difficulty finding a reviewer, please reach out on the [mailing list](https://kafka.apache.org/contact).

If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed.

github-actions[bot] avatar Dec 10 '25 03:12 github-actions[bot]