decaton icon indicating copy to clipboard operation
decaton copied to clipboard

Provide round-robin mode in SubPartitioner for non-null key

Open ocadaruma opened this issue 2 years ago • 0 comments

  • Currently, Decaton's SubPartitioner uses murmur2 hash of the key to determine the subpartition to ensure per-key processing ordering guarantee.
  • However, sometimes we want to use round-robin strategy even for non-null key. e.g.:
    • Topic is consumed by multiple consumers, and some consumers need ordering guarantee (e.g. windowed-processing) but some consumers don't (just collecting stats).
    • In latter type of consumers, using hash-partitioning doesn't make sense. (just spoils the processing throughput when key-bursting happens)
  • It's nice to have a feature to enable round-robin mode even for non-null keyed topics.

ocadaruma avatar Dec 14 '21 07:12 ocadaruma