kafka-go icon indicating copy to clipboard operation
kafka-go copied to clipboard

way to get RevokedPartitions & AssignedPartitions event when consumer group rebalance

Open NeoyeElf opened this issue 3 years ago • 1 comments

In my case, I have serveral workers to consume kafka message. My program will create serveral mem queues to handle the message sequentially by Message.Key, and handle the commit of offsets manually.

When consumer group reBalancing, I want to be notified to ignore the unHandled messages that don't match the new assigend partitions in the mem queue.

  1. Is there any way to meet my needs?
  2. If rebalance occurs, and I commit the partition offsets that don't match the new assigend partitions, what will happend? will CommitMessages returns an error?

Thanks.

NeoyeElf avatar Apr 13 '21 07:04 NeoyeElf

+1 I am also looking for a way to be notified of the RevokedPartitions & AssignedPartitions event when there is a consumer group rebalance

onkarbanerjee avatar Apr 29 '21 09:04 onkarbanerjee

Any updates on this? It's such an essential feature especially considering there is no support for static membership of consumer group. Are there currently any way to make sure a consumer X gets reassigned the same partitions after a rebalance? Or at least a way to know when a rebalance is about to happen to manage things ourselves?

louisboilard avatar Dec 23 '22 23:12 louisboilard