kafka-go icon indicating copy to clipboard operation
kafka-go copied to clipboard

Consumer group leaveGroup fail when group leader down

Open zydovech opened this issue 1 year ago • 1 comments

If current consumer group leader down , will trigger rebalance, but before a new leader is elected, findCoordinator still returns the old leader,rebalance will fail .

  • Unable to establish connection to consumer group coordinator for group xxx: dial tcp 127.0.0.1:9092

then https://github.com/segmentio/kafka-go/blob/f2d9e087266352d276b3cb951b4c668c114b8b75/consumergroup.go#L753C1-L754C1 will fail , If the memberID is set to empty at this time, the new memberID will be used next time.

The new rebalance needs to wait for the previous member to join, causing the rebalance time to become longer.

I would like to ask, would it be better if this place is not left empty?

zydovech avatar Oct 27 '23 06:10 zydovech

logs:

kafka3 | [2023-10-27 08:00:50,599] INFO [GroupCoordinator 1002]: Dynamic Member with unknown member id joins group xxxxx in Empty state. Created a new member id xxxxx-72fae797-dd90-4d46-b4a0-9651457d43ee for this member and add to the group. (kafka.coordinator.group.GroupCoordinator) kafka3 | [2023-10-27 08:00:50,600] INFO [GroupCoordinator 1002]: Preparing to rebalance group xxxxx in state PreparingRebalance with old generation 76 (__consumer_offsets-0) (reason: Adding new member xxxxx-72fae797-dd90-4d46-b4a0-9651457d43ee with group instance id None) (kafka.coordinator.group.GroupCoordinator)

kafka1 | [2023-10-27 08:01:18,314] INFO [GroupCoordinator 1001]: Dynamic Member with unknown member id joins group xxxxx in Stable state. Created a new member id xxxxx-e74e92b1-9ccb-48a5-8db2-75ecb988b8bf for this member and add to the group. (kafka.coordinator.group.GroupCoordinator) kafka1 | [2023-10-27 08:01:18,325] INFO [GroupCoordinator 1001]: Preparing to rebalance group xxxxx in state PreparingRebalance with old generation 77 (__consumer_offsets-0) (reason: Adding new member xxxxx-e74e92b1-9ccb-48a5-8db2-75ecb988b8bf with group instance id None) (kafka.coordinator.group.GroupCoordinator) kafka1 | [2023-10-27 08:01:45,027] INFO [GroupCoordinator 1001]: Member xxxxx-72fae797-dd90-4d46-b4a0-9651457d43ee in group xxxxx has failed, removing it from the group (kafka.coordinator.group.GroupCoordinator)

zydovech avatar Oct 27 '23 07:10 zydovech