kafka-go
kafka-go copied to clipboard
Consumer group leaveGroup fail when group leader down
If current consumer group leader down , will trigger rebalance, but before a new leader is elected, findCoordinator still returns the old leader,rebalance will fail .
- Unable to establish connection to consumer group coordinator for group xxx: dial tcp 127.0.0.1:9092
then https://github.com/segmentio/kafka-go/blob/f2d9e087266352d276b3cb951b4c668c114b8b75/consumergroup.go#L753C1-L754C1 will fail , If the memberID is set to empty at this time, the new memberID will be used next time.
The new rebalance needs to wait for the previous member to join, causing the rebalance time to become longer.
I would like to ask, would it be better if this place is not left empty?
logs:
kafka3 | [2023-10-27 08:00:50,599] INFO [GroupCoordinator 1002]: Dynamic Member with unknown member id joins group xxxxx in Empty state. Created a new member id xxxxx-72fae797-dd90-4d46-b4a0-9651457d43ee for this member and add to the group. (kafka.coordinator.group.GroupCoordinator) kafka3 | [2023-10-27 08:00:50,600] INFO [GroupCoordinator 1002]: Preparing to rebalance group xxxxx in state PreparingRebalance with old generation 76 (__consumer_offsets-0) (reason: Adding new member xxxxx-72fae797-dd90-4d46-b4a0-9651457d43ee with group instance id None) (kafka.coordinator.group.GroupCoordinator)
kafka1 | [2023-10-27 08:01:18,314] INFO [GroupCoordinator 1001]: Dynamic Member with unknown member id joins group xxxxx in Stable state. Created a new member id xxxxx-e74e92b1-9ccb-48a5-8db2-75ecb988b8bf for this member and add to the group. (kafka.coordinator.group.GroupCoordinator) kafka1 | [2023-10-27 08:01:18,325] INFO [GroupCoordinator 1001]: Preparing to rebalance group xxxxx in state PreparingRebalance with old generation 77 (__consumer_offsets-0) (reason: Adding new member xxxxx-e74e92b1-9ccb-48a5-8db2-75ecb988b8bf with group instance id None) (kafka.coordinator.group.GroupCoordinator) kafka1 | [2023-10-27 08:01:45,027] INFO [GroupCoordinator 1001]: Member xxxxx-72fae797-dd90-4d46-b4a0-9651457d43ee in group xxxxx has failed, removing it from the group (kafka.coordinator.group.GroupCoordinator)