franz-go
franz-go copied to clipboard
Support force using ListOffsetsRequest to fetch offset
Why have this proposal
Some cloud service providers (such as alibabacloud) haven't implemented OffsetsForLeaderEpoch.
What happened
Franz-go uses OffsetsForLeaderEpoch
to fetch leader epoch/End Offset and compare consumer offset between leader. After restart, will trigger reset offset behaviour.
Log
[root@xxxx dig]# ./kcl_linux_amd64 misc list-offsets TARGET-TOPIC
BROKER TOPIC PARTITION START END ERROR
101 TARGET-TOPIC 0 170 270
102 TARGET-TOPIC 1 100 100
103 TARGET-TOPIC 2 100 100
101 TARGET-TOPIC 3 100 100
102 TARGET-TOPIC 4 36 100
103 TARGET-TOPIC 5 0 85
[root@xxxx dig]# ./kcl_linux_amd64 misc offset-for-leader-epoch TARGET-TOPIC
BROKER TOPIC PARTITION LEADER EPOCH END OFFSET ERROR
101 TARGET-TOPIC 0 0 0
101 TARGET-TOPIC 3 0 0
102 TARGET-TOPIC 1 0 0
102 TARGET-TOPIC 4 0 0
103 TARGET-TOPIC 2 0 0
103 TARGET-TOPIC 5 0 0
[root@xxxx dig]# ./kcl_linux_amd64 group describe -v xxxx
GROUP xxxx
COORDINATOR 103
STATE Stable
BALANCER cooperative-sticky
MEMBERS 3
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG MEMBER-ID CLIENT-ID HOST
TARGET-TOPIC 0 270 270 0 kgo-xxxx kgo /xxxx
TARGET-TOPIC 1 100 100 0 kgo-xxxx kgo /xxxxx
TARGET-TOPIC 2 100 100 0 kgo-xxxx kgo /xxxx
TARGET-TOPIC 3 100 100 0 kgo-xxxx kgo /xxxx
TARGET-TOPIC 4 100 100 0 kgo-xxxx kgo /xxxx
TARGET-TOPIC 5 85 85 0 kgo-xxxx kgo /xxxxx