franz-go icon indicating copy to clipboard operation
franz-go copied to clipboard

ConsumeResetOffset AtEnd causes `LEADER_NOT_AVAILABLE` error

Open MarcinGinszt opened this issue 5 months ago • 3 comments

In my integration tests, I'm pushing some messages to Kafka topic kafka-test-topic and then I'm consuming them them. I'm using franz-go version v1.15.4.

If I am using one of the the offsets kgo.ConsumeResetOffset(kgo.NewOffset().AfterMilli(time.Now().Unix())), or kgo.ConsumeResetOffset(kgo.NewOffset().AtStart()), or not specifying offset at all, then there is not error and I'm successfully retrieving the event.

If the client- consumer has set option kgo.ConsumeResetOffset(kgo.NewOffset().AtEnd()), I can't retrieve the messaeg and I'm getting the error-

INF assigning partitions why="newly fetched offsets for group my-group" how="assigning everything new, keeping current assignment" input="kafka-test-topic[0{-1 e-1 ce0}]"
INF metadata update triggered why="loading offsets in new session from assign"
INF metadata update triggered why="fetch had inner topic errors from broker 1001: UNKNOWN_TOPIC_OR_PARTITION{kafka-test-topic[0]}"
INF metadata update triggered why="re-updating due to inner errors: LEADER_NOT_AVAILABLE{kafka-test-topic}"

Is that expected behaviour?

MarcinGinszt avatar Jan 10 '24 14:01 MarcinGinszt