Offset increase by 2 due to control batchesc

[rkempter]

Using a transactional producer, it is not guaranteed that offsets are always incremented by 1. Due to control batches, it is possible that offsets are incremented by 2.

This can lead to false positives, as the lag can be equal to 1, although the consumer has consumed all messages available.

Has anyone come across this and how did you reduce the number of false positives?

Resources: Control Batch: http://kafka.apache.org/documentation/#controlbatch Issue observed: https://stackoverflow.com/questions/56182606/in-kafka-when-producing-message-with-transactional-consumer-offset-doubled-upc

[Thiago-Dantas]

yep, can confirm Burrow is alerting on unconsumed control batches. In our case we have s3 sink consuming from the topic. S3 Sink happily skips the control batch but until there are new messages on the topic the lag sits at 1 which Burrow doesn't like

[Thiago-Dantas]

From what I understand from here sarama doesn't even pass down the control batch record and since that's the only offset left at the topic it just sits there