error: Offsets must be always increasing! There is something terribly wrong in your index!

[andresgomezfrr]

I have a deployment with this error:

Offsets must be always increasing! There is something terribly wrong in your index!

What is the exact meaning of this error?

[itadventurer]

Hi, This means, that somehow Kafka-backup gets an offset from Kafka that is lower that the latest entry in the current backup. Imagine following procedure:

You backup messages with the offset 1,2,3. Then something (this is the big question. What happened at your side?) happens – and Kafka Backup wants to write the offset 2 again. This is obviously bad as we would overwrite existing data.

[andresgomezfrr]

Maybe the kafka-backup write the offset inside the backup before persist the consumer offset? Something goes wrong the kafka-backup restart, recover for the latest persist offset and try to save this message again in the backup... it is possible?