Test what happens if a part of the last message is deleted

[itadventurer]

This may happen when Backup Files are copied during a running Backup.

With bad time following things may happen:

Partially Corrupt Data file

• OS copies a data file. The last message is not written fully
• Next, the OS copies the corresponding index file.

Expected behavior: The restore process should at least warn the user that there is a truncated message and they should delete the index and recreate it.

Index out of sync with the data file

• OS copies a data file.
• New messages are written to the corresponding partition
• OS copies the index file

The index has now more messages than exist. Kafka Backup should at least tell the user what is going on.

Ideas to resolve the issues

• Silently ignore the errors and ignore the index files
• Check consistency of index and data files before restore
• Drop index files altogether as they are just a performance optimization and rely only on the data files.