Liveness bug in compacted, static topics

[nacrooks]

Capability downgrading for compacted topics works as follows:

assume the current "latest" timestamp update is (timestamp, partition_count, pid, offset).

We are not guaranteed to ever receive a message with that offset (as it could have been compacted away). Instead, we assume that whenever we see an offset "next_offset" that is >= to offset, we downgrade the capability.

Unfortunately, this assumes that we will "see" a greater offset, which is not necessarily the case for static topics.

This bug manifested itself in a topic with a retention time of about a week. The topic was empty (but fetch_metadata showed the max offset pre compaction). Instead of showing an empty table (like we do for empty topics), we got stuck on "Incomplete Timestamp".

[ruchirK]

I'm pretty sure this is no longer an issue with real time topics because they no longer depend on metadata requests for timestamping?

[aljoscha]

This is about long-removed code.