Topics: Message count in overview incorrect when using cleanup policy COMPACT

[joschi]

Describe the bug (Actual behavior)

When using topics with clean-up policy COMPACT (log.cleanup.policy=compact) the Message Count metric on the topics overview page is incorrect.

https://github.com/provectus/kafka-ui/blob/v0.6.0/kafka-ui-react-app/src/components/Topics/Topic/Overview/Overview.tsx#L19-L25

The code to calculate the message count (SUM(partition.offsetMax - partition.offsetMin)) only works on non-compacted topics (with cleanup policy DELETE).

Expected behavior

The Message Count metric on the topics overview page should either be correct or not shown at all for topics with cleanup policy

Set up

Kafka UI 0.6.0 using the official Docker image deployed as a container in Kubernetes (EKS).

Steps to Reproduce

1. Create topic with log.cleanup.policy=compact.
2. Produce messages into the topic, some of which have the same partition key.
3. Trigger or wait for log compaction to be performed on the Kafka broker(s) for the topic.
4. Compare the actual number of messages (for example with the topics statistics view in Kafka UI) with the Message Count shown on the topics overview page.

Screenshots

[joschi]

https://github.com/redpanda-data/console/issues/83 describes a way for an approximate number of messages per partition by using the average message size (calculated by reading just some messages from the topic) and the partition sizes.

[nmichaud2]

I think having it in the statistics is fine. I'm not sure it would be a good idea to approximate unless it is clearly indicated it is an approximation !