kminion
kminion copied to clipboard
Export numerical kafka_topic_info parameters/labels as individual metrics
Currently kminion_kafka_topic_info
metric exposes all topic parameters as labels. [1]
I've got an use-case where I'd need the replication_factor
and min_insync_replicas
values to be used in promql queries to verify if a given topic has under-min replicas, e.g:
kafka_topic_partition_replicas < "min_insync_replicas"
or a given topic has under-replicated partitions:
kafka_topic_partition_replicas < "replication_factor"
To do so, the only solution I found is to have these numerical topic parameters exposed as separate metrics, e.g:
kminion_kafka_topic_info_min_insync_replicas
kminion_kafka_topic_info_replication_factor
Would this be a good improvement for the kminion @weeco ?
[1] Sample:
kminion_kafka_topic_info{cleanup_policy="delete", container="kafka-minion", endpoint="metrics",
max_message_bytes="1048588", message_format_version="2.5-IV0", message_timestamp_type="CreateTime",
min_insync_replicas="2", partition_count="1", replication_factor="1", retention_bytes="-1", retention_ms="10000",
segment_bytes="536870912",
topic_name="00faec4f386fc8dda9688d402cc92d0c1082ee0d426909841e6b4d11750deb65"}
I think Kafka itself has a metric called under min isr. I think introducing a similiar metric would make sense. Literally you could expose via a metric how many under.min.isr.partitions exist on a given topic:
Example:
kminion_kafka_topic_under_min_isr_partitions{topic_name="xy"} 4
WDYT?