kafka-lag-exporter
kafka-lag-exporter copied to clipboard
Lags not reported for topics with no active member
Describe the bug As per #66, consumer groups with no active members also export the lags for all the topics. However in case where a group consumes from multiple topics and it has active members in only some topic, no metrics are reported for the topics without active member.
when group has no active members
$ kafka-consumer-groups --bootstrap-server $BROKERS --describe --group my-group
Consumer group 'my-group' has no active members.
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
my-group topic-1 0 100 100 0 - - -
my-group topic-2 0 100 100 0 - - -
$ curl -s http://localhost:8000 | grep my-group | grep kafka_consumergroup_group_lag
kafka_consumergroup_group_lag{cluster_name="test", group="my-group", topic="topic-1", partition="0", member_host="unknown", consumer_id="unknown", client_id="unknown"} 0.0
kafka_consumergroup_group_lag{cluster_name="test", group="my-group", topic="topic-2", partition="0", member_host="unknown", consumer_id="unknown", client_id="unknown"} 0.0
when group has some active members
$ kafka-consumer-groups --bootstrap-server $BROKERS --describe --group my-group
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
my-group topic-1 0 100 100 0 my-consumer-id /10.11.0.1 rdkafka
my-group topic-2 0 100 100 0 - -
$ curl -s http://localhost:8000 | grep my-group | grep kafka_consumergroup_group_lag
kafka_consumergroup_group_lag{cluster_name="test", group="my-group", topic="topic-1", partition="0", member_host="/10.11.0.1 ", consumer_id="my-consumer-id", client_id="rdkafka"} 0.0
To Reproduce application.conf
kafka-lag-exporter {
port = 8000
reporters.prometheus.port = 8000
clusters = [
{
name = "test"
bootstrap-brokers = "localhost:29092"
}
]
}
Environment
- Version: 0.6.8
- Version of Apache Kafka cluster: Confluent-kafka-2.12-5.3.2
- Run with Standalone
any idea? i see same problem, metrics are empty
Hi, I just got the same issue. I can't help on code, but let me know if there are some tests to do to help analysis