kafka.log.partition.size metric

[mthoretton]

Hello,

I was looking in the doc, issues and internet, no way to find how to push kafka.log.partition.size metric. It's not part of the default of the default metrics https://docs.datadoghq.com/integrations/kafka/.

Do I need a custom conf or a custom check? thanks!

[mthoretton]

So, in order to collect it, you need a custom conf (which is actually based on JMX). See https://github.com/DataDog/integrations-core/blob/master/kafka/datadog_checks/kafka/data/conf.yaml.example

This is the conf to add. The order of the parameters in bean_regex matters! I was able to debug thanks to https://docs.datadoghq.com/integrations/faq/troubleshooting-jmx-integrations/?tab=agentv6 .

init_config:
  is_jmx: true
  collect_default_metrics: true
  conf:
    ...
    - include:
        domain: 'kafka.log'
        bean_regex: 'kafka\.log:type=Log,name=Size,topic=(.*?),partition=(.*?)(?:,|$)'
        tags:
          topic: $1
          partition: $2
        attribute:
          Value:
            alias: kafka.log.partition.size
            metric_type: gauge

By adding that we may hit the default 350 JMX metrics limit though.

Feel free to add that to the doc (I could also create a PR) or close this issue.

[seanlutner]

It would be super awesome if this was part of the documentation. I just spent about an hour feeling like I was crazy because I didn't have this metric and there was no indication that I wouldn't by default.

[jamiealquiza]

Sorry about that - I forgot that it's a non-default because our monitoring config was setup so long ago, but I'll take note to document this.

[xkrt]

Is it possible to gather just topics sizes without breakdown by partitions? It will be very helpful to just monitor topics sizes and not to hit 350 metrics limit.

[jamiealquiza]

Is it possible to gather just topics sizes without breakdown by partitions? It will be very helpful to just monitor topics sizes and not to hit 350 metrics limit.

Unfortunately topicmappr will need the per-partition sizes since it's making placement decisions at the partition level.

FWIW, the metrics input that it uses for this is formatted in a standardized way that can be sourced from any input as long as it follows the format. See the 3p implementations of https://github.com/DataDog/kafka-kit/tree/master/cmd/metricsfetcher. For reasons other than per-host metrics limits, I've thought about the idea of having an agent/script that's gathering the partition sizes for me in a way other than summing the kafka.log.partition.size by {topic,partition}.