prometheus-kafka-adapter icon indicating copy to clipboard operation
prometheus-kafka-adapter copied to clipboard

Include regex match for metrics in MATCH

Open rajasreesekar opened this issue 3 years ago • 2 comments

Hi all

Can you plan to have regex match for metrics in MATCH rules. Please provide a supporting document too that explains the format. For example , if i want to receive metrics starting with container_memory and namespace starting with test.

"['container_memory.*{namespace=\"test.*\"}']"

Without regex, have to add each and every metrics one by one with combinations of difference namespaces/ other fields. The list and complexity involved is huge.

rajasreesekar avatar Mar 24 '22 12:03 rajasreesekar

Hi @rajasreesekar, you are right, MATCH is not even documented in the main https://github.com/Telefonica/prometheus-kafka-adapter/blob/master/README.md. Do you feel like contributing that change? I personally won't have the time to do it in the short-medium term.

Matching rules were introduced in https://github.com/Telefonica/prometheus-kafka-adapter/pull/64 by @Gallardot, in case it's helpful.

Another option would be to filter them in prometheus, before sending them to prometheus-kafka-adapter. This way you won't even need to filter them. It should be something along these lines:

    global:
      scrape_interval: 30s
      
    scrape_configs:
      - job_name: 'federate''
        params:
          'match[]':
            - container_memory.*{namespace=\"test.*\"} # YOUR FILTER

    remote_write:
      - url: http://prometheus-kafka-adapter:8080/receive

palmerabollo avatar Mar 27 '22 15:03 palmerabollo

@palmerabollo Ok, will take a look at the filter at prometheus level, But i suppose that filter is common for all remote_writes right?. Not specific to one . Have you tried the same in Openshift?

rajasreesekar avatar Mar 30 '22 09:03 rajasreesekar

But i suppose that filter is common for all remote_writes right?

If I'm not wrong you can have multiple remote_writes with different configurations in prometheus

Have you tried the same in Openshift?

No

palmerabollo avatar Aug 31 '23 23:08 palmerabollo