beats icon indicating copy to clipboard operation
beats copied to clipboard

metricbeat/module/prometheus/collector: Move deduplication logic of metrics from after processing to before processing

Open shmsr opened this issue 1 year ago • 0 comments

While investigating a bug report, we noticed the deduplication is done here where it should've been done earlier. A problem was faced when rate calculation was done for counter metrics but half of the metrics were dropped because there were duplicates and the rate was always reported as 0 (if the counter value is the same for duplicates too).

How to deduplication still needs to be discussed. For metrics with timestamps, we can easily deduplicate for metrics with no timestamps, we have to figure out how to duplicate properly in case there are metrics with exact same label k-v in a scrape but with different values.

See the internal issue for more details: https://github.com/elastic/sdh-beats/issues/4249#issuecomment-1907776052

shmsr avatar Jan 30 '24 08:01 shmsr