m3 icon indicating copy to clipboard operation
m3 copied to clipboard

[query] Normalize options for aggregated histograms

Open arnikola opened this issue 4 years ago • 1 comments

What this PR does / why we need it: Due to the way the aggregator and prom style histograms interact, there is an edge case where histogram bucket datapoints arrive with slight drift; this is fine and should not impact data quality, but can lead to counter resets which cause incorrect and unpredictable behavior when querying. This PR gathers all aggregated histograms by series, then normalizes them in such a way that bucket value at each timestamp is monotonically increasing across le tags (when viewed in descending le size; i.e. +Inf should always have a smaller value than 0.1)

TODO:

  • [x] test for histogram gathering
  • [ ] integration test
  • [x] criteria for normalizing counter series

arnikola avatar Jun 11 '21 14:06 arnikola

Codecov Report

Merging #3556 (81da16a) into master (28c1d95) will decrease coverage by 0.4%. Report is 468 commits behind head on master. The diff coverage is n/a.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##           master   #3556     +/-   ##
========================================
- Coverage    56.2%   55.9%   -0.4%     
========================================
  Files         550     550             
  Lines       61886   61506    -380     
========================================
- Hits        34799   34391    -408     
- Misses      23992   24006     +14     
- Partials     3095    3109     +14     
Flag Coverage Δ
aggregator 57.1% <ø> (ø)
cluster ∅ <ø> (∅)
collector 54.3% <ø> (ø)
dbnode 60.3% <ø> (-0.5%) :arrow_down:
m3em 46.4% <ø> (ø)
metrics 19.8% <ø> (ø)
msg 74.4% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 28c1d95...81da16a. Read the comment docs.

codecov[bot] avatar Jun 15 '21 21:06 codecov[bot]