Rolling window on aggregator sampling

[alxnik]

Use Case

If I have an input that has an ever increasing cumulative value, I want to use an aggregator (such as basicstats diff) in order to calculate the delta from the previous value retrieved by an input.

Expected behavior

The aggregator utilizes a rolling window of measurements to calculate its results, defined by a variable on the aggregator level. The window size is defined by the period variable. Additionally, the aggregator will output data on every interval of the input, not just once every period.

ex. the input has a 10s interval, the aggregator has a 20s period and always keeps the last 2 values of the input measurement which makes the delta calculation feasible. The aggregator will output every 10s which is the input interval and not every 20s

Actual behavior

The period variable on the aggregators works as a tumbling window. This means that for the diff and non_negative_diff functionality of basicstats, cannot capture all deltas.

Ex. The input has a 10s interval, the aggregator has a 20s period. The aggregator outputs every 20s and returns the delta of the last 10s of the period, which means it discards the first 10s of the period.

Additional info

No response

[srebhan]

Is my understanding correct that for a series

x1, x2, x3, x4

you want the following as output

x2-x1, x3-x2, x4-x3

If so, we would need a new processor plugin to do this as aggregation is really over time (aka batches of metrics).

[alxnik]

Your understanding on my requirement is correct.

I do agree that an aggregation is over a batch of metrics, and I think a rolling window would fit this criteria. Something like "calculate the aggregate over the last 10 metrics". This would make the aggregation output on the "interval", while the "period" would be the window size. This could be applied not just on diff but on avg, mean, max, min etc.

FWIW, I eventually solved this specific case with a processor and some starlark code since it just needs the previous value. A proper rolling window would be more challenging though.