Instant query time splitting loses counter increases between time splitting boundaries

[pracucci]

Example

For example, let's consider a counter series with the following samples:

• T: 0 V: 1
• T: 29 V: 2
• T: 59 V: 3
• T: 91 V: 40
• T: 120 V: 41

Running the query without splitting

Running the instant query rate(metric[2m]) at time 120 will return the result 0.333333333333333, because computed as follows:

1. Sample the interval [0, 120] (first and last sample) computing a value increase of value 41 - 1 = 40
2. The sampled interval start timestamp (0) matches with the timestamp of the 1st sample, and the sampled interval end timestamp (120), so the no extrapolation triggers
3. The final rate value is 40 / 120 = 0.333333333333333

Running the query with splitting

Running the same query with a split interval of 1m would result in the following query:

sum without() (
    concat(
        increase(metric[1m] offset 1m)
        increase(metric[1m])
    )
) / 120

Each partial query computes the following values (ignoring extrapolation cause doesn't change much the math for this case):

• increase(metric[1m] offset 1m): 3 - 1 = 2
• increase(metric[1m]): 41 - 40 = 1

The final query result is 2 + 1 / 120 = 0.025 which is very far from the expected value 0.333333333333333

[bboreham]

not many instant queries have a time range > 24h

However the ones we most want to speed up do.