Aggregation buckets are shifted to the next bucket start

[volodymyrpavlenko]

Hi!

When aggregating, heroic is setting bucket time as a next bucket start time.

This can be reproduced with the following sequence:

1. start heroic docker image: docker run --rm -p 8080:8080 -p 9091:9091 spotify/heroic
2. index the following data points:

curl -XPOST -H "Content-Type: application/json" http://localhost:8080/write --data-binary @- << EOF
{
  "series": {
    "key": "foo",
    "tags": {
      "foo": "bar"
    }
  },
  "data": {
    "type": "points",
    "data": [
      [1591005600001, 1.1],
      [1591092000001, 2.1],
      [1591092000002, 2.2]
    ]
  }
}
EOF

This will index the following datapoints:

2020-06-01 10:00:00.001: 1.1

2020-06-02 10:00:00.001: 2.1
2020-06-02 10:00:00.002: 2.2

3. Query heroic:

curl -XPOST -H "Content-Type: application/json" http://localhost:8080/query/metrics --data-binary @- << EOF
{
  "range": {"type": "absolute", "start": 1590969600000, "end": 1591142400000},
  "filter": ["and", ["key", "foo"]],
  "aggregation": {
            "type": "group",
            "of": null,
            "each": [
                {
                    "type": "count",
                    "sampling": {
                        "unit": "seconds",
                        "value": 86400
                    }
                }
            ]
        }
  }
}
EOF

Where

start = 1590969600000, 2020-06-01 00:00:00.000
end = 1591142400000, 2020-06-03 00:00:00.000

Observed result:

1591056000000 = 1.0  (2020-06-02 00:00:00.000)
1591142400000 = 2.0 (2020-06-03 00:00:00.000)

Expected result:

1590969600000 = 1.0  (2020-06-01 00:00:00.000, count of point during 2020-06-01) 
1591056000000 = 2.0 (2020-06-02 00:00:00.000, count of point during 2020-06-02)

[volodymyrpavlenko]

Seems that disabling com.spotify.heroic.end_bucket_stategy feature fixes this.

Can be disabled in query time with including -com.spotify.heroic.end_bucket_stategy as an additional feature.

Documentation says:

Enabled by default.

Use the legacy bucket strategy by default where the resulting value is at the end of the timestamp of the bucket.