vectordotdev
Aggregate metrics transform(s)
Aggregate Vector's metric types into coarser metrics across a configurable time interval.
Proposed 1.0 aggregations
- count
- diff
- max
- min
- mean
- sum
- stdev
Ref:
- https://github.com/vectordotdev/vector/issues/2609
- https://github.com/vectordotdev/vector/issues/12745
This could be a follow-up, but I think it'd be useful to be able to aggregate histogram statistics as well to reduce the samples down to just p50, p95. p99, etc. This would reduce the amount of data being sent with the trade-off of less granularity downstream.
I would also be interested in this kind of transform.
High level use case: detection of log stoppages based on metrics. We use an enterprise log platform which imposes license limits. Therefore, we filter logs based on use cases instead of sending all of the logs. It is possible that a system sent its logs, but due to the filtering it may seem like it hasn't sent anything leading to false positives.
I would like to use metrics based on the event stream before filtering to overcome this issue. Unfortunately, metric events are also impacting the license volume, so aggregating the metrics per host over fixed time intervals seems like the way to go.
Noting, per https://github.com/timberio/vector/discussions/8628, this should also permit aggregating across metric tags.
