zed
zed copied to clipboard
Aggregation rule
The following text was present in a retired "lake design" document (see #3803). It has been established that this was really a pending to-do, so this issue tracks its ultimate implementation and corresponding updates to docs.
An aggregation rule allows the creation of any index keyed by one or more fields
(primary, second, etc.), typically the result of an aggregation.
The aggregation is specified as a Zed query.
For example,
zed index create IndexGroupEx agg "count() by field"
creates a rule that creates an index keyed by the group-by keys whose
values are the partial-result aggregation given by the Zed expression.
> This is not yet implemented. The query planner would replace any full object
> scan with the needed aggregation with the result given in the index.
> Where a filter is applied to match one row of the index, that result could
> likewise be extracted instead of scanning the entire object.
> This capability is not generally useful for interactive search and analytics
> (except for optimizations that suit the interactive app) but rather is a powerful
> capability for application-specific workflows that know the pre-computed
> aggregations that they will use ahead of time, e.g., beacon analysis
> of network security logs.