cube
cube copied to clipboard
Epic: Streaming pre-aggregations
Currently, pre-aggregations are limited to batching data, and it is almost impossible to design efficient pre-aggregations for real-time data.
In the first phase of streaming pre-aggregations we will support kSQL as a data source, and pre-aggregation will subscribe to changes from kSQL. The major difference from traditional pre-aggregations is that streaming pre-aggregations are push-based vs. pull-based. Unlike traditional pre-aggregations, streaming ones do not periodically pull the source database for updates but instead subscribe to the data source to receive updates.