database-stream-processor
database-stream-processor copied to clipboard
Streaming and Incremental Computation Framework
... once its architecture is relatively stabilized.
TopK (and sorting, being closely related) is a super important operator for streaming systems and databases as a whole, as is being able to sort data and so we should...
Things like `.recursive()` don't have support for `OrdIndexedZSet` which makes working with maps a lot harder
We need to add a `.count()` operator that counts the number of values for any given key, e.g. `(K, V).count() -> (K, isize)`
See for instance the `q4` query in the nexmark benchmark.
Currently pretty much every operator expects a sorted & consolidated collection and produces a sorted & consolidated collection, but this work is very often redundant. Ephemeral operators like `.filter()` or...
A lot of our sorting and more general consolidation can be done faster via SIMD. This'd be a huge win if we got it working as sorting currently takes a...