Andrey Novitskiy
Andrey Novitskiy
Python buffering implementation is p0, Rust will add a significant perf increase and can be used in both Python and Rust based Operators
Tumbling == Sliding where slide equals to window size. Tumbling Windowing allows triggering windowed aggregation at fixed time intervals (e.g. OHLCV). This will also unlock tiling
Currently we automatically pass request time timestamp (`ts` parameter in `@on_demand` function) and let user use it downstream. This is not ideal as it does not allow users to do...
Volga uses same execution mode (STREAMING) for bounded and unbounded processing. We need to add BATCH mode to optimize bounded to ensure fault tolerance and proper resources utilisation. Flink example:...
We need to have orderedness guarantees for order-sensitive operator (e.g. Join, WindowedAgg). Currently WindowOperator performs sort on each window trigger, Join does not sort at all. We need to pass...
We currently have no proper backpressure mechanism, we simply check if ZeroMQ buffers are full on send (catching an exception) and if not, wait until the message is pushed through....
Currently we only allow using one key->value pair as a look up key for datasets/operators. We want to support composite keys
We want to be able to schedule workers on nodes in a smart way. e.g. pipeline first strategy, resource collocation strategy, min-network strategy
User's feedback: "This would be the first piece of feedback then. Reliable counters are important in some use cases (e.g. in our use case for recommendation system). What others (and...
Master task tracking all of the tasks for On-Demand Compute Layer