chronon
chronon copied to clipboard
[Algo] Two Stack + Hop and Sawtooth + two stack
Summary
Implementing variants of the two stack algorithms:
- two stack with hopping
- using two stack to replace the hopRangeCache in sawtooth Test result can be found here.
This PR is meant to be a POC, not intended to be merged.
A new interesting factor was observed: likely due to JVM's JIT, all algorithms ran faster in the second and third runs.
Result: https://docs.google.com/spreadsheets/d/1Mdh_0sv4H-6Ep6URjnFFI3sJ2oeXK88J4wBY32TwOUw/edit#gid=562347570
Perf tldr: with simple aggregation like average, two stacks + hop perform quite well. Replacing hop range cache with two stacks did not significantly improve perf.
With expensive aggregation like top k, two stack + hops still performs relatively well in smaller query set, but suffers from large query set.
Test Plan
- [x] Added Unit Tests
- [ ] Covered by existing CI
- [ ] Integration tested
Checklist
- [ ] Documentation update
Reviewers
@nikhilsimha