databend
databend copied to clipboard
Tracking: enhance hash join spilling
Summary
- [x] https://github.com/datafuselabs/databend/issues/14771
- [x] make partition number configurable: part of https://github.com/datafuselabs/databend/pull/14794
- [x] support left-related join spilling: #14828
- [x] make hash join spilling stable and add enough tests: part of https://github.com/datafuselabs/databend/pull/14794
- [x] support right-related join spilling: https://github.com/datafuselabs/databend/pull/14853
- [x] open hash join spilling by default: https://github.com/datafuselabs/databend/pull/14843
- [x] support cross join spilling: https://github.com/datafuselabs/databend/pull/14887
- [x] https://github.com/datafuselabs/databend/issues/14963
- [ ] support recursive spilling when the restored partition still can't fit into memory.
Bug fixes:
- https://github.com/datafuselabs/databend/pull/14733
- https://github.com/datafuselabs/databend/pull/14794
- https://github.com/datafuselabs/databend/pull/14860
- https://github.com/datafuselabs/databend/pull/15089
Refactor:
- https://github.com/datafuselabs/databend/pull/14743
- https://github.com/datafuselabs/databend/pull/14768
- https://github.com/datafuselabs/databend/pull/14919
- https://github.com/datafuselabs/databend/pull/14942
Others:
- A repo to test databend's join spilling: https://github.com/xudong963/NOOOM
Others:
- A repo to test databend's join spilling: https://github.com/xudong963/NOOOM
Great, I didn't know have this one. Can you port this into wizard? https://github.com/datafuselabs/wizard
Others:
- A repo to test databend's join spilling: https://github.com/xudong963/NOOOM
Great, I didn't know have this one. Can you port this into wizard? https://github.com/datafuselabs/wizard
Will do it