horaedb
horaedb copied to clipboard
Enhance to hybrid storage format
Describe This Problem
Hybrid storage format with basic feature is already merged, but there are still things to be done to gain good performance.
Open an new issue to tracking those.
Proposal
Write
- [x] Support variable-length type for
ListArray - [ ] Support table without tsid, only a
row idis required - [ ] Persist hybrid collapsible info
Read
- [ ] Support timestamp column filter, some extra columns may be needed
- [x] Support variable-length type for
ListArray - [ ] Enable a total ordering, to support query with pagination
Misc
- [ ] Compaction support
- [ ] Ensure row group size is large enough, in case of list length within same row_id is to small
- [ ] Use dictionary array type to represent non-collapsible columns to reduce memory usage.
- [ ] Benchmark between two format
Additional Context
https://github.com/CeresDB/ceresdb/issues/77