horaedb icon indicating copy to clipboard operation
horaedb copied to clipboard

Enhance to hybrid storage format

Open jiacai2050 opened this issue 3 years ago • 0 comments

Describe This Problem

Hybrid storage format with basic feature is already merged, but there are still things to be done to gain good performance.

Open an new issue to tracking those.

Proposal

Write

  • [x] Support variable-length type for ListArray
  • [ ] Support table without tsid, only a row id is required
  • [ ] Persist hybrid collapsible info

Read

  • [ ] Support timestamp column filter, some extra columns may be needed
  • [x] Support variable-length type for ListArray
  • [ ] Enable a total ordering, to support query with pagination

Misc

  • [ ] Compaction support
  • [ ] Ensure row group size is large enough, in case of list length within same row_id is to small
  • [ ] Use dictionary array type to represent non-collapsible columns to reduce memory usage.
  • [ ] Benchmark between two format

Additional Context

https://github.com/CeresDB/ceresdb/issues/77

jiacai2050 avatar Aug 30 '22 02:08 jiacai2050