[BanyanDB] Optimizing Column Encoding

[hanahmily]

Search before asking

• [X] I had searched in the issues and found no similar feature requirement.

Description

There are several optimizations we should apply to the column encoding, the column refers to tags and fields.

• Move constant values within a block to the metadata.
• Encode each column based on its data type (string, int64, or float64).
• Use low cardinality encoding when the column has a limited set of value options within a block.
• Consider encoding array types using a columnar strategy. If the array size is consistent within a block, transform the arrays into a matrix and group and encode the values within the same column.

Use case

No response

Related issues

No response

Are you willing to submit a pull request to implement this on your own?

• [ ] Yes I am willing to submit a pull request on my own!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct