Before stabilizing vng...

[jamii]

Some changes worth trying before vng is stabilized:

• Don't use containers for primitives.
  • For fixed-length types just concatenate all the bytes. Reading can be zero-copy.
  • For integers, just write varints without containers. This halves the number of varints we have to decode on read.
  • For strings/bytes, concatenate bytes into one column and (varint) lengths into another. Reading only requires decoding the lengths.
• Try replacing varints with a faster encoding (eg https://arxiv.org/pdf/1709.08990.pdf).
• Try compressing type/union tags (maybe RLE).