parquet-go icon indicating copy to clipboard operation
parquet-go copied to clipboard

Support encoding extensions

Open Pryz opened this issue 3 years ago • 0 comments

We have been talking about supporting encoding that are not standards Parquet encoding in this library. Idea will be to implement additional encoding mechanisms that can be leverage for specific types of data.

For example, we could implement a XOR-based encoding like used in Gorilla.

To avoid confusion, these additional encoding mechanisms will be hosted in a dedicated package github.com/segmentio/parquet-go/encoding/x.

As a second step, we could also adapt the schema mechanism to allow dynamic values in the parquet tag but this is likely out of scope for now.

Pryz avatar Jul 26 '22 23:07 Pryz