polars icon indicating copy to clipboard operation
polars copied to clipboard

Rust API for editing metadata when creating Parquet and IPC/Arrow output

Open thinkingfish opened this issue 9 months ago • 1 comments

Description

Currently there doesn't seem to be a way to add or modify metadata (as key-val map) either at the file or column level when exporting dataframe/series to parquet/arrow formats. This could be useful for preserving data lineage and key attributes during data transformation, as Polars gets used as part of a much more extensive data pipeline.

thinkingfish avatar Apr 26 '24 20:04 thinkingfish

Yeap, would accept a PR on this.

ritchie46 avatar Apr 27 '24 07:04 ritchie46

Are we talking about doing this with polars natively?

deanm0000 avatar May 07 '24 04:05 deanm0000