gazelle_plugin
gazelle_plugin copied to clipboard
[NSE-956] allow to write parquet with compression
What changes were proposed in this pull request?
This patch adds support for writing parquet with compression
df.coalesce(1).write.format("arrow").option("parquet.compression","zstd").save(path)
this patch depends on https://github.com/oap-project/arrow/pull/122
Signed-off-by: Yuan Zhou [email protected]
How was this patch tested?
pass jenkins
https://github.com/oap-project/native-sql-engine/issues/956