gazelle_plugin icon indicating copy to clipboard operation
gazelle_plugin copied to clipboard

[NSE-956] allow to write parquet with compression

Open zhouyuan opened this issue 3 years ago • 1 comments

What changes were proposed in this pull request?

This patch adds support for writing parquet with compression

df.coalesce(1).write.format("arrow").option("parquet.compression","zstd").save(path)

this patch depends on https://github.com/oap-project/arrow/pull/122

Signed-off-by: Yuan Zhou [email protected]

How was this patch tested?

pass jenkins

zhouyuan avatar Jul 05 '22 05:07 zhouyuan

https://github.com/oap-project/native-sql-engine/issues/956

github-actions[bot] avatar Jul 05 '22 05:07 github-actions[bot]