trino
trino copied to clipboard
Support setting compression_codec table properties for Iceberg Connector
This PR add support for setting compression_codec table properties for Iceberg Connector.
Users are able to run the following command now to specify the compression_codec to create a new table with compression_codec set to ZSTD.
CREATE TABLE example (
c1 INTEGER,
c2 DATE,
c3 DOUBLE
)
WITH (
format = 'AVRO',
partitioning = ARRAY['c1', 'c2'],
sorted_by = ARRAY['c3'],
compression_codec = 'ZSTD'
);
Users are able able to change the compression_codec via statement Alter Table Set Properties.
Example
ALTER TABLE example SET PROPERTIES compression_codec = 'GZIP';
The write_compression users specify will take precedence over session variable iceberg.compression-codec. If the user change file_format without changing write_compression, it will inherit the write_compression set for the original file format, if the write_compression for the previous file format is set.
If the user is trying to set a write_compression that is inconsistent with the file format, the system will throw an exception.
The compatibility matrix for write_compression and file format is the following
| Orc | Parquet | Avro | |
|---|---|---|---|
| NONE | ✅ | ✅ | ✅ |
| SNAPPY | ✅ | ✅ | ✅ |
| LZ4 | ✅ | ❌ | ❌ |
| ZSTD | ✅ | ✅ | ✅ |
| GZIP | ✅ | ✅ | ✅ |
Description
Additional context and related issues
Release notes
( ) This is not user-visible or is docs only, and no release notes are required. ( ) Release notes are required. Please propose a release note for me. ( ) Release notes are required, with the following suggested text:
## Section
* Fix some things. ({issue}`20401`)