trino icon indicating copy to clipboard operation
trino copied to clipboard

Add extra_properties to iceberg table properties

Open ndrluis opened this issue 1 year ago • 18 comments

Description

This PR allows us to pass through additional properties to Iceberg when creating a table in Trino. The additional properties can be provided in the following format

extra_properties = MAP(ARRAY['extra.property.one', 'extra.property.two'], ARRAY['one', 'two'])

The extra properties will not be exposed in SHOW CREATE.

The purpose of this feature is to add properties that can be used by catalogs or other tools. For example, in Tabular, there is a property to optimize tables. For our use case, we want to create our tables with the optimizer.enabled value set to False, and we cannot achieve this without a way to handle arbitrary properties.

Additional context and related issues

Fixes #17427

Release notes

(x) Release notes are required, with the following suggested text:

# Iceberg
* Add support for `extra_properties` table properties. ({issue}`17427`)

ndrluis avatar Jan 16 '24 21:01 ndrluis

regarding maven-checks failure :red_circle:

do the following before submitting the PR to ensure that the code you are changing follows the code conventions:

./mvnw clean install -P errorprone-compiler -DskipTests -nsu -pl :trino-iceberg

findinpath avatar Jan 17 '24 09:01 findinpath

@findinpath @oneonestar @mosabua I made the requested changes!

ndrluis avatar Jan 17 '24 18:01 ndrluis

@findinpath Done!

ndrluis avatar Jan 17 '24 21:01 ndrluis

@mosabua ready for review!

ndrluis avatar Jan 22 '24 17:01 ndrluis

I will work with maintainers for further review.

mosabua avatar Jan 22 '24 18:01 mosabua

The f6fec7a commit includes changes to the Trino table properties for Iceberg naming, adding the suffix to maintain the pattern that already exists but was not being used for ORC_BLOOM_FILTER_FPP and ORC_BLOOM_FILTER_COLUMNS.

I believe that this PR is ready for review.

cc/ @findinpath @oneonestar @ebyhr

ndrluis avatar Feb 14 '24 17:02 ndrluis

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

github-actions[bot] avatar Mar 25 '24 17:03 github-actions[bot]

following this PR, any ETA on when this is going to be released?

jeremytee97 avatar Mar 29 '24 04:03 jeremytee97

@oneonestar Done!

ndrluis avatar Apr 01 '24 01:04 ndrluis

Please rebase on master to resolve logical conflicts.

ebyhr avatar Apr 04 '24 01:04 ebyhr

@ebyhr Done!

ndrluis avatar Apr 08 '24 20:04 ndrluis

Can this go across the finish line with your help @alexjo2144 @ebyhr @findinpath and @cwsteinbach ?

mosabua avatar Jul 22 '24 22:07 mosabua

Any updates on this one? We'd love to see this one in the next release

zhmaeff avatar Jul 29 '24 12:07 zhmaeff

@zhmaeff Thank you for the review! I have fixed it.

ndrluis avatar Jul 30 '24 21:07 ndrluis

@mosabua @ebyhr @findepi looks like this PR is ready for merge

zhmaeff avatar Jul 30 '24 23:07 zhmaeff

Ping @ebyhr @findepi

ndrluis avatar Aug 13 '24 22:08 ndrluis