trino icon indicating copy to clipboard operation
trino copied to clipboard

Iceberg Catalog show create catalog fails when glue database has extra parameters

Open raj-manvar opened this issue 6 months ago • 13 comments

Description

When a Glue database has extra parameters, Iceberg connectors fails with io.trino.spi.TrinoException: No PropertyMetadata for property: error

Additional context and related issues

It's possible for a Glue schema to have extra property metadata, for example below database has extra "CreatedBy" property: ( relevant AWS glue client doc )

raj-manvar:~/Downloads$ aws glue get-database --name rajmanvartest
{
    "Database": {
        "Name": "rajmanvartest",
        "Description": "raj manvar test",
        "LocationUri": "s3a://rajmanvartest/",
        "Parameters": {
            "CreatedBy": "raj-manvar"
        },
        "CreateTime": "2025-04-18T16:36:35-05:00",
        "CreateTableDefaultPermissions": [
            {
                "Principal": {
                    "PrincipalIdentifier": "IAM_ALLOWED_PRINCIPALS"
                },
                "Permissions": [
                    "ALL"
                ]
            }
        ],
        "CatalogId": "1111111111111"
    }

For such cases, when running a show create schema rajmanvartest query fails with the below stacktrace

io.trino.spi.TrinoException: No PropertyMetadata for property: CreatedBy
	at io.trino.metadata.PropertyUtil.lambda$toSqlProperties$0(PropertyUtil.java:197)
	at com.google.common.collect.RegularImmutableMap.forEach(RegularImmutableMap.java:300)
	at io.trino.metadata.PropertyUtil.toSqlProperties(PropertyUtil.java:190)
	at io.trino.sql.rewrite.ShowQueriesRewrite$Visitor.showCreateSchema(ShowQueriesRewrite.java:674)
	at io.trino.sql.rewrite.ShowQueriesRewrite$Visitor.visitShowCreate(ShowQueriesRewrite.java:520)
	at io.trino.sql.rewrite.ShowQueriesRewrite$Visitor.visitShowCreate(ShowQueriesRewrite.java:219)
	at io.trino.sql.tree.ShowCreate.accept(ShowCreate.java:61)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.rewrite.ShowQueriesRewrite.rewrite(ShowQueriesRewrite.java:216)
	at io.trino.sql.rewrite.StatementRewrite.rewrite(StatementRewrite.java:54)
	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:93)
	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:87)
	at io.trino.execution.SqlQueryExecution.analyze(SqlQueryExecution.java:289)
	at io.trino.execution.SqlQueryExecution.<init>(SqlQueryExecution.java:222)
	at io.trino.execution.SqlQueryExecution$SqlQueryExecutionFactory.createQueryExecution(SqlQueryExecution.java:892)
	at io.trino.dispatcher.LocalDispatchQueryFactory.lambda$createDispatchQuery$0(LocalDispatchQueryFactory.java:153)
	at io.trino.$gen.Trino_457_20250428_213635_2.call(Unknown Source)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:76)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1570)

This is because the getSchemaProperties for Iceberg Connector's implementation for Glue catalog type implemented here also adds the Glue database parameters.
The above implementation is called from the Iceberg Connector's Connector Metadata implementation here

However, IcebergSchemaProperties only supports the location of the table as the available property, because of this any additional Glue database parameters fails the show create schema query.

The Hive Connector's implementation for Glue type only queries the table location and doesn't consider Glue database parameters as implemented here.

This MR updates the Iceberg connector to not consider Glue database parameters so that Iceberg Catalog's show create schema for these type of Glue schema succeeds.

Release notes

( ) This is not user-visible or is docs only, and no release notes are required. (x) Release notes are required. Please propose a release note for me. ( ) Release notes are required, with the following suggested text:

## Section
* Fix some things. ({issue}`issuenumber`)

raj-manvar avatar Apr 29 '25 03:04 raj-manvar