Glue v2

[dain]

Description

New metstore implementation based in the Glue v2 apis. New caching caching system built directly into Glue which supports caching of the full objects returned during listing operations.

Release notes

(X) Release notes are required, with the following suggested text:

# Hive/Delta (not Iceberg)
* Update Glue to V2 REST interface. The old implementation can be restored by setting `hive.metastore="glue-v1`, but this will eventually be removed. ({issue}`20657 `)

[findinpath]

FYI there is also another PR tackling the AWS Glue library update https://github.com/trinodb/trino/pull/17866

[dain]

@findinpath this is closer to a full rewrite than an upgrade, so this would invalidate that other PR

[github-actions[bot]]

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

[sudohainguyen]

jmx metrics io.trino.plugin.hive.metastore.glue.GlueHiveMetastore and io.trino.plugin.hive.metastore.cache are gone since I upgraded to 448 possibily caused by this change I think

[findepi]

@sudohainguyen GlueHiveMetastore isn't used when Glue v2 is used. io.trino.plugin.hive.metastore.glue.InMemoryGlueCache is used instead and the code looks like it exports some metrics. please verify

[sudohainguyen]

hmm I didn't see the classname you mentioned when exploring in jconsole this is all I got

[findepi]

maybe it's not exported because it's not public? i don't know how this works. you will need to experiment a bit

[sudohainguyen]

yeah I tried 😅 but not able to get metrics from InMemoryGlueCache