[Improvement]: Integrate with Nessie/Polaris REST catalog

[nqvuong1998]

Search before asking

• [X] I have searched in the issues and found no similar issues.

What would you like to be improved?

• Support connect to Nessie/Polaris REST Catalog as an option on Catalogs UI
• Add documentation to integrate with Nessie/Polaris REST Catalog

How should we improve?

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[nqvuong1998]

cc @zhoujinsong

[czy006]

We may wait for Nessie/Polaris to enter the first version of the apache Incubator and have a stable api

[zhoujinsong]

Amoro already supported the custom catalog implementation for Iceberg Format. So you can choose Iceberg Format and custom metastore type for both Nessie and Polaris.

Here is an example of using RESTCatalog for Polaris:

And you can also set the catalog-impl to org.apache.iceberg.nessie.NessieCatalog

Of course, if this requirement is generic enough, we can also add separate "type" options for Nessie and Paloris (Rest).

[nqvuong1998]

Hi @zhoujinsong

1. Does Amoro support advanced authentication properties for the Nessie/Polaris REST catalog, such as Bearer authentication settings and OAuth2 settings?

• Example (Spark with Nessie REST Catalog):

  • spark.sql.catalog.nessie.oauth2-server-uri
  • spark.sql.catalog.nessie.credential
  • spark.sql.catalog.nessie.scope

2. Does Amoro require a custom jar to run with Nessie, such as org.projectnessie.nessie-integrations:nessie-spark-extensions-3.5_2.12?

• Example (Spark with Nessie REST Catalog): spark.sql.extensions=org.projectnessie.spark.extensions.NessieSparkSessionExtensions,org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions[bot]]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'