airbyte
airbyte copied to clipboard
Destination Iceberg: add AWS Glue Catalog support
Tell us about the problem you're trying to solve
The Iceberg connector is currently supporting Hive/Jdbc/Hadoop catalog. I'd like this connector to support the AWS Glue Catalog.
Describe the solution you’d like
Regarding the Iceberg documentation (https://iceberg.apache.org/docs/latest/aws/) and what the current connector code looks like, I guess we already include the mandatory dependencies :
implementation 'org.apache.iceberg:iceberg-spark-runtime-3.3_2.13:1.0.0'
implementation "software.amazon.awssdk:bundle:2.17.131"
implementation "software.amazon.awssdk:url-connection-client:2.17.131"
We probably should only have to add a new entry into CatalogType + adding an Implementation of IcebergCatalogConfig interface
Are you willing to submit a PR?
I'm not yet enough experienced about Airbyte and Apache Iceberg to do so. But will do it if nobody does 😄
Do you have any plans to offer support for this?
Would be really interested as well!
We are also very interested, looking forward to this support!
Very interested in this. Looking forward to having this use case supported!
Fixed in https://github.com/airbytehq/airbyte/pull/45707