peerdb icon indicating copy to clipboard operation
peerdb copied to clipboard

Support for Apache Iceberg

Open mvgijssel opened this issue 1 year ago • 7 comments

Hi folks!

Not sure if this is the right place to ask, but are there plans to support Apache Iceberg natively? I understand it's possible to create a S3 peer to store files in the AVRO format, but this would still require an additional pipeline to convert those files for Apache Iceberg.

mvgijssel avatar Jun 13 '24 13:06 mvgijssel

#1792

serprex avatar Jun 13 '24 13:06 serprex

I need to learn how to search 🙈

mvgijssel avatar Jun 13 '24 14:06 mvgijssel

Hi there @mvgijssel! Just a quick question, what are you planning to use as the underlying iceberg catalog?

iamKunalGupta avatar Jun 14 '24 06:06 iamKunalGupta

I was contemplating loading it into Snowflake Iceberg tables!

mvgijssel avatar Jun 14 '24 10:06 mvgijssel

I was contemplating loading it into Snowflake Iceberg tables!

Are you planning to use the Snowflake Catalog or or use some other Catalog?

iamKunalGupta avatar Jun 14 '24 18:06 iamKunalGupta

Snowflake catalog sounds the easiest! If that doesn’t work we can run our own / check if AWS has something.

mvgijssel avatar Jun 14 '24 18:06 mvgijssel

#1697 #1869 #1870

iamKunalGupta avatar Jun 24 '24 23:06 iamKunalGupta