pathway
pathway copied to clipboard
Add Apache Iceberg input connector
Is your feature request related to a problem? Please describe. There is a need for Apache Iceberg input and output connectors. They need to work in both, streaming and static modes. In the case of a streaming mode, updates must be applied incrementally, without fully rereading the table.
Describe the solution you'd like There is a native Rust crate for Iceberg. It can be used for an Iceberg connection.
Describe alternatives you've considered Currently, Iceberg storage can only be accessed by implementing a custom Python input connector. However, this solution is not stable enough for production.