iceberg-rust
iceberg-rust copied to clipboard
Tracking Issue: Integration with Datafusion
Motivation:
Once we merge #324 and complete the basic groundwork, the integration crate can be continuously refined. Due to for example more supported features like predicate pushdown, etc.
This list is supposed to evolve over time, so feel free to add missing todos.
Todos:
- [ ] proper integration tests for table scan (requires us to setup a table with actual snapshots / data )
- [ ] a cache for IcebergCatalogProvider and IcebergSchemaProvider / so data does not become stale
- [ ] improve impl ExecutionPlan for IcebergTableScan (once we support filter pushdown, etc.)
https://crates.io/crates/object_store_opendal is available to help integration with object_store.