Delta Lake Support Roadmap

[kevinzwang]

Features

• [x] Basic reads
• [x] Partitioned reads
• [x] Basic writes
• [ ] https://github.com/Eventual-Inc/Daft/issues/1954
  • Blocked by delta-rs
• [x] Utilizing statistics from table metadata for reads
• [ ] https://github.com/Eventual-Inc/Daft/issues/1953
• [ ] https://github.com/Eventual-Inc/Daft/issues/1955
• [ ] Partitioned writes
• [ ] Copy-on-write
• [ ] Deletion vector writing
  • Blocked by delta-rs
• [ ] Add support for writing new DeltaLake tables to catalogs such as AWS Glue and Unity Catalog

Bugs and Testing

• [ ] https://github.com/Eventual-Inc/Daft/issues/2129
• [ ] https://github.com/Eventual-Inc/Daft/issues/2348