embucket-labs
embucket-labs copied to clipboard
[EPIC] Implement Snowflake Hybrid Tables
Snowflake supports Hybrid Tables for smaller and medium amounts of data.
These are used by dbt pipelines to store metadata about the pipelines.
These tables are typically interacted with using row-wise operations like INSERT and UPDATE.
Iceberg Tables are not a suitable as a storage solution for Hybrid tables.
We need to find a table implementation that supports these use cases.
Possible solutions:
- SlateDB tables
- Avro tables
Tasks:
- [ ] Create design document