uwdata
Support Databases other than DuckDB
Create new connectors, for example to Postgres, Clickhouse, or Snowflake. Identify breakage points for Mosaic’s current SQL query generation and design solutions for translating from an internal "canonical" query language to different DB dialects.
A good starting point would be to leverage multi database support in DuckDB.
We're interested in BigQuery + StarRocks adapters. Once there's an adapter pattern, we'd be happy to help contribute SQL interface translations.
https://github.com/tobymao/sqlglot could be an interesting experiment to build a rest server that proxies queries to another database. I'm sure it's still tricky because of data loading but worth a try.
Wrote documentation for how to connect (PostgreSQL, MySQL, SQLite) using DuckDB extensions:
https://github.com/uwdata/mosaic/pull/430
I drafted PR #681 as an experiment to provide support to multiple databases using https://github.com/ibis-project/ibis which in turn uses https://github.com/tobymao/sqlglot under the hood, as suggested by @domoritz. What do you think about the approach?
You're amazing! I have to dig deeper into your pull request but it looks very exciting.
@jheer / @domoritz How would you feel about having an option to send commands to the server as a json representation of the mosaic SQL AST? That would be easier for a db translation layer as compared to having to interpret the materialized SQL