wren-engine
wren-engine copied to clipboard
Support Bi boost
BigQuery
Syntax and data
Metric tables:
- [x] parser: make sure the statement includes metric only. https://github.com/Canner/canner-metric-layer/pull/214
- [x] sql rewrite: rewrite statement and query duckdb with pre-aggregation table https://github.com/Canner/canner-metric-layer/pull/214
- [x] 🔥check return data format or types
- [x] exclude unsupported syntax which can be executed in duckdb,e.g explain, show, describe, insert, create. https://github.com/Canner/canner-metric-layer/pull/214
- [x] provide a workaround for https://github.com/duckdb/duckdb/issues/5673
Storage:
- [x] export data to parquet: https://github.com/Canner/canner-metric-layer/pull/206
- [x] import data to duckdb: https://github.com/Canner/canner-metric-layer/pull/206
- refresh:
- [x] 🔥fix rate
- [ ] expression
Canner
pre-condition
can used in canner
Storage:
- [x] *🔥canner provide export parquet api: provide APIs to export parquets to cloud storage and duckdb support s3 and gcs only.
- [x] *🔥cleaner: canner can clear temp parquets or provide APIs which can be used by accio
- [ ] *🔥 sync result: wire protocol used
- [x] *🔥 async result: web team sql explorer?
- [x] 🔥 adjust refreshing codes
🔥 implementation of export and import data from canner
- [x] 🔥export data from canner
- [x] 🔥import data to duckdb: file base; ~s3, azure blob,~ local (no gcs in canner)
note: * implemente in canner ; 🔥 critical
Others
- [ ] memory management
- [x] 🔥extract module for bi boost
https://github.com/duckdb/duckdb/pull/8302 has been merge
cc @liun03 @wwwy3y3 @brandboat
https://github.com/duckdb/duckdb/pull/8302 has been merge
Many thanks !!!