oso
oso copied to clipboard
Expose our Iceberg models as public data sets
What is it?
In a BigQuery world, it was really easy to just expose all our models as a public data on Analytics Hub.
If we move to a world where more models run on sqlmesh+Trino, how can we expose this functionality?
Couple thoughts:
- Export parquet to Cloudflare R2 https://github.com/opensource-observer/oso/issues/919
- Expose parquet on GCS with consumer pays
- Copy data into BigQuery to expose in Analytics Hub as usual
I wonder how much storage costs, but I'm inclined towards #3 to avoid too much disruption.