oso
oso copied to clipboard
Improve performance of `timeseries_metrics_by_artifact_v0`
What is it?
This table is too big to query efficiently. We can't even export it to BigQuery. A few things:
- We should look to better partitions and other performance optimizations in the model itself
- We should probably consider not exporting every metric permutation that exists
IMO, It's like a P3 to optimize performance on BigQuery.
On Clickhouse, there's indices, order_bys, and other built-in optimizations that will help performance without making any explicit changes to the data. (No need to filter or explicitly partition)
What's the use case you're solving for?
I think this is a duplicate of https://github.com/opensource-observer/oso/issues/3182