oso icon indicating copy to clipboard operation
oso copied to clipboard

Improve performance of `timeseries_metrics_by_artifact_v0`

Open ccerv1 opened this issue 8 months ago • 2 comments

What is it?

This table is too big to query efficiently. We can't even export it to BigQuery. A few things:

  • We should look to better partitions and other performance optimizations in the model itself
  • We should probably consider not exporting every metric permutation that exists

ccerv1 avatar Apr 30 '25 04:04 ccerv1

IMO, It's like a P3 to optimize performance on BigQuery.

On Clickhouse, there's indices, order_bys, and other built-in optimizations that will help performance without making any explicit changes to the data. (No need to filter or explicitly partition)

What's the use case you're solving for?

ryscheng avatar Apr 30 '25 15:04 ryscheng

I think this is a duplicate of https://github.com/opensource-observer/oso/issues/3182

ryscheng avatar Jun 03 '25 19:06 ryscheng