promscale_extension
promscale_extension copied to clipboard
Optimize queries in the drop_metric_chunks path
While working on https://github.com/timescale/promscale/pull/1634, we discovered that some queries on the metrics retention code path could be significantly sped up.
WITH conf AS MATERIALIZED (SELECT _prom_catalog.get_default_retention_period() AS def_retention)
SELECT count(*)
FROM _timescaledb_catalog.dimension_slice ds
INNER JOIN _timescaledb_catalog.dimension d ON (d.id = ds.dimension_id)
INNER JOIN _timescaledb_catalog.hypertable h ON (h.id = d.hypertable_id)
INNER JOIN _prom_catalog.metric m ON (m.table_name = h.table_name AND m.table_schema = h.schema_name)
JOIN conf ON TRUE
WHERE ds.range_start < _timescaledb_internal.time_to_internal(now() - coalesce(m.retention_period, conf.def_retention))
AND ds.range_end < _timescaledb_internal.time_to_internal(now() - coalesce(m.retention_period, conf.def_retention));
Unlike the naïve query, based on show_chunks
, this seems to fall consistently under 70ms on a test system regardless of the number of chunks it discovers.
The naïve query for comparison, which is, roughly, what the maintenance job does presently:
SELECT count(sc.sc)
FROM _prom_catalog.metric m,
LATERAL _prom_catalog.get_default_retention_period() ret_p,
LATERAL _prom_catalog.get_storage_hypertable_info(m.table_schema, m.table_name, m.is_view) hi,
LATERAL public.show_chunks(hi.hypertable_relation,
older_than=>coalesce(m.retention_period,
ret_p)) sc;
It takes from 30ms to over a second, depending on the number of chunks it counts.