qryn
qryn copied to clipboard
Clickhouse Overcommit error
Qryn threw the following error Code: 241. DB::Exception: Memory limit (total) exceeded: would use 7.21 GiB (attempt to allocate chunk of 4476253 bytes), maximum: 7.20 GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker.: (while reading column labels): (while reading from part data/3bebf7bf-6abc-4a92-ae16-834c90cbafbc/20240925_0_24268_13230/ in table qryn.time_series (3bebf7bf-6abc-4a92-ae16-834c90cbafbc) located on disk s3WithKeeperDisk of type s3, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSelect(pool: PrefetchedReadPool, algorithm: Thread). (MEMORY_LIMIT_EXCEEDED) (version 24.5.1.22957 (official build))
When digging on the clickhouse console, the query that failed appears to be...
WITH idx AS (select fingerprint
from qryn
.time_series_gin
as time_series_gin
where ((((key
= 'name') and (val
= 'kamailio_app_server_dead'))) and (date
>= toDate(fromUnixTimestamp(1727294240))) and (date
<= toDate(fromUnixTimestamp(1727294540))) and (type
in (0,0))) group by fingerprint
having (groupBitOr(bitShiftLeft(((key
= 'name') and (val
= 'kamailio_app_server_dead'))::UInt64, 0)) = 1)), raw AS (select argMaxMerge(last) as value
,fingerprint
,intDiv(timestamp_ns, 15000000000) * 15000 as timestamp_ms
from metrics_15s
as metrics_15s
where ((fingerprint
in (idx)) and (timestamp_ns
>= 1727294240000000000) and (timestamp_ns
<= 1727294540000000000) and (type
in (0,0))) group by fingerprint
,timestamp_ms
order by fingerprint
,timestamp_ms
), timeSeries AS (select fingerprint
,arraySort(JSONExtractKeysAndValues(labels, 'String')) as labels
from qryn
.time_series
where ((fingerprint
in (idx)) and (type
in (0,0)))) select any(labels) as stream
,arraySort(groupArray((raw.timestamp_ms, raw.value))) as values
from raw as raw
any left join timeSeries as time_series on time_series
.fingerprint
= raw.fingerprint group by raw
.fingerprint
order by raw
.fingerprint
FORMAT RowBinary
This caused the prometheus query (for alarm monitoring) to fail: kamailio_app_server_dead{}
Am I doing something stupidly wrong?