clickhouse-operator icon indicating copy to clipboard operation
clickhouse-operator copied to clipboard

Published 20 hours ago • verified publisher icon Altinity

make `metrics-exporter` queries parallel

Open Slach opened this issue 4 years ago • 3 comments

All metrics related to system.parts on HDD + Raid0 + ClickHouse 19.16 take long time when we have many system.parts

SELECT COUNT(*)
FROM system.parts
┌─COUNT()─┐
│   28022 │
└─────────┘

1 rows in set. Elapsed: 3.406 sec.

Slach avatar Jan 18 '21 14:01 Slach

Actually, it should also be improved on ClickHouse side: https://github.com/ClickHouse/ClickHouse/issues/19570

filimonov avatar Jan 25 '21 12:01 filimonov

Why not both?

erikdubbelboer avatar Jan 25 '21 15:01 erikdubbelboer

@Enmk cause most of queries on current metrics exporter use UNION ALL ... and make in parallel on clickhouse side but queries related to system.parts make a lot of "fstat syscall" for calculate all system.parts columns instead of only requested,

so, system.parts related queries is the slowest part of generating /metrics endpoint

Slach avatar Jan 25 '21 15:01 Slach