hyperdx
hyperdx copied to clipboard
use distinct limit to replace groupUniqArray to get recommend value.
current, we use groupUniqArray to search db to get recommend value.
it need query all data:
SELECT groupUniqArray(1)(LogAttributes['logtag']) from otel_logs settings max_rows_to_read=3000000,read_overflow_mode='break'
// 1 row in set. Elapsed: 1.927 sec. Processed 3.30 million rows, 1.98 GB (1.71 million rows/s., 1.03 GB/s.)
Peak memory usage: 772.08 MiB.
the distinct not need to query all data:
select distinct(LogAttributes['logtag']) from otel_logs limit 1 settings max_rows_to_read=3000000,read_overflow_mode='break'
// 1 row in set. Elapsed: 1.306 sec. Processed 307.24 thousand rows, 139.80 MB (235.19 thousand rows/s., 107.02 MB/s.)
Peak memory usage: 724.45 MiB.