ntop
ntopng clickhouse export ERROR SQL error / ERROR SQL query
Environment:
- OS name: Ubuntu
- OS version: 22.04
- Architecture: amd64
- ntopng version/revision: Enterprise XL v.6.6.251119 rev.26996 (Ubuntu 22.04.4 LTS)
What happened: after upgrading from v6.4 top to last available stable got error in log:
24/Nov/2025 13:56:51 [ClickHouseDB.cpp:484] ERROR SQL error (zmq://192.168.5.2:5555c): DB::Exception: Block structure mismatch in (columns with identical name must have identical structure) stream: different types: materialize(interface_id) UInt16 UInt16(size = 0) materialize(interface_id) Nullable(UInt16) Nullable(size = 0, UInt16(size = 0), UInt8(size = 0))
24/Nov/2025 13:56:51 [ClickHouseDB.cpp:483] ERROR SQL query (zmq://192.168.5.2:5555c): SELECT severity, (toUnixTimestamp(tstamp) - toUnixTimestamp(tstamp) % 58) as slot, count(*) count FROM all_alerts_view WHERE (((tstamp >= 1763979992 AND tstamp <= 1763981792) OR (tstamp >= 1763893592 AND tstamp < 1763979992 AND alert_status = 2))) AND ( (require_attention = 1 AND NOT alert_status = 1) ) AND (interface_id = 51) GROUP BY severity, slot ORDER BY severity, slot ASC
How did you reproduce it? start ntopng
Debug Information:
When opening ntopng, and it triggers this alert, in which page is it going to load for ntopng? I mean, is it going to open the dashboard?
i was appears after upgrade. when i fist enter to web admin. after 24hours i didn't encounter this. may be not triggered some path leading to this error.
will continue to monitor.
Do i need to create a separate issue ticket for the problem possible related to upgraded 6.4 to 6.6 version? encountered first OOM. Nov 27 12:18:28 systemd[1]: ntopng.service: A process of this unit has been killed by the OOM killer. Nov 27 12:18:29 systemd[1]: ntopng.service: Main process exited, code=killed, status=9/KILL Nov 27 12:18:29 systemd[1]: ntopng.service: Failed with result 'oom-kill'. Nov 27 12:18:29 systemd[1]: ntopng.service: Consumed 59min 44.276s CPU time.
Hi @MrdUkk please update to the latest version and let me know if this issue reappears in case i'll send you a debug build
Just to understand, the memory you are displaying was the 6.4 version (when it was low) and 6.6 version (when it's high)?
yes. sorry missed that. v6.4 was in use. on 11/24 i installed v6.6 via deb package upgrade. nothing was changed in environment.
i will try today install new build
Hi @MrdUkk please update to the latest version and let me know if this issue reappears in case i'll send you a debug build
issue was not fixed in Version | ntopng Enterprise XL v.6.6.251203 rev.27056 (Ubuntu 22.04.4 LTS)
fast spontaneously memory grow up from 4gb to 60gb of RAM over 2 hours running. flows limit 1.5mlns. (running values avg observed 650k-990k) hosts limit 250k. (running values 150k avg)
thats look regress from previous v6.6 build . they use 5 -> 58 gb over few hours (~10x rate). and current build ~15x rate in memory consuption.
Hi @MrdUkk regarding the memory issue, we found the problem and trying to address it, just to be sure, try restarting ntopng without clickhouse, and the memory should drop to the previous 5 GB. Are you still crashing? Also do you still have the SQL Error?
currently we are in email communication and triage stage with @cardigliano , current status - it was working flawlessly in v6.6 without clickhouse export. memory consumption is the same as in v6.4. When i was uncommending clickhouse export - memory explosive growth by a factor of ~ x13 and didnt stop. For example: host works over a year with 32gb of RAM. Currently we installed max available hardware memory 128gb and after 3days of run ntopng process consumed 81gb of RAM today with clickhouse exporter enabled. and memory consumption by ntopng process continued to growup. i think withing few days we again got OOM.
SQL Error was postponed currently because crashing in OOM more nasty thing :(