Improve memory allocation to avoid OOM

[lucaderi]

Both

• NetworkInterface::sortHosts()
• NetworkInterface::sortFlows()

do a static alloc

 retriever->elems = (struct flowHostRetrieveList *)calloc(sizeof(struct flowHostRetrieveList), retriever->maxNumEntries);

that can cause OOM in case of large hashes.

Solution

• allocate only a number of items large as the current number of entries
• enlarge the memory via realloc as needed

[cardigliano]

It is likely the reported OOM comes from the top talkers computation. See the logs below:

[LuaEngine.cpp:1493] WARNING Script failure [/usr/share/ntopng/scripts/lua/pro/rest/v2/get/interface/top/remote/talkers.lua][/usr/share/ntopng/scripts/lua/pro/modules/top_talkers.lua:32: attempt to index a nil value (local 'hosts_stats')] [NetworkInterface.cpp:6481] WARNING Out of memory :-( [LuaEngine.cpp:1493] WARNING Script failure [/usr/share/ntopng/scripts/lua/pro/rest/v2/get/interface/top/local/talkers.lua]

[MrdUkk]

i think v6.6 is affected very seriously. encountered a LOT of OOMs . uptime of ntopng daemon is between 4 and 10 minutes and then crash OOM. 30 exporters. 30 interfaces. hosts on each over 15.000 . flows over 1m. config relevant params. host has 32gb memory ---max-num-flows=1512000 ---max-num-hosts=262000

[cardigliano]

@MrdUkk please drop me an email, I will provide instructions to debug this (cardigliano at ntop.org)