memgraph
[BUG] Community_detection does not release memory while it's processing, uses > 2x db size
Memgraph version 2.10.2
Environment AWS EC2 Ubuntu 22 running memgraph-platform in a docker container
Describe the bug I run the following query on BlueJ to do community detection:
CALL community_detection.get() YIELD node, community_id WITH node, community_id WHERE 'Person' IN labels(node) SET node.community_id = community_id;
And the DB size on a fresh start uses 13 out of 28 gigs of memory
Once I run the query, it'll execute and allocate memory until all of the 28 Gb avalable is taken, and keeps going -- up until there's no memory left for allocating file / network handles, so ssh and other services become unavailable.
The only solution is to reboot the server as it'll stay in this state for multiple days
To Reproduce
Download the latest bluej db snapshop from https://bluej.memgraph.com/dump/bluej-latest.cypherl.gz and run the query on a linux host (& other platforms presumably) with 32 gb of ram. The machine will crash
Expected behavior No crash. Crash bad
Screenshots of start of query & moment the server became unavailable:
This issue started from this one on Memgraph: https://github.com/memgraph/memgraph/issues/1219
Issue here is that we need to add mgp_track_current_thread for all allocations happening in new threads
