Elasticsearch debug toolbar causes memory leak

[igo]

Using Elasticsearch debug toolbar causes memory leak and thus crash when elasticsearch is extensively used. Not sure if this is general debug-toolbar problem or specific for elasticsearch-debug-toolbar but I did not notice this problem with other panels. Seems like after each elastic query an elastic_panel.panel.ElasticQueryInfo instance is created which resides in memory forever. I used http://mg.pov.lt/objgraph/objgraph.html#objgraph.show_growth to find out what objects are kept in memory. During tests I ended up with more than 100.000 of ElasticQueryInfo instances.

[Benoss]

I am not surprised, Django itself as leaks in DEBUG mode because it is keeping some references. But in this case those objects should be cleared when collector.clear_collection() is called. And this is done for every request. This is probably a django debug toolbar issue.

[igo]

It leaks even with DEBUG=False. Question is why is this leak relevant only for django-elasticsearch-debug-toolbar and not for every debug-toolbar plugin?

[Benoss]

I'will try to investigate, but if you have any idea do not hesitate :)

[ggaughan]

I think the patched_log_request_success keeps a history for every thread (DEBUG or not) (invoked by Connections), but suspect that the collector.clear_collection() only runs for the thread in charge of the panel (and only when DEBUG), leaving the other thread histories to leak.

[Benoss]

Good catch, The collector is part of core debug toolbar, This plugin should not be the only one leaking :(