hal
hal copied to clipboard
emsec
Memory leak by design in the console and log
Describe the bug Both the Python console and the log have infinite scrollback. This causes high memory consumption of a few tens to hundreds additional MB when you leave HAL running for a long time (I like to have the tool in a VM and just hibernate that VM, i.e. HAL is never closed) or you print an insanely large amount of data from your Python script.
To Reproduce
- Leave HAL open long enough (and actively work on a project)
- or print large amounts of data from Python
- or execute a gigantic Python script, which is copied as a whole into the log every time it runs
- run
topand see the memory usage of the hal process
Expected behavior HAL should only keep the last few hundred lines of console and log loaded and visible. Older data should be removed from the widgets to save memory.
If you really, really want an infinite scrollback buffer you could put it on a disk and load it on demand / map it into memory.
The logger is actually restricted to a maximum of 1000 blocks. @RenWal Can you give us such a large script you use?
I've just added the maximum block count for the python console as well. @RenWal Does it have an impact for you?
I can still see some memory consumption but the rate has decreased a lot.
This is my test case: print('12345\nA'*1000)
When loading this into the editor and hitting run 100 times (do not use a loop), I can see an increase of the resident set size by about 68.6MiB in the unpatched and about 2.4MiB in the patched version.
Note that in this case, the console reaches the block limit but the log does not: Line breaks in print statements are stripped from the log, so only 100 lines (1 for each execution) exist. This would explain the 2.4MiB memory consumption.
This is probably also what made me think that both log and console were leaking, as I was printing large amounts of data with line breaks from a single print statement, thus the log's block limit was never hit.