spyder-ide
Memory leak with Matplotlib interactive backends
Description
What steps will reproduce the problem?
Open Spyder, run no code, and leave main IDE window open. A process called 'python' appears in top and consumes growing amounts of memory, seemingly with no upper limit and causing system slowing after apprixmately 2 hours on a 32gb machine.
This was never noticed prior to updating to Spyder 5.2.2 through anaconda (default channel).
Versions
- Spyder version: 5.2.2
- Python version: 3.8.13
- Qt version: 5.12.9
- PyQt5 version: 5.12.3
- Operating System: Linux 5.15.0-43-generic
Dependencies
# Mandatory:
atomicwrites >=1.2.0 : 1.4.0 (OK)
chardet >=2.0.0 : 4.0.0 (OK)
cloudpickle >=0.5.0 : 2.0.0 (OK)
cookiecutter >=1.6.0 : 1.7.3 (OK)
diff_match_patch >=20181111 : 20200713 (OK)
intervaltree >=3.0.2 : 3.1.0 (OK)
IPython >=7.6.0;<8.0.0 : 7.31.1 (OK)
jedi >=0.17.2;<0.19.0 : 0.18.1 (OK)
jellyfish >=0.7 : 0.9.0 (OK)
jsonschema >=3.2.0 : 3.2.0 (OK)
keyring >=17.0.0 : 23.4.0 (OK)
nbconvert >=4.0 : 6.4.4 (OK)
numpydoc >=0.6.0 : 1.4.0 (OK)
parso >=0.7.0;<0.9.0 : 0.8.3 (OK)
pexpect >=4.4.0 : 4.8.0 (OK)
pickleshare >=0.4 : 0.7.5 (OK)
psutil >=5.3 : 5.9.0 (OK)
pygments >=2.0 : 2.11.2 (OK)
pylint >=2.5.0 : 2.14.5 (OK)
pyls_spyder >=0.4.0 : 0.4.0 (OK)
pylsp >=1.3.2;<1.4.0 : 1.3.3 (OK)
pylsp_black >=1.0.0 : 1.0.0 (OK)
qdarkstyle =3.0.2 : 3.0.2 (OK)
qstylizer >=0.1.10 : 0.1.10 (OK)
qtawesome >=1.0.2 : 1.0.3 (OK)
qtconsole >=5.2.1;<5.3.0 : 5.2.2 (OK)
qtpy >=1.5.0 : 2.0.1 (OK)
rtree >=0.9.7 : 0.9.7 (OK)
setuptools >=49.6.0 : 61.2.0 (OK)
sphinx >=0.6.6 : 5.0.2 (OK)
spyder_kernels >=2.2.1;<2.3.0 : 2.2.1 (OK)
textdistance >=4.2.0 : 4.2.1 (OK)
three_merge >=0.1.1 : 0.1.1 (OK)
watchdog >=0.10.3 : 2.1.6 (OK)
xdg >=0.26 : 0.27 (OK)
zmq >=17 : 23.2.0 (OK)
# Optional:
cython >=0.21 : None (NOK)
matplotlib >=2.0.0 : 3.5.1 (OK)
numpy >=1.7 : 1.23.1 (OK)
pandas >=1.1.1 : 1.4.3 (OK)
scipy >=0.17.0 : 1.7.3 (OK)
sympy >=0.7.3 : None (NOK)
Hey @scott-warren, thanks for reporting. Our next version (5.3.3) will contain several fixes related to memory leaks. Please wait until the end of month, when it's going to be released, and try again. For older versions, there's nothing we can do, sorry.
By the way, I'm also on Linux and using Spyder for several days in a row and haven't experienced the kind of memory leak you described.
Finally, to try versions still not available in Anaconda, you need to create a new environment with conda-forge packages. For that, please close Spyder, open a terminal and run the following commands there:
conda create -n spyder-cf -c conda-forge spyder
conda activate spyder-cf
spyder
I've been dealing with this bug myself across several different versions, where spyder is leaking memory from uptime alone. An idle spyder session would go from 3gb to 13gb memory consumption overnight and cause heavy computer lag that only resolved by closing down the spyder session.
I may have identified a workaround: changing the graphics backend in Preferences > IPython console > Graphics > Graphics backend > Backend: to "Inline" seems to have resolved the uptime-related memory leak. I have tested that Tkinter and Automatic backends produce the uptime memory leak.
About my installation:
- Spyder version: 5.3.2 | Python 3.10.4 64-bit | Qt 5.15.2 | PyQt5 5.15.7 | Linux 5.4.0-124-generic
- Running on Ubuntu 20.04.4
- This instance of spyder is installed in an empty conda environment running Python 3.10, in which I pip-installed spyder.
Hey @danielsvedberg, thanks for chiming in. Do you see that problem with Jupyter too? The thing is we use the same architecture as Jupyter to run code, so if the memory leak also shows up there, it means the problem could be caused by that architecture or by Matplotlib itself. If not, we need to investigate further to solve it in Spyder.
To check that, you can start a notebook and then run there
%matplotlib tk
or
%matplotlib qt
which activates either the Tkinter or Automatic backend, respectively.
Hey @ccordoba12, sorry for the late follow-up. Just performed the check you asked--opened up a jupyter notebook in the same conda environment as my spyder installation, and tried running both %matplotlib tk and %matplotlib qt. Neither produces the leak in jupyter.
Ok, thanks for the confirmation. This seems hard to debug but we'll try to address it in the next couple of months.
Should be fixed by https://github.com/ipython/ipykernel/pull/990
Thanks @impact27 for your help with this!
@ccordoba12 Should this be closed as this is an upstream issue
Although it's an upstream issue, we can increase our minimal required version of ipykernel in spyder-kernels (once a new version is released) so that our users don't encounter this bug in the future.