Notebook too large to backup

[ibobak]

Type: Bug

Behaviour

When a jupyter notebook cell is being executed, the rendering of output happens with huge delay. During this delay I am getting these logs:

024-07-24 21:36:44.139 [error] Error: Notebook too large to backup
    at r.snapshot (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:2223:59569)
    at async I.backup (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:2223:48595)
    at async vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:2204:6230
2024-07-24 21:45:12.596 [error] [12f] potential listener LEAK detected, having 180 listeners already. MOST frequent listener (6):: Error
    at c.create (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:88:57139)
    at h.q [as onDidChange] (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:91:1100)
    at Object.P [as onWillAddFirstListener] (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:790:67117)
    at u.q [as onDidChange] (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:91:1280)
    at k.F (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:2496:109971)
    at k.render (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:2496:107439)
    at P.w (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:2496:113169)
    at P.render (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:2496:112729)
    at new $ (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:2496:148874)
    at f.o (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:655:1608)
    at f.createInstance (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:655:1100)
    at X.renderElement (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:2684:28983)
    at _.renderElement (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:227:2257)
    at T.Z (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:184:16137)
    at vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:184:15157
    at n.transact (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:180:8020)
    at T.Y (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:184:15030)
    at T.Y (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:1134:51171)
    at T.ib (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:184:20194)
    at l.B (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:93:732)
    at l.C (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:93:802)
    at l.fire (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:93:1018)
    at d.value (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:180:55517)
    at l.B (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:93:732)
    at l.fire (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:93:949)
    at L.n (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:93:26768)
    at L.setScrollPositionNow (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:93:25766)
    at E.P (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:180:49852)
    at C.b (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:180:49668)
    at HTMLDivElement.<anonymous> (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:140:1173)
2024-07-24 21:45:27.927 [error] [12f] potential listener LEAK detected, having 178 listeners already. MOST frequent listener (2):: Error
    at c.create (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:88:57139)
    at h.q [as onDidChange] (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:91:1100)
    at Object.P [as onWillAddFirstListener] (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:790:67117)
    at u.q [as onDidChange] (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:91:1280)
    at d.D (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:1134:45843)
    at d.z (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:1134:45352)
    at d.didRenderCell (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:1134:45531)
    at d.renderCell (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:1134:36116)
    at vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:1134:36853
    at y (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:1134:36307)
    at N.d (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:1134:36845)
    at N.execute (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:139:32615)
    at ot (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:139:32888)
    at vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:139:33100

Steps to reproduce:

I don't know how to reproduce in a test project. This is a commercial project - I can't share it, a notebook has 200 cells in it. It has many matplotlib outputs, many prints, logs, etc.

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

Extension version: 2024.10.0 VS Code version: Code 1.91.1 (f1e16e1e6214d7c44d078b1f0607b2388f29d729, 2024-07-09T22:08:12.169Z) OS version: Linux x64 6.4.6-060406-generic Modes:

• Python version (& distribution if applicable, e.g. Anaconda): 3.10.12
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Conda
• Value of the python.languageServer setting: Pylance

User Settings

languageServer: "Pylance"

Installed Extensions

Extension Name	Extension Id	Version
.NET Install Tool	ms-dotnettools.vscode-dotnet-runtime	2.1.1
autoDocstring - Python Docstring Generator	njpwerner.autodocstring	0.6.1
autopep8	ms-python.autopep8	2024.0.0
Black Formatter	ms-python.black-formatter	2024.2.0
C#	ms-dotnettools.csharp	2.39.29
C# Dev Kit	ms-dotnettools.csdevkit	1.8.14
Checkpoints	micnil.vscode-checkpoints	1.3.3
Code Runner	formulahendry.code-runner	0.12.2
Conventional Commits	vivaxy.vscode-conventional-commits	1.25.0
Data Wrangler	ms-toolsai.datawrangler	1.4.2
Dev Containers	ms-vscode-remote.remote-containers	0.375.1
Django	batisteo.vscode-django	1.15.0
Docker	ms-azuretools.vscode-docker	1.29.1
Docs View	bierner.docs-view	0.1.0
Edit csv	janisdd.vscode-edit-csv	0.9.2
Flake8	ms-python.flake8	2023.10.0
Git Extension Pack	donjayamanne.git-extension-pack	0.1.3
Git Extension Pack (GPack)	SeyyedKhandon.gpack	2.0.0
Git File History	pomber.git-file-history	1.0.1
Git History	donjayamanne.githistory	0.6.20
Github Light Theme	Hyzeta.vscode-theme-github-light	7.14.2
GitHub Plus Theme	thenikso.github-plus-theme	1.4.3
GitHub Theme	GitHub.github-vscode-theme	6.3.4
gitignore	codezombiech.gitignore	0.9.0
GitLens — Git supercharged	eamodio.gitlens	15.2.0
Grey Light+ Pro	elanzalaco.grey-light-plus-pro	0.0.1
Inline SQL	qufiwefefwoyn.inline-sql-syntax	2.16.0
IntelliCode	VisualStudioExptTeam.vscodeintellicode	1.3.1
IntelliCode API Usage Examples	VisualStudioExptTeam.intellicode-api-usage-examples	0.2.8
IntelliCode for C# Dev Kit	ms-dotnettools.vscodeintellicode-csharp	2.1.11
JavaScript Debugger	ms-vscode.js-debug	1.91.0
JavaScript Debugger Companion Extension	ms-vscode.js-debug-companion	1.1.2
Jinja	wholroyd.jinja	0.0.8
julia light	JuliaLight.julia-light	0.4.0
Jupyter	ms-toolsai.jupyter	2024.6.0
Jupyter Cell Tags	ms-toolsai.vscode-jupyter-cell-tags	0.1.9
Jupyter Keymap	ms-toolsai.jupyter-keymap	1.1.2
Jupyter Notebook Renderers	ms-toolsai.jupyter-renderers	1.0.18
Jupyter PowerToys	ms-toolsai.vscode-jupyter-powertoys	0.1.1
Jupyter Slide Show	ms-toolsai.vscode-jupyter-slideshow	0.1.6
Jupyter Theme	sam-the-programmer.jupyter-theme	0.1.3
JupyterLab Light Theme	MiguelCorralJr.jupyterlab-light-theme	0.1.1
Light Pink Theme	mgwg.light-pink-theme	0.6.0
Markdown All in One	yzhang.markdown-all-in-one	3.6.2
Markdown Preview Enhanced	shd101wyy.markdown-preview-enhanced	0.8.13
MZ Light Grey Theme	Malaz-YI.mz-light-grey	2.1.1
Night Owl	sdras.night-owl	2.0.1
Notepad++ color themes	ShayanAhmedKhan.notepadpp-color-theme	0.2.0
Numbered Bookmarks	alefragnani.numbered-bookmarks	8.5.0
One Dark Pro	zhuangtongfa.material-theme	3.17.2
One Dark Theme	mskelton.one-dark-theme	1.14.2
Open in GitHub, Bitbucket, Gitlab, VisualStudio.com !	ziyasal.vscode-open-in-github	1.3.6
Path Intellisense	christian-kohler.path-intellisense	2.9.0
Project Manager	alefragnani.project-manager	12.8.0
Pylance	ms-python.vscode-pylance	2024.7.1
Pylint	ms-python.pylint	2023.10.1
Python	ms-python.python	2024.10.0
Python Debugger	ms-python.debugpy	2024.8.0
Python Environment Manager	donjayamanne.python-environment-manager	1.2.4
Python Extension Pack	donjayamanne.python-extension-pack	1.7.0
Python Extension Pack	LeoJhonSong.python-extension-pack	3.0.0
Python Indent	KevinRose.vsc-python-indent	1.18.0
Remote - SSH	ms-vscode-remote.remote-ssh	0.112.0
Remote - SSH: Editing Configuration Files	ms-vscode-remote.remote-ssh-edit	0.86.0
Remote - Tunnels	ms-vscode.remote-server	1.5.2
Remote Development	ms-vscode-remote.vscode-remote-extensionpack	0.25.0
Remote Explorer	ms-vscode.remote-explorer	0.4.3
Studio Icons	jtlowe.vscode-icon-theme	1.6.6
Table Visualizer for JavaScript Profiles	ms-vscode.vscode-js-profile-table	1.0.9
Visual Studio Keymap	ms-vscode.vs-keybindings	0.2.1
WSL	ms-vscode-remote.remote-wsl	0.88.2
Zoomer	anthonyattard.zoomer	0.3.1

System Info

Item	Value
CPUs	Intel(R) Xeon(R) CPU E5-2696 v4 @ 2.20GHz (88 x 1453)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	2, 2, 2
Memory (System)	251.76GB (218.99GB free)
Process Argv	--crash-reporter-id 27d42247-63fb-4d9e-9cb0-87d9974843dc
Screen Reader	no
VM	50%
DESKTOP_SESSION	ubuntu-xorg
XDG_CURRENT_DESKTOP	Unity
XDG_SESSION_DESKTOP	ubuntu-xorg
XDG_SESSION_TYPE	x11

A/B Experiments

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscoreces:30445986
vscod805:30301674
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
2i9eh265:30646982
962ge761:30959799
pythongtdpath:30769146
welcomedialog:30910333
pythonnoceb:30805159
asynctok:30898717
pythonregdiag2:30936856
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
accentitlementst:30995554
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
9c06g630:31013171
pythoncenvpt:31062603
a69g1124:31058053
dvdeprecation:31068756
dwnewjupyter:31046869
2f103344:31071589
impr_priority:31102340
refactorc:31101458
wkspc-onlycs-t:31102394

[guidocioni]

This started happening to me as well.. My notebook does not have many cells but it has one cell with a plotly trace that contains many scatter points, which is causing the ipynb to hover around 95 MB in size. I understand this is large for a notebook, but I don't understand what this "backup" should be and in any case it shouldn't really make the saving process hang for a few seconds.

[amunger]

backing up happens to be able to restore unsaved content, so if the file is saved, no backup is needed. Failing to backup occurs specifically to avoid causing any performance issues, so that shouldn't be the cause of any delay. The other log about too many listeners could be causing issues tho

[guidocioni]

backing up happens to be able to restore unsaved content, so if the file is saved, no backup is needed. Failing to backup occurs specifically to avoid causing any performance issues, so that shouldn't be the cause of any delay. The other log about too many listeners could be causing issues tho

Well I cycled through all the output logs and couldn't find anything writing at the same time I was saving the file... The only one is the notebook backup.

But I have to say that the original issue was citing celle execution while for me the Problem is indeed a delay in saving this particular notebook.

[amunger]

Saving a notebook that large can indeed put some pressure on performance - the entire 90 MB JSON string is serialized before it's written to disk.

[aodj-snjallgogn]

Are there any settings to control this? My notebook is 58MB according to ls.

[amunger]

notebook.backup.sizeLimit will adjust the limit if that's what you're looking for, but it will start degrading performance at some point.

You can also try ipynb.experimental.serialization which makes it do the json serialization on the worker thread.

[amunger]

Try these settings and let us know if that helps resolve the performance issues

"ipynb.experimental.serialization": true
"notebook.experimental.remoteSave": true

[ibobak]

I would gladly help testing, but with all respect, 5 months passed, and I am working in a different company now.