Plugin consumes multiple gigabytes of disk space per workday.

[WyattGosling]

Environment data

• Language Server version: 2023.12.1
• OS and version: linux x64
• Python version (and distribution if applicable, e.g. Anaconda):
• python.analysis.indexing: true
• python.analysis.typeCheckingMode: off

Note: using remote-ssh plugin from a macOS (Sonoma 14.2.1) to a Red Hat server (Red Hat Enterprise Linux Server release 7.9 (Maipo))

Code Snippet

Not Applicable

Repro Steps

1. Open VSCode
2. Use the remote-ssh plugin to connect to my work server
3. Do Python programming all day
4. Check $HOME/.vscode-server/extensions/ms-python.vscode-pylance-2023.12.1/dist

Expected behavior

1. $HOME/.vscode-server/extensions/ms-python.vscode-pylance-2023.12.1/dist contains some data. Maybe a few hundred megabytes, or maybe the low gigabytes.

Actual behavior

$ cd $HOME/.vscode-server/extensions/ms-python.vscode-pylance-2023.12.1/dist

$ > ls -lah core*
-rw-------. 1 wgosling docker    0 Dec 21 05:14 core.30902
-rw-------. 1 wgosling docker 214M Jan  5 02:40 core.21085
-rw-------. 1 wgosling docker    0 Jan  8 23:49 core.8384
-rw-------. 1 wgosling docker 512K Jan  9 03:46 core.4373
-rw-------. 1 wgosling docker    0 Jan  9 10:46 core.18433
-rw-------. 1 wgosling docker 5.0G Jan 10 00:17 core.31729
-rw-------. 1 wgosling docker 5.1G Jan 10 00:48 core.1242
-rw-------. 1 wgosling docker 5.0G Jan 10 23:40 core.21087
-rw-------. 1 wgosling docker 5.1G Jan 11 22:51 core.15653
-rw-------. 1 wgosling docker 5.0G Jan 12 06:41 core.2256
-rw-------. 1 wgosling docker 5.0G Jan 13 01:37 core.22619
-rw-------. 1 wgosling docker 5.0G Jan 13 04:06 core.3129
-rw-------. 1 wgosling docker 4.9G Jan 13 04:07 core.1003
-rw-------. 1 wgosling docker 115M Jan 15 20:29 core.11711
-rw-------. 1 wgosling docker    0 Jan 16 02:28 core.21777
-rw-------. 1 wgosling docker    0 Jan 16 04:53 core.23665
-rw-------. 1 wgosling docker    0 Jan 16 22:50 core.2371
-rw-------. 1 wgosling docker    0 Jan 16 22:50 core.2451

Note: I manually sorted the above output by date/time to make it easier to read.

Notice there are multiple 5GB files here. There are 2 which are a few hundred KB, and several empty ones.

My home directory is on a shared storage server and I have a filesystem limit of 50GB for my home directory. I've been noticing since before Christmas that the Pylance plugin has started running my homedir out of space every few workdays.

I notice that some days can have multiple of these core files. I'm guessing that this has something to do with the fact I have multiple workspaces open at a given time and will have one remote Window open per workspace.

[WyattGosling]

I have an update with info that might help.

I have a multi-root workspace. Only one of the roots has Python code. The others include codebases in C++. I was curious if the problem had to do with traversing the C++ projects. In that root, I set “python.analysis.exclude”: “*”. After more than a week I checked in the dist directory and found there weren’t any core files. I then change the setting to exclude only the build directory. It’s been at least week, and I still don’t have any core files.

My best guess is that the cores have something to do with the large number of symlinks that exist in the build directory and the fact these can lead to loops.

[rchiodo]

@WyattGosling, core files should be from pylance crashing. Can you share the source you work on? We don't generate the .core files, but the OS does when we crash. The fact that they are so large, looks to me like there's an out of memory situation going on.

[bschnurr]

@WyattGosling do you see this in your "Python Language Server" output window logs? Enumeration of workspace source files is taking longer than 10 seconds

see https://github.com/microsoft/pylance-release/issues/3300#issuecomment-1406947296

[WyattGosling]

@bschnurr If I remove python.analysis.exclude, then I will see that message in the logs.

@rchiodo I cannot share the source code I work on. find build/ | wc -l shows over 200,000 files.

[bschnurr]

are you using "openFileOnly" mode "python.analysis.diagnosticMode": "openFilesOnly",

also see this issue on how to exclude folders https://github.com/microsoft/pylance-release/issues/4006#issuecomment-1443559636

[bschnurr]

Closing, sounds like exclude is working