"Analyzing in background" plus high memory usage = system crash

[deichorn]

Environment data

• Language Server version: 0.5.45.0
• OS and version: Ubuntu 18.04.4
• Python version (& distribution if applicable, e.g. Anaconda): 3.8.2

Expected behaviour

Background analysis should complete in a reasonable amount of time, and the process should not hog all available memory on the machine.

Actual behaviour

At some point, I noticed the popular Analyzing in background, X items left... message was 'stuck' in the status bar. I'm not sure how long it had been there. Some time later (sorry, hard to say exactly how long -- maybe 20 minutes?), my machine became incredibly slow, to the point of being unusable. I tried to run top, and it eventually spit out one screen of information:

I had two instances of VS Code running, and so there were two language server instances. You can see that one of them had basically used up all available memory. I had to reboot my machine because it was so busy that I couldn't kill the language server.

After the reboot, I enabled "python.analysis.logLevel": "Trace". After changing several files, I noticed the stuck "Analyzing" message again, so I grabbed the PLS output. I will attach that file. In my very inexperienced view, it seems like the same analysis is repeating over and over. I checked memory usage, and the server process memory usage was starting to climb again, so I closed VS Code.

When I opened VS Code again, the initial analysis completed within a couple seconds. This is the last few lines in the output window:

Analysis complete: 617 modules in 4003.08 ms.
Analysis version 1030 of 617 entries has been completed in 2420.4 ms.

I will also attach the full log.

Logs

Runaway log: language server repeating analysis.txt

Post restart log: language server after restart.txt

Code Snippet / Additional lnformation

I'm not sure what might have triggered the first problem. I can say that the second problem started soon after I had used a formatter to reformat several files (7) in one particular folder. I don't know if maybe edits to several files all "at once" could be related or not. Earlier (before the first problem), I had also been switching between a couple different git branches, which would have been another action that would change a bunch of files "at once".

I'm happy to collect &provide further information if needed.

[deichorn]

Just an extra note -- it does seem like I can reproduce this at will by switching git branches (which causes several files to change almost instantly), so if there is something I can do to provide further diagnostics, please let me know.

It seems to always repeat the same analysis that is shown in the attached output where it is constantly repeating this analysis:

Analysis version 1108 of 207 entries has started.
Analysis of cmd (User) queued. Dependencies: cmd (stub), ujson, kxplatform.platform, kxplatformesp, kxplatformesp.espplatform, wstlsys, io
Out of order analysis of cmd(User) completed in 9.08 ms.
Analysis of modules loop on depth 0 in 89.02 ms:
    cmd(User)
    oldpysnmp.asn1(User)
    kxplatformesp.espsnmpalarmtable(User)
    oldpysnmp.v2c(User)
    oldpysnmp.v1(User)
    aisnmp.managerV2(User)
    kxplatformesp.espplatform(User)
    aisnmp.manager(User)
    kxplatform.platform(User)
    queue(User)
    aisnmp.udpsocketmanager(User)
Analysis version 1108 has been completed in 89.49 ms with 203 entries analyzed and 4 entries skipped.
Analysis version 1109 of 207 entries has started.

We do have a file named cmd.py that imports the modules in the Dependencies list. This is a private repo, so I will not be able to provide the entire repo, but if there are specific questions about the files and how they are organized, I would be happy to answer as much as I can.

[MikhailArkhipov]

Part of the fact that second time is faster is b/c first time there is fair amount of Create built-in compiled (scraped) module which means LS ran python.exe to inspect compiled module and generate stub. The stub is the cached.

[MikhailArkhipov]

#1460

[deichorn]

Forgive me, I'm not sure I follow.

I looked at #1460, and there are some similarities (memory usage grows until the system becomes unusable), but there are also differences.

#1460 seems to be a problem that occurs at VS Code startup. My problem will start long after startup. (When I start VS Code, the initial analysis seems to work fine. It completes in 1 or 2 seconds, and everything functions normally for a long time.)

Also, my machine is not creating more threads.

$ ps -efL | grep Python.LanguageServer | wc -l
23

I check that after starting VS Code. Then, when it happened again this afternoon, I checked it again and it was still 23.

I can attached today's trace output if you'd like, but it basically looks the same as the first attachment. The Analysis of cmd (User) queued repeats constantly with no end (until I reload VS Code).

[deichorn]

This problem is crashing my system on a regular basis.

The LS should not repeat the same analysis over and over until all memory is consumed and the system crashes.

Today's repeating analysis loop is different than previous loops, so I am attaching a new trace file.

2020-05-15 trace output.txt

[elchiapp]

I switched to Pylance as suggested in https://github.com/microsoft/python-language-server/issues/2007 and this has been solved. Also everything seems to be snappier. I almost saturated my swap partition twice with the MS language server.