[Feature request] Faster syntax highlighting & autocompletion for larger files (10 000 lines of code +)

[ItsCubeTime]

Hi, Ive been working on a tool for a couple of months now where most of the code have all gone in a single file (due to some limitations with the framework I use, it have proved to be more work effective than spreading the code across several files as Im the only contributor to the project).

Ive noticed that it takes quite long however for the syntax highlighting (and to some extent also the autocompletion) to update when making changes further down in the file (several seconds even on an i9-12900K, 32gb DDR4 system) - a lot longer than it takes to fire up the script with python and start using it.

If theres any room for speed improvements, I would highly appreciate it.

https://user-images.githubusercontent.com/20190653/164794082-a534e569-4630-4ca3-9ed0-b3f1ee58aa76.mp4

Demo files

In case a test file would be useful, the __init__.py in this zip is the file in question slow syntax highlight demo files.zip The code is 100% written by me, so I have the rights to post it here. Unfortunately this code cant actually run on its own & it also has several dependencies that the language server is likely going to complain about the absence of when testing on another system. If desired I can list all the modules that can be installed via pip

[erictraut]

Are you using VS Code with Pylance? I just tried opening your file __init__.py with the latest version of Pylance (2022.4.2) using VS Code, and code analysis and syntax highlighting takes about 800ms on my computer. That's longer than typically seen for a Python source file, but it's reasonable given the number of lines of code in this file. You said that it takes "several seconds" on your machine? That implies that you're seeing times in excess of 3000ms?

Pylance is already highly optimized, so it's unlikely that we could make significant performance improvements for analysis of typical code patterns. I don't see anything especially atypical about the code in your file (e.g. highly nested loops, thousands of code flow statements within a single function, or thousands of local variables). Your code even includes type annotations, which reduces the need for the analyzer to infer variable types — a common source of performance issues.

The recommended workaround here is to break your code into multiple modules.

[heejaechang]

I am going to take a look where we spend time most of time during semantic colorization.

[ItsCubeTime]

I am going to take a look where we spend time most of time during semantic colorization.

Thank you, much appreciated 💝✨

[judej]

Moving this issue to discussion as an enhancement request for comments and upvotes.