Slow performance of language server (reproducible)

[driskell]

Describe the bug When running in a large project (monorepo), after saving a file, there is a huge CPU spike in the language server and all operations start to take a long time for files with a lot of references. For example, just hovering over a class method for its PHPDoc will take seconds. In our internal repository it takes around 30 seconds on my MacBook Pro 2019 for it to finish indexing after a single file save.

To Reproduce

cd reproduction
git clone https://github.com/drupal/drupal.git
rm -rf drupal/.git
cd drupal
composer install
cd ..
cp -rf drupal drupal2
cp -rf drupal drupal3
code .

Next open the Intelliphense output and wait for indexing to complete, which will take a minute.

Then open drupal/core/modules/node/src/Entity/Node.php. Find preSave method and hover over it. It might happen instant or take a little due to fresh open. Now once it is showing the PHPDoc instantly, add a few new lines at the start of the preSave function and then save the file. You will see didSave happen in the Intelliphense console. Now go to hover over the preSave (or anything else) and it will say "Loading..." and you'll see the Intelliphense output send the notification for the hover. It will not respond for around 10s on my laptop. Meanwhile the language server is 100+% CPU.

For extra performance issues, add a drupal4 and a drupal5. In my case it seems to exponentially get worse each time you add a new one and when I got to drupal5 I was persistent 100% for around 10 mins and nothing worked.

I realise this is not necessarily a real world example, but it is replicates the same behaviour I have with an internal mono-repo containing around 15 independant symfony projects, a large Drupal project and another large PHP project.

Expected behavior Instant responses from language server.

Screenshots N/A

Platform and version MacBook Pro Core Quad i5 16 GB macOS 12.3.1 VSCode Version: 1.67.2 Intelliphense: 1.8.2

[driskell]

For now, if it helps others, I just exclude all projects in the monorepo from indexing except the one I'm working on, or where I can, only open that subfolder in VSCode. (Sometimes the projects are "embedded" within others, thus needing the exclusion at workspace level.) I do lose Git integrations and some Editorconfig and other global settings from the monorepo root but at least the Intelliphense works.

Keen to help improve this so happy to test things. Part of me does wonder if it's related to multiple instances of a class being available in same or different versions in different files and perhaps something in the language server when it comes to inheritance is calculating all possible combinations or something. It would make sense why a save to a single file causes a huge spurt of activity and why it seems to exponentially get worse.

[bmewburn]

Why do you open a single folder with multiple projects? Because of PHP autoloading mechanisms it is difficult to know (with static analysis) what other files are being included/required for a given file. As such the extension considers a folder the equivalent of a single project and there will be problems when there are name clashes for symbols.

If you have multiple projects you are better off using a multi root workspace which would then provide better separation and configuration of each project. Or even just a vscode instance for each project if they are completely independent of each other.

[driskell]

Why do you open a single folder with multiple projects?

It’s a single repository containing a series of related services and associated utilities so for VSCode extensions involving git to work it would need the whole repository opened. Opening a specific folder is problematic as it then doesn’t have access to shared configa like the root vscode settings and the editorconfigs, etc.

Because of PHP autoloading mechanisms it is difficult to know (with static analysis) what other files are being included/required for a given file. As such the extension considers a folder the equivalent of a single project and there will be problems when there are name clashes for symbols.

Not sure what the solution is or if this is a layout that will be willingly supported but I did feel it wasn’t an uncommon setup even if not common. Seems like the solution is here though - consider the folder. Perhaps there can be a setting to launch multiple language servers by configuring the folder list ahead of time. An alternative could be to automate that and stop at the nearest composer.lock. Maybe through a configuration flag?

If you have multiple projects you are better off using a multi root workspace which would then provide better separation and configuration of each project. Or even just a vscode instance for each project if they are completely independent of each other.

The reason to have the current layout is to avoid separation and separate configuration of each project.

[finn-matti]

I have a similar setup to @driskell, I think. I commented about it in another issue just now: https://github.com/bmewburn/vscode-intelephense/issues/1285#issuecomment-1151021935

@bmewburn I think that monorepos in PHP land will become more and more the norm (at least in the JS/TS world they are already), so performant intelephense support for projects like this would be highly appreciated.

[bmewburn]

There may be some things that can be done to better solve symbol name clashes but there always going to be a performance cost to opening a huge folder with many different projects. More code to index, bigger indexes, plus there is also currently a hard limit on the number of files that can be indexed. Monorepos have disadvantages.

I have with an internal mono-repo containing around 15 independant symfony projects

If they are separate then there is no reason to not open them individually. I believe git features will still work in vscode if .git is in a parent folder.

Opening a specific folder is problematic as it then doesn’t have access to shared configa like the root vscode settings and the editorconfigs

Use a multi root workspace with workspace settings.

[finn-matti]

Use a multi root workspace with workspace settings.

One reason why this is cumbersome is that you are using a monorepo to run tasks on all its modules at once. These scripts and config files you'll have to edit regularly, too. So ideally you want to add the whole monorepo as a "root" to the workspace as well.

Can you exclude a "root" from indexing through intelephense? If so, you can have a "root" per module of the monorepo to make it easier for intelephense, but also be able to view and edit files in the root of the monorepo without taking a significant performance hit. But: If I understand correctly exclusions are filesystem based so this is not possible at the moment. Correct @bmewburn?

P.S.: "root" = a project in a mult root workspace; root = the root of the monorepo.

[AlexKratky]

This will be issue I have mentioned in #2229 . I have monorepo with 4 project, but for example implementing one feature means to edit 2-3 projects at once and largest project have directory that shares code for remaining 3 projects (services, models etc.) I will try the workspace and see if it helps, if I understand it right, if I use workspace it will start 4 seperated language servers instead of one?

[matzeeable]

In addition to @AlexKratky and @driskell: I do not know how Intelephense exactly work in the background, but I just want to post my thinking.

Context

I think performance is a thing for all monorepos and PHP. We also have a single monorepo including Node backends, frontend React package, and PHP projects. This is mainly managed by PNPM as we are mainly in Node / TypeScript development. But we develop also WordPress plugins, and they communicate with our Node backends, that's why we have all this in one monorepo (big advantage when e.g. integration tests). Here is preview:

Use a multi root workspace with workspace settings.

This isn't currently a solution for us because we use other extensions (including custom extensions) which need to "see" the whole monorepo instead of single folders within it. Also, e.g. https://github.com/symfony/symfony is working within a monorepo.

Comparison with "monorepo"-compatible language servers

When we use TypeScript, the IntelliSense is fast. Why? TypeScript proceeds in this way and indexes files per project, not per workspace or open folder. Projects are determined using the tsconfig.json#include. There, it must be listed which files belong to this project (similar to composer.json#autoload). As soon as a TypeScript file is opened in VSCode, a tsconfig.json is searched for in a parent folder until the current file is listed there.

Proposal

Intelephense perhaps can provide a so-called "Index-mode" which can be configured like "intelephense.indexMode": "folder" | "vendor-json". folder is the current way Intelephense works. vendor-json can work like this:

1. Do not index by default (or perhaps with index-warmup by searching all non-excluded vendor.json files in current folder)
2. When opening a .php file, "recursively" find a vendor.json file upwards the current folder of the opened file
3. For each found vendor.json file check if the current opened files is part of vendor.json#autoload|autoload-dev
4. If yes, use index explicitly for the found vendor.json

[bobmagicii]

i just spent all day checking all the performance based bug reports trying to find one that was half way relevant to mine. the result is none of them are good fits but this one suggests its a lots-of-files problem. this report specifically fits the bill of my weak vps on digital ocean cratering itself with node from vscode using all the ram and cpu while intelephense is indexing.

just wanted to tell somebody, anybody, that my projects instantly started sucking butt one day and after finally trying to figure out why, discovered it was google's fault. here is my vendor directory sorted by number of php files they contain.

added **/vendor/google/{apiclient,apiclient-services}/** to my exclude and suddenly it went from indexing 26k files to 4k files. since the monorepo examples in this thread also have vendor directories, is google screwing with yall too?

[driskell]

i just spent all day checking all the performance based bug reports trying to find one that was half way relevant to mine. the result is none of them are good fits but this one suggests its a lots-of-files problem. this report specifically fits the bill of my weak vps on digital ocean cratering itself with node from vscode using all the ram and cpu while intelephense is indexing.

just wanted to tell somebody, anybody, that my projects instantly started sucking butt one day and after finally trying to figure out why, discovered it was google's fault. here is my vendor directory sorted by number of php files they contain.

added **/vendor/google/{apiclient,apiclient-services}/** to my exclude and suddenly it went from indexing 26k files to 4k files. since the monorepo examples in this thread also have vendor directories, is google screwing with yall too?

Good catch. Just a thought but did you checkout the composer setting for Google APIs? It will cleanup that folder for you and remove any service you don’t use: https://github.com/googleapis/google-api-php-client#cleaning-up-unused-services