phptools-docs
phptools-docs copied to clipboard
DEVSENSE
Holy RAM
I guess this is why it's so slow. Why is this service taking up so much memory?!
Thank you for reporting this. I know it's happening but we couldn't find out why.
May I ask you for more details? I'd be happy to fix this asap.
- what is the size of your workspace
- maybe used packages?
- for how long is it running, how long is the VSCode workspace opened
- or any steps to reproduce?
I'd appreciate any info.
Sure. The entire workspace is nearly 20GB in total. But, that's all files and not just PHP files, which the language server should only be processing. I'm using the following command which is giving me roughly 1GB for all the PHP files in the workspace.
find . -name "*.php" -exec du -ch {} \; | awk '{total+=$1}END{print total}'
The editor has been running for maybe a day - not even sure. VSCode update and restarts itself a lot and I've had to restart it a lot recently with this plugin. That shouldn't be a huge consideration or issue. It's not like it's been running beyond what would be considered normal.
No idea on reproducing. This issue doesn't occur with Intelliphense.
It does count the vendor directories, as it should. They must be indexed and used in auto-complete.
Thank you! Yes, vendor should be included ... @d8vjork may have found one possible bug (https://github.com/DEVSENSE/phptools-docs/issues/285#issuecomment-1438273371) but it would not cause 6gigs of RAM used.
We'll do more testing, I'd be very interested in reproducing this issue on our machines; we'd perform complete memory profiling ...
It gets worse. Quitting VSCode isn't killing this node server or whatever is handling the indexing. Of course, VSCode was force-quit since it was bringing my system to it's knees by draining available memory and causing system paging.
It ended up corrupting memory on dozens of other apps in the process. Going to have to move back to Intelliphense until this issue is resolved.
I would really love to be able to reproduce this issue :-)
There is probably an infinite loop happening .. we'll keep trying.
@jakubmisek have the same issue, maximum I had was 8GB of RAM after executing lots of composer commands related to install/update. Maybe extension doesn't free memory if file is no longer in use?
@ggolda thank you for letting me know - I'll try composer commands subsequently. Is it something specific? Installing packages? Removing packages? Updating?
@jakubmisek just tried it again, was doing a global laravel update via composer update several times, followed by composer remove {package-name} and composer require {package-name} and my RAM consumption went from 1GB idle to 5GB after executing these commands. Also VSCode reported that it is trying to index 46k files after each command (I guess its fine in this case).
RAM consumption haven't gone down after leaving it in this state for a couple minutes without doing anything.
@ggolda I have a suspicion there is a memory leak in parsing .phar files .. I'll try updating Laravel a few times on various OS's
alright, every time I update/copy a new phpstan.phar file, it consumes about 300MB more RAM and never return it back
I've fixed the memory leak happening when updating a .phar files.
We'll prepare a pre-release within a day.
There are other memory-related improvements for future updates, but this one is fixing the issue at least for me (it happened on most composer update actions)
Hi there, been having similar issues.
The devsense.php.ls process gobbles up memory like there's no tomorrow. Saw ~9GB yesteryday evening after couple hours working on a Shopware 6 project (which is PHP & Symfony based).
When I start vscode and the language server starts indexing files or some such? That uses up 50% of my CPU, increasing memory usage up to ~4.5GB once it's finished.
After that, it keeps increasing it's memory usage the longer I use vscode. Which for me is generally just working files, php and otherwise. No composer installs or vendor changes or some such.
edit: Just realized, Shopware 6 (or more precisely Symfony?) caches alot of PHP, including twig-templates as PHP code. When I clear/warmup the cache the devsense.php.ls process uses 8% cpu for a few seconds and memory usage moves up and down. I guess the language server tries to reindex the deletes/inserts/updates of the cached PHP files?
Anyway, am trying the prerelease v1.32.12898, and will report back. Starting behaviour and the ~4.5GB usage of it is still the same though.
System Information:
vscode: (Extension first installed on 1.76.1, just updated vscode to check if it helps) Version: 1.76.2 Commit: ee2b180d582a7f601fa6ecfdad8d9fd269ab1884 Date: 2023-03-14T17:57:21.103Z Electron: 19.1.11 Chromium: 102.0.5005.196 Node.js: 16.14.2 V8: 10.2.154.26-electron.0 OS: Linux x64 5.15.0-67-generic Sandboxed: No
OS: Linux Ubuntu 22.04.2 LTS
Beyond this issue (had to increase swap, thank god for nvme ssds :D), the extension is great though! Thanks!
Report: After couple of hours working with prerelease v1.32.12898 the ram usage of the devsense.php.ls went down over time.
From the ~4.5GB after vscode startup, to ~3.4GB now.
Although I then went back to stable release and while it started out with ~5.0GB, it went down to ~3.8GB also.
So yeah. I guess today I can't reproduce the excessive ram usage today...
However, I can confirm, the language server does try to index cached PHP files. Is there a way to exclude files/directories from being indexed? Probably would reduce the memory usage for me:
[...] (A whole lot more of these)
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/ef' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/f0' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/f2' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/f5' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/f7' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/f8' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/f9' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/fa' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/fd' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/fe' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/ff' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/pools/system' ...
> Removed path 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/Symfony/Config' ...
[...] (A whole lot more of these)
> Indexing directory 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler' ...
> Indexing directory 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/69' ...
@Songworks thank you for all the information!
Please exclude the "cache" directory using the Visual Studio Code setting "files.exclude", i.e.:
"files.exclude": ["/cache/"]
I hope that will help.
If you'd have anything unexpected in the output again, please let me know. We're trying to improve the RAM usage within every new update.
@jakubmisek Thank you! That helped at lot. On startup the devsense.php.ls is now down to ~2.7GB! :)
However, the language server still seems to detect and index changes to the cache (I guess while a local server with php-fpm is running, it is making some changes to the cache):
PHP Tools server started.
PID: 1094816
Processing files: *.php; *.phtml; *.html5
> Parsing Phar file '/project/path/source-code/ci/vendor/phpstan/phpstan/phpstan.phar' ...
> Parsing Phar file '/project/path/source-code/ci/vendor/phpstan/phpstan/phpstan.phar' ...
> Parsing Phar file '/project/path/source-code/ci/vendor/bin/phpstan.phar' ...
> Parsing Phar file '/project/path/source-code/ci/vendor/bin/phpstan.phar' ...
> 1 file system change(s) ...
> Indexing directory 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/e6' ...
> 1 file system change(s) ...
> Indexing directory 'file:///project/path/source-code/var/cache/dev_h404a97683f5f1de1a7d8afd72c86db43/profiler/1c' ...
[...] And so on
settings.json:
{
"files.exclude": {
"node_modules": true,
"var/cache": true
},
}
@Songworks ... However, the language server still seems to detect and index changes to the cache
I believe it's only logging the information to the output window, but then it's not indexing it. I'll update the log so it's less confusing) Also, I'll make sure it's really working ...
@Songworks That's great, but even 2.7GB is way too high IMO. Right now, intelephense, on my system, is running at 1GB - at least I assume that's the plugin using 1GB, since it's the highest. Our codebase is quite large as well. So, it seems to be corresponding with the total size of all PHP files in our codebase, which makes sense.
It's also worth mentioning that we don't have any "cache" files, unless we're considering Composer autoload files as cache.
We do regenerate our models/entities, which will result in new files, quite a lot of them. I would assume this plugin is watching file pointers and dropping these from it's internal index.
I think RAM as a "cache" system is good for performance, generally speaking disks are slower even the faster SSD doesn't have anything to do with a module of RAM.
2.7GB is nothing compared to what other stuff like Docker consumes as minimum requirement
@oojacoboo thank you for the information. We'll keep working on it. I think the 50% decrease in RAM usage is achievable.
And yes, our PHP extension is maintaining the internal index indeed, keeping only files included in the workspace. The issue was in leaking .phar files.
My memory usage just got to 100% and I found this in the system log. This is AFTER I closed vscode, btw. Why is there so many processes, why is the memory usage so high and why do they keep running when vscode is closed?
Hello Marina,
this is a bug. There should be exactly one devsense.php.ls instance for one code window.
I've noticed that too yesterday, but figured it was a problem on my end as vscode(-insider) crashed several times.
Anyway, checked it just now:
- Opened vscode: devsense.php.ls gets started
- Waiting for indexing to stop, then closing vscode: devsense.php.ls is still running (doing not much other than using up memory)
- Opening vscode again: Another devsense.php.ls gets started
devsense Version: v1.34.13313
Version: 1.80.0-insider Commit: f3aa9a201b2455e6556797e8ffb7145d77adb792 Date: 2023-06-14T09:02:36.066Z Electron: 22.5.7 Chromium: 108.0.5359.215 Node.js: 16.17.1 V8: 10.8.168.25-electron.0 OS: Linux x64 5.15.0-73-generic
Cheers!
@Songworks closing code should stop devsense.php.ls immediately. Sadly this issue is not happening on our test machines.
We just had four people in our team test it. For three of us (Ubuntu, ubuntu, mac) the processes did not stop, for one person (Mac) it did.
@marinaglancy thank you for the quick response. We did some fixes and we'll be doing more testing today. I expect to release a pre-release in the evening.
Weirdly, this is still not happening on our test machines.
If it helps, I just reverted to v1.34.13120 and now the processes stop when I close vscode, so it's some recent regression