Extremely slow renaming folders

[yajo]

Type: Bug

Using extension bisect, I detected Pylance as guilty for extreme slowliness when renaming a folder.

My workspace contains about 23600 python files, so maybe a slight slowdown can become huge if it tries to scan everything.

Extension version: 2024.2.2 VS Code version: Code 1.86.1 (31c37ee8f63491495ac49e43b8544550fbae4533, 2024-02-07T09:07:19.849Z) OS version: Linux x64 6.7.4-200.fc39.x86_64 Modes:

System Info

Item	Value
CPUs	AMD Ryzen 7 5700U with Radeon Graphics (16 x 4250)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	6, 2, 1
Memory (System)	15.00GB (7.29GB free)
Process Argv	--unity-launch --crash-reporter-id e128b671-8216-478b-b40b-73378743f4d6
Screen Reader	no
VM	0%
DESKTOP_SESSION	gnome
XDG_CURRENT_DESKTOP	GNOME
XDG_SESSION_DESKTOP	gnome
XDG_SESSION_TYPE	wayland

A/B Experiments

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscoreces:30445986
vscod805:30301674
binariesv615:30325510
vsaa593:30376534
py29gd2263:30899288
c4g48928:30535728
azure-dev_surveyone:30548225
a9j8j154:30646983
962ge761:30959799
pythongtdpath:30769146
pythonidxpt:30866567
pythonnoceb:30805159
asynctok:30898717
pythontestfixt:30902429
pythonregdiag2:30936856
pyreplss1:30897532
pythonmypyd1:30879173
pythoncet0:30885854
pythontbext0:30879054
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
bf62j303:30959262
pydisww1:30959800
edj9j583:30958191
a89i1917:30961429
3ef8e399:30964150
bg6jg535:30958357
ccp2r6:30958158

[yajo]

See the bug in action:

https://github.com/microsoft/pylance-release/assets/973709/8987d191-145a-47cb-b22d-900a0ebd354e

[bbrendon]

my remote vscode has been very slow for the past few days. I think i've narrowed it down to pylance. I wonder if your folder rename problem is associated to these cpu issue reports? I have disabled pylance and I think my cpu issue is gone.

https://github.com/microsoft/pylance-release/issues/5070 https://github.com/microsoft/pylance-release/issues/5308

[debonte]

@yajo, is your code publicly available? It would be helpful for us to be able to repro this on our own machines.

[yajo]

Not exactly that code I showed above, but a pretty similar environment. Follow these steps:

1. Install these dependencies.
2. Clone https://gitlab.com/moduon/moduon-odoo-addons
3. Run invoke git-aggregate develop. It'll take a while.
4. When it finishes, you'll have a code workspace file dropped there. Open it in workspace mode.
5. Wait until all vscode extensions load.
6. Make sure pylance is enabled.
7. Open some python file from some addon.
8. Rename its folder.

[debonte]

Run invoke git-aggregate develop. It'll take a while.

This fails for me with the following errors. Any suggestions?:

(INFO) [03:47:19] git_aggregator.repo  l10n-spain   Cloning git repository https://github.com/moduon/l10n-spain.git in /opt/odoo/custom/src/l10n-spain
fatal: could not create work tree dir '/opt/odoo/custom/src/l10n-spain': Permission denied
(ERROR) [03:47:19] git_aggregator.repo  l10n-spain   /opt/odoo/custom/src/l10n-spain> error calling ('git', 'clone', '--filter=blob:none', '-b', '16.0', '--depth', '100', 'https://github.com/moduon/l10n-spain.git', '/opt/odoo/custom/src/l10n-spain')
fatal: could not create work tree dir '/opt/odoo/custom/src/odoo': Permission denied

[yajo]

You seem to be in a folder where your user doesn't have write permissions.

[debonte]

I'm not. I'm running it from within my home directory. The directory with permissions issues is under /opt, it's not relative to the directory where I'm running invoke.

I tried running invoke via sudo but theinvoke command is not recognized in that context. I tried pre-creating /opt/odoo via sudo and granting my user access to it. That didn't change anything. I also tried running several other invoke commands that I found in the odoo getting started docs, thinking that they would be able to properly initialize this directory, but they did not.

[yajo]

Are you running podman or docker? If it's podman you probably need to follow these steps: https://github.com/Tecnativa/doodba-copier-template/blob/main/docs/faq.md#how-to-use-with-podman

[debonte]

Are you running podman or docker?

No, I'm using a physical Linux (Ubuntu) machine.

[yajo]

No, I mean that which of those you have installed (and configured) in your host machine. You need one of them to reproduce the problem.

[debonte]

No, I mean that which of those you have installed (and configured) in your host machine. You need one of them to reproduce the problem.

I see. I have Docker installed because that's what the dependency list you pointed me to told me to install. But the file system error I'm seeing is about the file system on my host machine, not a Docker container.

From the instructions you provided above, it's not clear to me when Docker is supposed to come into play.

[yajo]

When you run invoke git-aggregate develop, you're using invoke as a task manager. One of the things it does is to download git code for development, using a specific docker-compose environment called setup-devel.yaml.

The error message means that the tool that runs within the container and downloads the code (git-aggregator) is not able to clone the repos into /opt/odoo/custom/, which is a container path that mounts your host's ./odoo/custom/.

It seems there's some kind of specific permissions issue. Sometimes it's related to different UID in the container and in the host (although the task manager is supposed to fix that automatically). It could be related to some SELinux/AppArmor policies or labeling... Try running invoke develop git-aggregate develop instead (sometimes prefixing the develop phase fixes permission issues).

If it doesn't work, we could schedule a videocall and I could help you with your environment. I'm sad that it's getting so hard for you to reproduce 😕. Thanks for your patience!

[debonte]

Still no luck.

I saw some docs online that seemed to say that the ./odoo/custom directory needed to be owned by the odoo group, but there is no such group on my host machine. Do I need to install odoo components on the host machine before running invoke git-aggregate?

we could schedule a videocall and I could help you with your environment.

That would be great! Please contact me (erikd at microsoft.com) and we can set up a time.

[szhang963]

Hi, @debonte. I also have the same problem. I think it is not related to any projects. It may occur at any project if you enable the pylance extension. I think the 'pylance' wants to rename all names in import for all python files, so it may search all python files and result in a slow rename operation. Therefore, how to turn off the function for pylance only?

[heejaechang]

currently, there is no turn off option, but if it takes more than a few seconds, message box will pop up with cancellation button, you can click it to cancel the operation. if cancelled, it will do only the regular rename, not semantic rename.

[szhang963]

@heejaechang, thanks. Changing it by setting the setting.json may be a better way.

[heejaechang]

@luabud how do you think?

[debonte]

@yajo, in gigantic workspaces, file or folder rename can be slow because we need to search every file for possible imports of the renamed file/folder. As pointed out above, when the "Running File Rename participants" popup appears, you can press "Cancel" to give up on semantic rename and simply rename the file.

You could also try adjusting your python.analysis.include/python.analysis.exclude settings to focus Pylance's analysis on a subset of the files in your workspace.

The suggestion above is to introduce an option to disable semantic support for file/folder renaming. That would make the rename fast, but you'd lose automatic updating of affected imports. What are your thoughts on that approach as a work around for people with gigantic workspaces?

[yajo]

I have no idea about how Pylance works so I might be saying something stupid here, but IMHO there should be some caching that stores relevant tokens in a format that lets pylance find quickly which files to rename. Otherwise we'd have the same problem for any refactor.

That should help too with time-to-ready when switching workspaces.

In any case, having a workaround would be good too.

[heejaechang]

@yajo it could be improved, but we already do, but still it depends and still cost will be relative to the size of workspace.