Excessive memory usage on autocomplete import

[rchen152]

See comment thread starting at https://github.com/facebook/pyrefly/issues/1521#issuecomment-3514890906 for more context.

I can reproduce this with the following:

• pip list with lots of dependencies

(test) PS C:\Users\kylei\projects\test> uv pip list
Package                  Version
------------------------ -----------
boto3                    1.37.38
botocore                 1.37.38
filelock                 3.16.1
fsspec                   2025.3.0
googleapis-common-protos 1.72.0
grpcio                   1.70.0
grpcio-status            1.70.0
jinja2                   3.1.6
jmespath                 1.0.1
markupsafe               2.1.5
mpmath                   1.3.0
networkx                 3.1
numpy                    1.24.4
pandas                   2.0.3
protobuf                 5.29.5
python-dateutil          2.9.0.post0
pytz                     2025.2
s3transfer               0.11.5
setuptools               75.3.2
six                      1.17.0
sympy                    1.13.3
torch                    2.4.1
typing-extensions        4.13.2
tzdata                   2025.2
urllib3                  1.26.20

• open empty file, no ram (<130mb)
• autocomplete import
• ram goes to 1.2gb

Originally posted by @kinto0 in #1521

[rchen152]

I was able to reproduce this. Interestingly, when I have another (not opened) file in my workspace that contains import torch, memory goes up as soon as I open an empty file. But if the empty file is the only thing in my workspace, nothing happens until I try to get completions.

My suspicion is that this is just our background indexing, so I'm not sure what to do about it, but I'll poke at it some more.

[kinto0]

when I have another (not opened) file in my workspace that contains import torch, memory goes up as soon as I open an empty file

do you know if the memory use goes up related to size of torch? or size of all of site-packages?

That might be another issue. This issue might be solved by a small optimization of the auto-import logic, since I imagine workspaces don't always import everything.

[rchen152]

I think it's size of all of site-packages, because the amount of memory consumed went up when I installed more packages.

[github-actions[bot]]

This issue has someone assigned, but has not had recent activity for more than 2 weeks.

If you are still working on this issue, please add a comment so everyone knows. Otherwise, please unassign yourself and allow someone else to take over.

Thank you for your contributions!