pyrefly gets really slow when run in environment with a lot of dependencies

[hauntsaninja]

Describe the Bug

I have a small 3k line project with not many dependencies

If I install its dependencies in a fresh environment, pyrefly runs in about 200ms:

 INFO 17 errors shown, 4 errors ignored, 19 modules, 485 transitive dependencies, 143,623 lines, took 0.20s, peak memory physical 130.6 MiB

However, if I run in a large monolithic Python virtual environment with over a thousand installs, pyrefly runs in ~30s:

 INFO 17 errors shown, 4 errors ignored, 19 modules, 17,920 transitive dependencies, 8,128,708 lines, took 26.33s, peak memory physical 1.9 GiB

The project is in a closed source repo, but I will try to see if I can make an open source reproduction.

In the meantime, is there anything I can do to figure out what the extra transitive dependencies pyrefly is checking are? --verbose doesn't seem to make a difference

pyrefly 0.16.3

[lolpack]

Hi @hauntsaninja thanks for reaching. I wonder if the issue is ignoring the correct files? We have tasks for https://github.com/facebook/pyrefly/issues/319 and https://github.com/facebook/pyrefly/issues/321. I wonder if you have generated artifacts or binaries that are getting pulled into to analysis that should not be.

[hauntsaninja]

By "environment" I mean Python virtual environment. It was the same project checkout. From the output attached, pyrefly is discovering the same number of modules to check, just the number of transitive dependencies explodes.

Is there a way to get insight into what pyrefly is checking? --verbose was not revealing

[lolpack]

Can you try excluding the venv directory with project_excludes https://pyrefly.org/en/docs/configuration/#project_excludes ? @connernilsen is there a way to output all of the search paths that we pull in? I don't think there is a way to show the transitive dependency graph today.

[hauntsaninja]

I'll try and report back! But the venv location in my case is out of tree

[connernilsen]

@lolpack we have pyrefly dump-config which will tell us what we analyze for a given config. It enumerates project_includes, and lets us know the site_package_path (along with a few other things).

@hauntsaninja, is the project you're working on open source? Would we be able to take a look/try to replicate it ourselves or is that not possible?

Also, do you have a pyrefly.toml or [tool.pyrefly] section in a pyproject.toml? Can you upload that here if so?

[hauntsaninja]

I'd tried pyrefly dump-config and it confirmed that the "Covered files" number was small and the same in both cases

No, not open source and no special pyrefly config. I'll try to make an open source repro, should be very easy, my guess is installing a random thousand packages and checking some small project would be enough to repro.

[hauntsaninja]

Try cloning https://github.com/hauntsaninja/pyp

If you make a new virtual environment and run uv pip install -e .; uv pip install pyrefly, you get:

λ hyperfine -w1 -i 'pyrefly check .'
Benchmark 1: pyrefly check .
  Time (mean ± σ):     112.7 ms ±   4.3 ms    [User: 189.4 ms, System: 55.2 ms]
  Range (min … max):   104.9 ms … 123.3 ms    26 runs

If you then install a thousand unrelated packages from PyPI, you get:

λ hyperfine -w1 -i 'pyrefly check .'
Benchmark 1: pyrefly check .
  Time (mean ± σ):     225.0 ms ±   6.2 ms    [User: 868.4 ms, System: 108.9 ms]
  Range (min … max):   216.3 ms … 234.4 ms    12 runs

I also think there might separately be an issue where if your sys.path is very long, you get additional slowness (mypy had a similar issue that I fixed a few releases ago)

[connernilsen]

Hey @hauntsaninja, I'm coming back around to this, but I'm oncall for our team this week and got stuck handling a few big internal issues that popped up. I'll be focusing on this more tomorrow or early next week.

[connernilsen]

Okay, taking a look again! (Also, pyp is a cool project! I'm going to start using it myself 😁)

Quick question which packages did you install for the thousand unrelated packages? I want to make sure it's not a specific package that's not causing it. Can you upload an equivalent of a requirements.txt here?

[hauntsaninja]

Yeah, sorry for being coy, just wanted to err on the side of not sharing too much about the specific environment. Here is a requirements.txt of 800ish packages you can use (filtered so that all packages in here are top 5000 most downloaded on PyPI): https://pastebin.com/raw/5nBviuGM

It looks like with pyrefly==0.18.0 and this environment, pyrefly crashes on my machine and is at least 500x slower. (Maybe a good thing, the 2x above on pypyp is still quite different from the 150x I saw on the closed source project (which is small but not quite as small as pypyp))

(env) ~/dev/pyp master λ uv venv env                          
Using CPython 3.12.9
Creating virtual environment at: env
Activate with: source env/bin/activate

(env) ~/dev/pyp master λ source env/bin/activate

(env) ~/dev/pyp master λ uv pip install pyrefly               
Using Python 3.12.9 environment at: env
Resolved 1 package in 4ms
Installed 1 package in 2ms
 + pyrefly==0.18.0

(env) ~/dev/pyp master λ hyperfine -w3 -i 'pyrefly check .'
Benchmark 1: pyrefly check .
  Time (mean ± σ):     126.7 ms ±  12.3 ms    [User: 202.9 ms, System: 54.1 ms]
  Range (min … max):   113.7 ms … 174.4 ms    22 runs
 
  Warning: Ignoring non-zero exit code.
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

(env) ~/dev/pyp master λ uv pip install -r r.txt
... 

(env) ~/dev/pyp master λ time pyrefly check .
[00:00:42] █████████████████████████████████████████████████████████████████████████████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░   97690/154705 
thread '<unknown>' has overflowed its stack
fatal runtime error: stack overflow
zsh: abort      pyrefly check .
pyrefly check .  301.67s user 22.13s system 684% cpu 47.313 total

[connernilsen]

@hauntsaninja no problem, I get that you're trying to keep things private for your use case 😄. In the future, if you would prefer, we can also send semi-sensitive stuff like that over direct messages in Discord.

Also thank you so much for sending this! I'm seeing the issue when I create the venv in the same directory, but not when it's outside of the project (i.e. wouldn't be included by the path we're checking, which is .). I know you mentioned that was an issue for you above, are you still seeing that?

Otherwise, I think part of this is how we implemented file path filtering with project-excludes, since we list all files first then filter them out if they match the excludes glob. I can look into how we're doing that to perform an exclude check more aggressively.

I also might try to do something where we automatically include site-package-path entries as project-excludes to make sure we're not enumerating those on accident in some cases.

[connernilsen]

Alright, I have two PRs that should fix different parts of this:

• #439 should make sure your site-package-path is always included in project-excludes. I'm still trying to figure out if this is something we should be doing by default. #319 should accomplish something similar, so this PR might not actually be needed.
• #440 is the main fix here, which fixes our globbing logic to not naively enumerate all files from a project-includes, then filter them afterward, and instead applies filtering as we go.

I tested this on the project setup from above, and it brought down from ~10-20s to ~1s.

[hauntsaninja]

Oh oops, sorry, I forgot between yesterday and two weeks ago about keeping the env in the same directory.

But I can reproduce a 5x slowdown with the env in a separate directory (with r.txt as linked above):

~/dev/pyp master λ uv venv ~/tmp/env
Using CPython 3.12.9
Creating virtual environment at: /Users/shantanu/tmp/env
Activate with: source /Users/shantanu/tmp/env/bin/activate

~/dev/pyp master λ source ~/tmp/env/bin/activate               

(env) ~/dev/pyp master λ uv pip install pyrefly==0.18  
Using Python 3.12.9 environment at: /Users/shantanu/tmp/env
Resolved 1 package in 148ms
Installed 1 package in 3ms
 + pyrefly==0.18.0

(env) ~/dev/pyp master λ hyperfine -w3 -i 'pyrefly check .'
Benchmark 1: pyrefly check .
  Time (mean ± σ):     119.8 ms ±   5.4 ms    [User: 197.0 ms, System: 56.2 ms]
  Range (min … max):   106.3 ms … 130.3 ms    23 runs
 
  Warning: Ignoring non-zero exit code.
 
(env) ~/dev/pyp master λ uv pip install -r ~/r.txt --quiet
uv pip install -r ~/code/openai/r.txt --quiet  0.38s user 4.72s system 102% cpu 4.974 total

(env) ~/dev/pyp master λ hyperfine -w3 -i 'pyrefly check .'           
Benchmark 1: pyrefly check .
  Time (mean ± σ):     627.0 ms ±  17.9 ms    [User: 2553.6 ms, System: 147.9 ms]
  Range (min … max):   609.6 ms … 661.1 ms    10 runs
 
  Warning: Ignoring non-zero exit code.

[connernilsen]

I'm hoping some of the changes I added above will help, but just to double check some stuff, can you run pyrefly check . --verbose, and add a couple lines of the log here?

I'm mainly looking for these two statements:

One of the first lines detailing the config:

DEBUG Config is: {project_includes: [/Users/connernilsen/Documents/typecheck/pyp/**/*], project_excludes: [/Users/connernilsen/Documents/typecheck/pyp/**/.[!/.]*, /Users/connernilsen/Documents/typecheck/pyp/**/*venv/**/*], search_path: [/Users/connernilsen/Documents/typecheck/pyp/], python_interpreter: None, python_environment: {python_platform: darwin, python_version: 3.12.9, site_package_path: [..., /Users/connernilsen/Documents/typecheck/pyp], site_package_path_source: Interpreter("/Users/connernilsen/Documents/typecheck/pyp/venv/bin/python3")}, replace_imports_with_any: []}

and the last line, which should look like this:

 INFO 6 errors shown, 9 errors ignored, 4 modules, 498 transitive dependencies, 156,328 lines, took 52.30s (listing 51.38s; checking 0.91s), peak memory physical 236.8 MiB

I saw some interesting stuff from my venv (other site package paths being included in that, probably just from some weird stuff I was trying to replicate this problem), so I'm curious if that may also be happening here. If it is, then it's possible the PRs above will help here

[hauntsaninja]

I don't get the DEBUG config log line:

~/dev/pyp master λ uv venv ~/tmp/env
Using CPython 3.12.9
Creating virtual environment at: /Users/shantanu/tmp/env
Activate with: source /Users/shantanu/tmp/env/bin/activate

~/dev/pyp master λ source ~/tmp/env/bin/activate

(env) ~/dev/pyp master λ uv pip install pyrefly==0.18                 
Using Python 3.12.9 environment at: /Users/shantanu/tmp/env
Resolved 1 package in 37ms
Installed 1 package in 2ms
 + pyrefly==0.18.0

(env) ~/dev/pyp master λ pyrefly check . --verbose
DEBUG Checking 4 files (listing took 0.07s)
DEBUG Running with 10 threads
[00:00:00] ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░       0/0      DEBUG Running epoch 1
ERROR /Users/shantanu/dev/pyp/pyp.py:264:37-70: Argument `(line: LiteralString) -> bool` is not assignable to parameter with type `(str) -> object` in function `itertools.takewhile.__new__` [bad-argument-type]
ERROR /Users/shantanu/dev/pyp/pyp.py:270:20-47: Expected class object, got `tuple[*tuple[Any, ...], type[Import], type[ImportFrom], type[Assign], type[AnnAssign], type[If], type[Try]]` [invalid-argument]
ERROR /Users/shantanu/dev/pyp/pyp.py:429:34-47: Object of class `stmt` has no attribute `body` [missing-attribute]
ERROR /Users/shantanu/dev/pyp/pyp.py:432:27-41: Object of class `stmt` has no attribute `value` [missing-attribute]
ERROR /Users/shantanu/dev/pyp/pyp.py:433:26-40: Object of class `stmt` has no attribute `value` [missing-attribute]
ERROR /Users/shantanu/dev/pyp/pyp.py:439:75-89: Object of class `stmt` has no attribute `value` [missing-attribute]
ERROR /Users/shantanu/dev/pyp/tests/test_find_names.py:7:8-14: Could not find import of `pytest`, looked in these locations (from default config for project root marked by `/Users/shantanu/dev/pyp/pyproject.toml`):
  Import root (inferred from project layout): "/Users/shantanu/dev/pyp/"
  Site package path (queried from interpreter at `/Users/shantanu/tmp/env/bin/python3`): ["/Users/shantanu/Library/Application Support/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12", "/Users/shantanu/Library/Application Support/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/lib-dynload", "/Users/shantanu/tmp/env/lib/python3.12/site-packages"] [import-error]
ERROR /Users/shantanu/dev/pyp/tests/test_pyp.py:14:8-14: Could not find import of `pytest`, looked in these locations (from default config for project root marked by `/Users/shantanu/dev/pyp/pyproject.toml`):
  Import root (inferred from project layout): "/Users/shantanu/dev/pyp/"
  Site package path (queried from interpreter at `/Users/shantanu/tmp/env/bin/python3`): ["/Users/shantanu/Library/Application Support/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12", "/Users/shantanu/Library/Application Support/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/lib-dynload", "/Users/shantanu/tmp/env/lib/python3.12/site-packages"] [import-error]
 INFO 8 errors shown, 9 errors ignored, 4 modules, 69 transitive dependencies, 22,775 lines, took 0.15s, peak memory physical 79.9 MiB

(env) ~/dev/pyp master λ hyperfine -w3 -i 'pyrefly check .'
Benchmark 1: pyrefly check .
  Time (mean ± σ):     119.5 ms ±   4.8 ms    [User: 197.8 ms, System: 54.6 ms]
  Range (min … max):   110.1 ms … 128.3 ms    23 runs
 
  Warning: Ignoring non-zero exit code.
 
(env) ~/dev/pyp master λ uv pip install -r ~/code/openai/r.txt --quiet
uv pip install -r ~/code/openai/r.txt --quiet  0.33s user 4.36s system 96% cpu 4.870 total

(env) ~/dev/pyp master λ pyrefly check . --verbose
DEBUG Checking 4 files (listing took 0.03s)
DEBUG Running with 10 threads
[00:00:00] ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░       0/0      DEBUG Running epoch 1
ERROR /Users/shantanu/dev/pyp/pyp.py:264:37-70: Argument `(line: LiteralString) -> bool` is not assignable to parameter with type `(str) -> object` in function `itertools.takewhile.__new__` [bad-argument-type]
ERROR /Users/shantanu/dev/pyp/pyp.py:270:20-47: Expected class object, got `tuple[*tuple[Any, ...], type[Import], type[ImportFrom], type[Assign], type[AnnAssign], type[If], type[Try]]` [invalid-argument]
ERROR /Users/shantanu/dev/pyp/pyp.py:429:34-47: Object of class `stmt` has no attribute `body` [missing-attribute]
ERROR /Users/shantanu/dev/pyp/pyp.py:432:27-41: Object of class `stmt` has no attribute `value` [missing-attribute]
ERROR /Users/shantanu/dev/pyp/pyp.py:433:26-40: Object of class `stmt` has no attribute `value` [missing-attribute]
ERROR /Users/shantanu/dev/pyp/pyp.py:439:75-89: Object of class `stmt` has no attribute `value` [missing-attribute]
 INFO 6 errors shown, 9 errors ignored, 4 modules, 499 transitive dependencies, 160,913 lines, took 0.60s, peak memory physical 204.6 MiB
pyrefly check . --verbose  2.47s user 0.16s system 430% cpu 0.612 total

(env) ~/dev/pyp master λ hyperfine -w3 -i 'pyrefly check .'
Benchmark 1: pyrefly check .
  Time (mean ± σ):     663.6 ms ±  82.3 ms    [User: 2650.0 ms, System: 172.3 ms]
  Range (min … max):   610.1 ms … 888.4 ms    10 runs
 
  Warning: Ignoring non-zero exit code.
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

[hauntsaninja]

Hmm but this might be a red herring because we now install pytest (also 5x slowdown is still less than the 150x slowdown in my closed source case where the number of diagnostics matched). Let me try with --no-deps to make sure pytest isn't included...

[hauntsaninja]

Ugh, yeah, it is just pytest. Sorry for the noise :-(

Let me go back to my closed source repro and see if I can find something shareable...

[connernilsen]

@hauntsaninja got it. Maybe test it out once I get #440 landed too to see if that addresses any of the issues with it, especially with #439 committed.

Note: #439 and other --project-excludes stuff won't do much without #440 also landed, since it fixes a big issue in how we were doing excludes previously. I'll do a release once both are landed to get this out quickly.

[hauntsaninja]

Okay, here's a shareable repro, building off the r.txt from earlier. Hopefully I haven't messed up.

Note this is maybe due to the large number of transitive dependencies pyrefly discovers, but I'm not sure why pyrefly is discovering these many transitive dependencies...

Script:

rm -rf proj
mkdir proj
cd proj

curl https://pastebin.com/raw/5nBviuGM > r.txt  # r.txt from previously in this issue
echo 'import watchfiles' > script.py

rm -rf ~/tmp/env
uv venv ~/tmp/env --python 3.12

source ~/tmp/env/bin/activate
uv pip install -r r.txt
uv pip install pyrefly==0.19

time pyrefly check --verbose script.py -j 8

Output:

DEBUG Checking 1 files (listing took 0.00s)
DEBUG Running with 8 threads
[00:00:00] ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░       0/0      DEBUG Running epoch 1
 INFO errors shown: 0, errors ignored: 0, modules: 1, transitive dependencies: 19,097, lines: 8,713,007, time: 17.00s, peak memory: physical 2.1 GiB

real	0m17.109s
user	2m1.011s
sys	0m4.745s

Perhaps interestingly, if you uninstall Pytorch, the time goes down from 17s to about 6s

watchfiles of course only has one dependency officially. Maybe if you go through all the delayed imports you somehow find your way to Pytorch and 19k deps? Note mypy and pyright are able to type check import watchfiles fast.

[connernilsen]

@hauntsaninja Alright, I just released version 0.19.1, which might fix some of the problems you're seeing (especially when the venv is in the same directory as the rest of your project, our filtering logic is better).

Let me know if you're still seeing issues, and I'll keep looking into it tomorrow.

[connernilsen]

@hauntsaninja If you try setting untyped-def-behavior to skip-and-infer-return-any, does that speed things up for you?

[hauntsaninja]

@connernilsen the repro I shared in https://github.com/facebook/pyrefly/issues/360#issuecomment-2961409433 is still slow for me with pyrefly==0.21

I still see it take 15s and have way more transitive deps than I naively expect. Were you able to reproduce?

untyped-def-behavior = skip-and-infer-return-any does make it slightly faster, but is still way slower than I would expect and is many times slower than other type checkers

(sorry for slowness, was traveling!)

[github-actions[bot]]

This issue has someone assigned, but has not had recent activity for more than 2 weeks.

If you are still working on this issue, please add a comment so everyone knows. Otherwise, please unassign yourself and allow someone else to take over.

Thank you for your contributions!

[connernilsen]

Sorry about the long delay, was busy with performance cycle at work and I forgot to come back to this. I just tried reproing with your more recent setup, and I'm seeing checks take around ~1s.

I'm currently seeing big improvements in a recent build of Pyrefly (hyperfine shows a few ms on main, while ~1s on v0.21-0.23, also seeing a drop in the number of transitive modules). I think I may have landed a bug fix today that may help here (i.e. if you don't have a pyrefly.toml or pyproject.toml, we might not find an active venv in some cases).

I'm going to put up a new Pyrefly version with the bug fixes right now, and then I'll send a few things to test out over. If this doesn't work, then I'll pull in others to see if we can figure out why we're having trouble getting a good repro.

[hauntsaninja]

Interesting. As you can see in my repro above, I don't have a pyrefly.toml or pyproject.toml.

I do see a noticeable improvement going from pyrefly 0.19 to 0.23, but it is still slow. I do not see a change in transitive dependencies:

0.19

(env) ~/tmp/proj λ uv pip install pyrefly==0.19
Using Python 3.12.9 environment at: /Users/shantanu/tmp/env
Resolved 1 package in 10ms
Installed 1 package in 3ms
 + pyrefly==0.19.0

(env) ~/tmp/proj λ time pyrefly check --verbose script.py -j 8
DEBUG Checking 1 files (listing took 0.00s)
DEBUG Running with 8 threads
[00:00:00] ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░       0/0      DEBUG Running epoch 1
 INFO errors shown: 0, errors ignored: 0, modules: 1, transitive dependencies: 19,134, lines: 8,755,553, time: 17.19s, peak memory: physical 2.1 GiB
pyrefly check --verbose script.py -j 8  109.31s user 4.72s system 651% cpu 17.494 total

(env) ~/tmp/proj λ hyperfine -w 1 -M 3 'pyrefly check --verbose script.py -j 8'
Benchmark 1: pyrefly check --verbose script.py -j 8
  Time (mean ± σ):     17.959 s ±  1.501 s    [User: 114.073 s, System: 4.799 s]
  Range (min … max):   16.905 s … 19.678 s    3 runs

0.23

(env) ~/tmp/proj λ uv pip install pyrefly==0.23
Using Python 3.12.9 environment at: /Users/shantanu/tmp/env
Resolved 1 package in 1ms
Installed 1 package in 2ms
 + pyrefly==0.23.0

(env) ~/tmp/proj λ time pyrefly check --verbose script.py -j 8
DEBUG Checking 1 files (listing took 0.00s)
DEBUG Running with 8 threads (5 MiB stack size)
[00:00:00] ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░       0/0      DEBUG Running epoch 1
 INFO errors shown: 0, errors ignored: 0, modules: 1, transitive dependencies: 19,134, lines: 8,756,230, time: 11.44s, peak memory: physical 2 GiB
pyrefly check --verbose script.py -j 8  72.45s user 3.62s system 643% cpu 11.825 total

(env) ~/tmp/proj λ hyperfine -w 1 -M 3 'pyrefly check --verbose script.py -j 8'
Benchmark 1: pyrefly check --verbose script.py -j 8
  Time (mean ± σ):     11.565 s ±  0.083 s    [User: 71.134 s, System: 3.661 s]
  Range (min … max):   11.469 s … 11.622 s    3 runs

[hauntsaninja]

I've reproduced on both my Macbook and a Linux box with the script from https://github.com/facebook/pyrefly/issues/360#issuecomment-2961409433

Is there a way to get output that would help figure out what the difference between our setups are? But either way I will try the new release! :-)

[connernilsen]

Got it, I changed some steps in the script (venv in current directorY), thinking it wouldn't cause problems, so I'll retry using exactly what you have.

I'm still waiting for the Pyrefly patch release to go through, so you can test out with that once it's done.

One thing that might be useful is to run pyrefly dump-config script.py to output some info about your config, but 0.23.1 will have an extra part of the output that will be useful to have here. I'll update you once it goes through (I expect within the next hour), and then we can continue debugging.

[connernilsen]

Oh wow, I'm seeing it now. It seems to be related to having the venv located outside of the project directory, but I'm not yet sure why that would be related to how we're finding imports. I'll continue digging into this to see what the problem is.

Thanks for your patience with me through all of this!

[hauntsaninja]

Yay, glad you have a repro! 0.23.1 is still slow for me, in case still useful here is my dump-config:

(env) ~/tmp/proj λ pyrefly --version
pyrefly 0.23.1

(env) ~/tmp/proj λ pyrefly dump-config script.py
Default configuration
  Using interpreter: /Users/shantanu/tmp/env/bin/python3
  Covered files:
    /Users/shantanu/tmp/proj/script.py
  Fallback search path (guessed from project_includes): ["/Users/shantanu/tmp/proj", "/Users/shantanu/tmp", "/Users/shantanu", "/Users", "/"]
  Site package path queried from interpreter: ["/Users/shantanu/Library/Application Support/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12", "/Users/shantanu/Library/Application Support/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/lib-dynload", "/Users/shantanu/tmp/env/lib/python3.12/site-packages"]

(env) ~/tmp/proj λ time pyrefly check --verbose script.py -j 8
DEBUG Checking 1 files (listing took 0.00s)
DEBUG Running with 8 threads (5 MiB stack size)
[00:00:00] ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░       0/0      DEBUG Running epoch 1
 INFO errors shown: 0, errors ignored: 0, modules: 1, transitive dependencies: 19,134, lines: 8,756,230, time: 11.79s, peak memory: physical 2 GiB
pyrefly check --verbose script.py -j 8  73.28s user 3.90s system 650% cpu 11.856 total

[connernilsen]

My current working theory is that we keep re-querying the interpreter a ton when the venv is outside of the directory for some reason. I need to double check why that would be happening (especially when we're not in LSP watch mode), since I'm not sure at the moment why that would be happening.