pyrefly icon indicating copy to clipboard operation
pyrefly copied to clipboard

Pyrefly Cannot Correctly Figure Out Dependencies

Open Notenlish opened this issue 7 months ago • 2 comments

Describe the Bug

(.venv) noten@noten:~/Documents/GitHub/hukuksuzlugun-bedeli$ ls
backend.sh  frontend.sh  pyproject.toml      README.md
backend   frontend    LICENSE      pyrightconfig.json

In my project, the backend module contains python code. Pyrefly cannot detect that I am using a virtualenv and therefore gives an error when I run pyrefly check, as it thinks I haven't installed those modules. My venv directory is .venv/.

I already have a pyproject.toml setup:

[tool.pyrefly]
search_path = ["backend/"]  # analyze these directories
#python_executable = ".venv/bin/python3"
# site_package_path=[".venv/lib/python3.12/site-packages"]
# python_interpreter=".venv/bin/python3"

I tried commenting and uncommenting the site_package_path and python_interpreter, it still doesn't work. I am guessing I am missing something, what am I doing wrong?

(.venv) noten@noten:~/Documents/GitHub/hukuksuzlugun-bedeli$ pyrefly check
 INFO Checking project configured at `/home/noten/Documents/GitHub/hukuksuzlugun-bedeli/pyproject.toml`
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/actions.py:2:21-27: Could not import `bcrypt` from `passlib` [missing-module-attribute]
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/evdsclient.py:10:25-42: No attribute `adapters` in module `requests` [missing-attribute]
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/evdsclient.py:17:14-25: Attribute `poolmanager` is implicitly defined by assignment in method `init_poolmanager`, which is not a constructor [implicitly-defined-attribute]
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/evdsclient.py:55:14-21: Attribute `session` is implicitly defined by assignment in method `__create_session`, which is not a constructor [implicitly-defined-attribute]
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/evdsclient.py:78:13-46: Item assignment is not supported on `DataFrame | Series | Unknown`
  Expected `__setitem__` to be a callable, got BoundMethod[DataFrame, (self: Self@DataFrame, key: Unknown, value: Unknown) -> None] | BoundMethod[Series, (self: Self@Series, key: Unknown, value: Unknown) -> None] | Unknown [not-callable]
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/evdsclient.py:103:33-76: Object of class `ndarray` has no attribute `to_list` [missing-attribute]
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/fetcher.py:208:27-58: Expected 0 type arguments for `FrequencyEVDS`, got 1 [bad-specialization]
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/fetcher.py:208:41-57: Expected a type form, got instance of `FrequencyEnum` [not-a-type]
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/main.py:53:5-23: Missing argument `self` in function `functools._Wrapped.__call__` [missing-argument]
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/main.py:60:1-19: Expected `Ellipsis` to be a ParamSpec value in function `starlette.applications.Starlette.add_middleware` [bad-argument-type]
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/main.py:61:5-19: Argument `type[CORSMiddleware]` is not assignable to parameter with type `_MiddlewareFactory[@_]` in function `starlette.applications.Starlette.add_middleware` [bad-argument-type]
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/main.py:227:33-56: Item assignment is not supported on `dict[str, str]`
  Argument `list[@_]` is not assignable to parameter with type `str` in function `dict.__setitem__` [bad-argument-type]
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/main.py:240:21-48: Item assignment is not supported on `dict[str, dict[str, list[Unknown]]]`
  Argument `dict[str, str]` is not assignable to parameter with type `dict[str, list[Unknown]]` in function `dict.__setitem__` [bad-argument-type]
ERROR /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/models.py:89:11-15: Class member `MetricDataPoint.Meta` overrides parent class `Model` in an inconsistent manner [bad-override]
 INFO 14 errors shown, 1 errors ignored, 15 modules, 1,610 transitive dependencies, 753,673 lines, took 1.76s (checking 1.06s; listing 0.69s), peak memory physical 494.4 MiB

If it helps, I am on Linux Mint 22.1 with Zed editor, however I don't think it's something related to Zed.

Sandbox Link

No response

(Only applicable for extension issues) IDE Information

No response

Notenlish avatar May 25 '25 17:05 Notenlish

Same issue here when tesing on Windows 11 with a uv created project

anzhi0708 avatar May 27 '25 06:05 anzhi0708

Hey @Notenlish and @anzhi0708, sorry, but I was out for part of last week, and I'm still catching up with stuff from this week. I'll get around to looking at this tomorrow or early next week. Just wanted to let you know that it's still in my queue!

connernilsen avatar May 29 '25 20:05 connernilsen

@Notenlish and @anzhi0708, sorry you're experiencing this issue. I'll try to help you get this working, and see if there are any bugs we need to fix in the process.

Can you both answer these questions for me?

  1. Just to double check, your venv is located at /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/.venv, right?
  2. What do you see when you ls .venv/lib/python3.12/site-packages?
  3. Do you have the virtual environment activated when you're running Pyrefly?
  4. Which version of Pyrefly are you running (pyrefly --version)?

Also @anzhi0708, are you running Pyrefly on cmd, powershell, linux subsystem, or something else?

connernilsen avatar Jun 03 '25 16:06 connernilsen

  1. yes
  2. output:
noten@noten:~/Documents/GitHub/hukuksuzlugun-bedeli$ ls .venv/lib/python3.12/site-packages
aerich                              colorama-0.4.6.dist-info         MarkupSafe-3.0.2.dist-info             rich_toolkit
aerich-0.9.0.dist-info              cv2                              mdurl                                  rich_toolkit-0.14.6.dist-info
aiofiles                            dateutil                         mdurl-0.1.2.dist-info                  ruff
aiofiles-24.1.0.dist-info           dictdiffer                       multidict                              ruff-0.11.9.dist-info
aiohappyeyeballs                    dictdiffer-0.9.0.dist-info       multidict-6.4.4.dist-info              shellingham
aiohappyeyeballs-2.6.1.dist-info    dns                              multipart                              shellingham-1.5.4.dist-info
aiohttp                             dnspython-2.7.0.dist-info        numpy                                  six-1.17.0.dist-info
aiohttp-3.11.18.dist-info           dotenv                           numpy-2.2.5.dist-info                  six.py
aioredis                            dotenv-0.9.9.dist-info           numpy.libs                             sniffio
aioredis-2.0.1.dist-info            email_validator                  opencv_python-4.11.0.86.dist-info      sniffio-1.3.1.dist-info
aiosignal                           email_validator-2.2.0.dist-info  opencv_python.libs                     soupsieve
aiosignal-1.3.2.dist-info           evds                             pandas                                 soupsieve-2.7.dist-info
aiosqlite                           evds-0.3.2.dist-info             pandas-2.2.3.dist-info                 sqlalchemy
aiosqlite-0.21.0.dist-info          fastapi                          passlib                                sqlalchemy-2.0.40.dist-info
alembic                             fastapi-0.115.12.dist-info       passlib-1.7.4.dist-info                starlette
alembic-1.15.2.dist-info            fastapi_admin                    pendulum                               starlette-0.46.2.dist-info
annotated_types                     fastapi_admin-1.0.4.dist-info    pendulum-3.1.0.dist-info               tomlkit
annotated_types-0.7.0.dist-info     fastapi_cli                      pip                                    tomlkit-0.13.2.dist-info
anyio                               fastapi_cli-0.0.7.dist-info      pip-24.0.dist-info                     tortoise
anyio-4.9.0.dist-info               frozenlist                       propcache                              tortoise_orm-0.25.0.dist-info
apscheduler                         frozenlist-1.6.0.dist-info       propcache-0.3.1.dist-info              typer
APScheduler-3.11.0.dist-info        greenlet                         psycopg2                               typer-0.15.4.dist-info
asyncclick                          greenlet-3.2.1.dist-info         psycopg2_binary-2.9.10.dist-info       typing_extensions-4.13.2.dist-info
asyncclick-8.1.8.0.dist-info        h11                              psycopg2_binary.libs                   typing_extensions.py
asyncpg                             h11-0.16.0.dist-info             __pycache__                            typing_inspection
asyncpg-0.30.0.dist-info            httpcore                         pydantic                               typing_inspection-0.4.0.dist-info
async_timeout                       httpcore-1.0.9.dist-info         pydantic-2.11.4.dist-info              tzdata
async_timeout-5.0.1.dist-info       httptools                        pydantic_core                          tzdata-2025.2.dist-info
attr                                httptools-0.6.4.dist-info        pydantic_core-2.33.2.dist-info         tzlocal
attrs                               httpx                            pygments                               tzlocal-5.3.1.dist-info
attrs-25.3.0.dist-info              httpx-0.28.1.dist-info           pygments-2.19.1.dist-info              urllib3
babel                               idna                             pypika_tortoise                        urllib3-2.4.0.dist-info
babel-2.17.0.dist-info              idna-3.10.dist-info              pypika_tortoise-0.5.0.dist-info        uvicorn
bcrypt                              imutils                          python_dateutil-2.9.0.post0.dist-info  uvicorn-0.34.2.dist-info
bcrypt-4.3.0.dist-info              imutils-0.5.4.dist-info          python_dotenv-1.1.0.dist-info          uvloop
beautifulsoup4-4.13.4.dist-info     iso8601                          python_multipart                       uvloop-0.21.0.dist-info
bs4                                 iso8601-2.1.0.dist-info          python_multipart-0.0.20.dist-info      watchfiles
certifi                             jinja2                           pytz                                   watchfiles-1.0.5.dist-info
certifi-2025.4.26.dist-info         jinja2-3.1.6.dist-info           pytz-2025.2.dist-info                  websockets
CHANGELOG.md                        LICENSE                          PyYAML-6.0.2.dist-info                 websockets-15.0.1.dist-info
charset_normalizer                  mako                             README.md                              _yaml
charset_normalizer-3.4.2.dist-info  mako-1.3.10.dist-info            requests                               yaml
click                               markdown_it                      requests-2.32.3.dist-info              yarl
click-8.1.8.dist-info               markdown_it_py-3.0.0.dist-info   rich                                   yarl-1.20.0.dist-info
colorama                            markupsafe                       rich-14.0.0.dist-info
  1. Not really sure why, but when I start zed it uses an unrelated virtual environment. If I do source .venv/bin/activate then I can use the correct venv but I'm not sure if this applies to the process running pyrefly. Even If I run pyrefly manually from the terminal with the correct venv it still cannot figure out dependencies correctly and thinks that they're not installed.
  2. 0.16.2

Notenlish avatar Jun 04 '25 07:06 Notenlish

@Notenlish and @anzhi0708, sorry you're experiencing this issue. I'll try to help you get this working, and see if there are any bugs we need to fix in the process.

Can you both answer these questions for me?

  1. Just to double check, your venv is located at /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/.venv, right?
  2. What do you see when you ls .venv/lib/python3.12/site-packages?
  3. Do you have the virtual environment activated when you're running Pyrefly?
  4. Which version of Pyrefly are you running (pyrefly --version)?

Also @anzhi0708, are you running Pyrefly on cmd, powershell, linux subsystem, or something else?

Hey @connernilsen good evening!

All the dependencies for my project installed by uv are located in the .venv/lib/python3.13/site-packages directory, and I'm not in an activated virtual environment. That's because all my command-line operations within the project are executed via uv, such as uv run python -m myproject.mymodule, uv run main.py, uv run python, etc.

Additionally, I use pyrefly across all platforms, including Ubuntu, Windows 11, and Termux (Android).

Currently, the issue seems to be that unless I explicitly run pyrefly init . and manually declare tool.pyrefly.python_interpreter = "./.venv/bin/python" (on Linux) or ./.venv/Scripts/python.exe (on Windows) in pyproject.toml, pyrefly will fall back to using the system's default global Python interpreter and its dependencies.

anzhi0708 avatar Jun 04 '25 11:06 anzhi0708

unless I explicitly run pyrefly init . and manually declare tool.pyrefly.python_interpreter = "./.venv/bin/python" (on Linux) or ./.venv/Scripts/python.exe (on Windows) in pyproject.toml, pyrefly will fall back to using the system's default global Python interpreter and its dependencies.

By the way, the editor I use is helix. From my perspective, once I understand how to select the Python interpreter for pyrefly, it seems to come down to a matter of choice — that is, whether pyrefly should default to using the system-wide interpreter, or automatically detect at startup whether it's inside a likely virtual environment directory and default to using the interpreter provided by that environment.

anzhi0708 avatar Jun 04 '25 11:06 anzhi0708

This is strange, it seems like you both have things set up correctly and that we should be finding everything.

Can you both run pyrefly check --verbose and attach the first few lines of that output (before the errors start)? There should be a line that looks like DEBUG: Config is ..., which is what I'm looking for.

Once you have that, can you find site_package_path_source, and if it's something that looks like Interpreter(<interpreter>), run <interpreter> -c "import sys; print(sys.path)", and also attach that to your response?

connernilsen avatar Jun 04 '25 20:06 connernilsen

Project Structure

PS C:\Users\anzhi\dev\py\cbm_correct.py> ls


    Directory: C:\Users\anzhi\dev\py\cbm_correct.py


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----        2025-05-28   4:30 PM                .venv
d-----        2025-05-28   5:53 PM                build
-a----        2025-05-28   4:29 PM            109 .gitignore
-a----        2025-05-28   4:29 PM              5 .python-version
------        2025-05-28   4:32 PM           8861 main.py
------        2025-06-05   9:58 AM            396 pyproject.toml
-a----        2025-05-28   4:29 PM              0 README.md
-a----        2025-05-28   4:50 PM          27648 uv.lock


PS C:\Users\anzhi\dev\py\cbm_correct.py> hx .\main.py



"python_interpreter: None" At this point, I have commented out the [tool.pyrefly] block in pyproject.toml, so when I edit main.py in my text editor, the pyrefly LSP reports that it cannot find the dependencies. Judging from the output python_interpreter: None, it seems that pyrefly failed to detect my virtual environment.

``` PS C:\Users\anzhi\dev\py\cbm_correct.py> pyrefly check --verbose INFO Found `C:\Users\anzhi\dev\py\cbm_correct.py\pyproject.toml` marking project root, checking root directory with default configuration DEBUG Config is: {project_includes: [C:\Users\anzhi\dev\py\cbm_correct.py\**\*], project_excludes: [C:\Users\anzhi\dev\py\cbm_correct.py\**\.[!\.]*, C:\Users\anzhi\dev\py\cbm_correct.py\**\*venv\**\*], search_path: [C:\Users\anzhi\dev\py\cbm_correct.py\], python_interpreter: None, python_environment: {python_platform: win32, python_version: 3.13.3, site_package_path: [C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.13_3.13.1008.0_x64__qbz5n2kfra8p0\DLLs, C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.13_3.13.1008.0_x64__qbz5n2kfra8p0\Lib, C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.13_3.13.1008.0_x64__qbz5n2kfra8p0, C:\Users\anzhi\dev\py\cbm_correct.py\.venv, C:\Users\anzhi\dev\py\cbm_correct.py\.venv\Lib\site-packages], site_package_path_source: Interpreter("C:\\Users\\anzhi\\dev\\py\\cbm_correct.py\\.venv\\Scripts\\python.exe")}, replace_imports_with_any: []} DEBUG Checking 3 files (listing took 3.00s) DEBUG Running with 4 threads [00:00:00] ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 0/0 DEBUG Running epoch 1 ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:14:12-17: No matching overload found for function `round`, reporting errors for closest overload: `(number: _SupportsRound1[@_], ndigits: None = ...) -> @_` [no-matching-overload] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:14:25-26: Argument `Literal[3]` is not assignable to parameter `ndigits` with type `None` in function `round` [bad-argument-type] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `NAType` has no attribute `fillna` [missing-attribute] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `NaTType` has no attribute `fillna` [missing-attribute] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `Number` has no attribute `fillna` [missing-attribute] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `Timedelta` has no attribute `fillna` [missing-attribute] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `Timestamp` has no attribute `fillna` [missing-attribute] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `float` has no attribute `fillna` [missing-attribute] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `ndarray` has no attribute `fillna` [missing-attribute] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `number` has no attribute `fillna` [missing-attribute] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:60:16-24: ` ExtensionArray | Index | Series | ndarray[Unknown, Unknown]) | None) -> DataFrame` [no-matching-overload] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:62:34-69: Missing argument `by` in function `pandas.core.frame.DataFrame.sort_values` [missing-argument] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:65:34-69: No matching overload found for function `pandas.core.frame.DataFrame.sort_values`, reporting errors for closest overload: `(by: Hashable | Sequence[Hashable], *, axis: Literal['columns', 'index', 'rows'] | int, ascending: Unknown, inplace: Literal[False], kind: Literal['heapsort', 'mergesort', 'quicksort', 'stable'], na_position: Literal['first', 'last'], ignore_index: bool, key: ((Series) -> ExtensionArray | Index | Series | ndarray[Unknown, Unknown]) | None) -> DataFrame` [no-matching-overload] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:65:34-69: Missing argument `by` in function `pandas.core.frame.DataFrame.sort_values` [missing-argument] ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:72:20-28: `


After that, I explicitly specified the interpreter path in pyproject.toml. python_interpreter: Some("C:\Users\anzhi\dev\py\cbm_correct.py\./.venv/Scripts/python.exe")

PS C:\Users\anzhi\dev\py\cbm_correct.py> hx .\pyproject.toml
PS C:\Users\anzhi\dev\py\cbm_correct.py> pyrefly check --verbose
 INFO Checking project configured at `C:\Users\anzhi\dev\py\cbm_correct.py\pyproject.toml`
DEBUG Config is: {project_includes: [C:\Users\anzhi\dev\py\cbm_correct.py\**\*], project_excludes: [C:\Users\anzhi\dev\py\cbm_correct.py\**\.[!\.]*, C:\Users\anzhi\dev\py\cbm_correct.py\**\*venv\**\*], search_path: [C:\Users\anzhi\dev\py\cbm_correct.py], python_interpreter: Some("C:\\Users\\anzhi\\dev\\py\\cbm_correct.py\\./.venv/Scripts/python.exe"), python_environment: {python_platform: win32, python_version: 3.13.3, site_package_path: [C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.13_3.13.1008.0_x64__qbz5n2kfra8p0\DLLs, C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.13_3.13.1008.0_x64__qbz5n2kfra8p0\Lib, C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.13_3.13.1008.0_x64__qbz5n2kfra8p0, C:\Users\anzhi\dev\py\cbm_correct.py\.venv, C:\Users\anzhi\dev\py\cbm_correct.py\.venv\Lib\site-packages], site_package_path_source: Interpreter("C:\\Users\\anzhi\\dev\\py\\cbm_correct.py\\./.venv/Scripts/python.exe")}, replace_imports_with_any: []}
DEBUG Checking 3 files (listing took 2.82s)
DEBUG Running with 4 threads
[00:00:00] ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░       0/0      DEBUG Running epoch 1
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:14:12-17: No matching overload found for function `round`, reporting errors for closest overload: `(number: _SupportsRound1[@_], ndigits: None = ...) -> @_` [no-matching-overload]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:14:25-26: Argument `Literal[3]` is not assignable to parameter `ndigits` with type `None` in function `round` [bad-argument-type]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `NAType` has no attribute `fillna` [missing-attribute]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `NaTType` has no attribute `fillna` [missing-attribute]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `Number` has no attribute `fillna` [missing-attribute]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `Timedelta` has no attribute `fillna` [missing-attribute]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `Timestamp` has no attribute `fillna` [missing-attribute]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `float` has no attribute `fillna` [missing-attribute]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `ndarray` has no attribute `fillna` [missing-attribute]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:40:27-85: Object of class `number` has no attribute `fillna` [missing-attribute]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:60:16-24: `<` is not supported between `SupportsAbs[Unknown]` and `Literal[0]`
  Argument `SupportsAbs[Unknown]` is not assignable to parameter with type `int` in function `int.__gt__` [bad-argument-type]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:62:34-69: No matching overload found for function `pandas.core.frame.DataFrame.sort_values`, reporting errors for closest overload: `(by: Hashable | Sequence[Hashable], *, axis: Literal['columns', 'index', 'rows'] | int, ascending: Unknown, inplace: Literal[False], kind: Literal['heapsort', 'mergesort', 'quicksort', 'stable'], na_position: Literal['first', 'last'], ignore_index: bool, key: ((Series) -> ExtensionArray | Index | Series | ndarray[Unknown, Unknown]) | None) -> DataFrame` [no-matching-overload]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:62:34-69: Missing argument `by` in function `pandas.core.frame.DataFrame.sort_values` [missing-argument]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:65:34-69: No matching overload found for function `pandas.core.frame.DataFrame.sort_values`, reporting errors for closest overload: `(by: Hashable | Sequence[Hashable], *, axis: Literal['columns', 'index', 'rows'] | int, ascending: Unknown, inplace: Literal[False], kind: Literal['heapsort', 'mergesort', 'quicksort', 'stable'], na_position: Literal['first', 'last'], ignore_index: bool, key: ((Series) -> ExtensionArray | Index | Series | ndarray[Unknown, Unknown]) | None) -> DataFrame` [no-matching-overload]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:65:34-69: Missing argument `by` in function `pandas.core.frame.DataFrame.sort_values` [missing-argument]
ERROR C:\Users\anzhi\dev\py\cbm_correct.py\main.py:72:20-28: `<` is not supported between `SupportsAbs[Unknown]` and `Literal[0]`
  Argument `SupportsAbs[Unknown]` is not assignable to parameter with type `int` in function `int.__gt__` [bad-argument-type]
 INFO 16 errors shown, 0 errors ignored, 3 modules, 855 transitive dependencies, 342,422 lines, took 6.87s (checking 4.05s; listing 2.82s), peak memory physical 252.5 MiB
PS C:\Users\anzhi\dev\py\cbm_correct.py>

anzhi0708 avatar Jun 05 '25 01:06 anzhi0708

Wait @anzhi0708, is it working for you when manually supplying python_interpreter or not? I'm seeing "site_package_path": [..., C:\Users\anzhi\dev\py\cbm_correct.py\.venv\Lib\site-packages], which seems right.

I also don't see any import errors in the output you linked, and it seems like the full output provided by Pyrefly. Are you seeing import errors, or is there another issue going on that you're seeing?

connernilsen avatar Jun 05 '25 13:06 connernilsen

Hey @connernilsen

(1) After I manually specified the exact python_interpreter, everything started working fine. (2) I use the pyrefly LSP 99% of the time—within my text editor. While editing code, the LSP reports import errors. I don’t understand why pyrefly check doesn’t report the same import errors when it fails to locate the interpreter.

anzhi0708 avatar Jun 05 '25 21:06 anzhi0708

I'm seeing "site_package_path": [..., C:\Users\anzhi\dev\py\cbm_correct.py.venv\Lib\site-packages], which seems right.

Oh! You're right. So in that case, would this be an LSP-specific issue?

anzhi0708 avatar Jun 05 '25 21:06 anzhi0708

Here's my editor setup (Helix 24.7, ~/.config/helix/languages.toml):

[language-server.ruff]
command = "ruff"
args = ["server"]

[language-server.pyrefly]
command = "pyrefly"
args = ["lsp"]

[[language]]
name = "python"
scope = "source.python"
injection-regex = "py(thon)?"
file-types = ["py", "pyi", "py3", "pyw", "ptl", "rpy", "cpy", "ipy", "pyt", { glob = ".python_history" }, { glob = ".pythonstartup" }, { glob = ".pythonrc" }, { glob = "*SConstruct" }, { glob = "*SConscript" }, { glob = "*sconstruct" }]
shebangs = ["python", "uv"]
roots = ["pyproject.toml", "setup.py", "poetry.lock", "pyrightconfig.json", "uv.lock"]
comment-token = "#"
language-servers = ["pyrefly", "ruff"]
# TODO: pyls needs utf-8 offsets
indent = { tab-width = 4, unit = "    " }

anzhi0708 avatar Jun 05 '25 21:06 anzhi0708

@anzhi0708, it might be. The CLI looks like it might be finding the right interpreter.

  1. What's giving you the impression that it's not (just want to make sure we're on the same page here 😄)?

Is it the python_interpreter: None thing? If so, that's an unfortunate state of things now, and I can work on clearing up the message there. It looks like on CLI, we're finding C:\\Users\\anzhi\\dev\\py\\cbm_correct.py\\.venv\\Scripts\\python.exe as your interpreter.

  1. Is that the interpreter you want Pyrefly to be using/where you have your dependencies installed?

In VSCode, you have the ability to select your interpreter, which we take into account in Pyrefly (unless you've provided a python_interpreter in your Pyrefly config) instead of searching for one with Pyrefly's logic. At the bottom right corner of your VSCode window, you might see something that looks like the following.

Image

If you click on that, you should be able to select an interpreter. Make sure it's the one you're expecting to use with your project.

Note: if you've having trouble finding it, make sure you have a Python file selected, and if it's still not working, try the instructions here.

  1. Once you try selecting a different interpreter, do the differences between CLI go away?

connernilsen avatar Jun 05 '25 21:06 connernilsen

@connernilsen Yes, I assumed the CLI hadn't found the interpreter because of the python_interpreter: None message.

You mentioned VS Code's Python interpreter selection and pyrefly's LSP behavior inside VS Code—honestly, I'm not particularly concerned with Microsoft products (or their extensions), but as a user, I would certainly appreciate convenient interpreter selection in pyrefly's LSP mode (for other text editors).

cbm_correct.py\\.venv\\Scripts\\python.exe is indeed the interpreter I want to use. After manually specifying the interpreter path (by editing pyproject.toml, since I’m using Helix Editor—a Rust-based editor similar to Vim but with built-in LSP support), the import errors in pyrefly LSP mode disappear. That solves all the issues I was encountering.

However, since other maintainers have mentioned that pyrefly is supposed to automatically pick up the venv even in LSP mode, I think that’s an area worth improving?

To summarize, the issue I encountered can be worked around using other methods, and it doesn't affect my ability to use Pyrefly ;-)

anzhi0708 avatar Jun 06 '25 04:06 anzhi0708

Can you both run pyrefly check --verbose and attach the first few lines of that output (before the errors start)

(.venv) noten@noten:~/Documents/GitHub/hukuksuzlugun-bedeli$ pyrefly check --verbose
 INFO Checking project configured at `/home/noten/Documents/GitHub/hukuksuzlugun-bedeli/pyproject.toml`
DEBUG Config is: {project_includes: [/home/noten/Documents/GitHub/hukuksuzlugun-bedeli/**/*], project_excludes: [/home/noten/Documents/GitHub/hukuksuzlugun-bedeli/**/.[!/.]*, /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/**/*venv/**/*], search_path: [/home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/, /home/noten/Documents/GitHub/hukuksuzlugun-bedeli], python_interpreter: None, python_environment: {python_platform: linux, python_version: 3.12.3, site_package_path: [/usr/lib/python3.12, /usr/lib/python3.12/lib-dynload, /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/.venv/lib/python3.12/site-packages], site_package_path_source: Interpreter("/home/noten/Documents/GitHub/hukuksuzlugun-bedeli/.venv/bin/python3")}, replace_imports_with_any: []}
DEBUG Checking 17 files (listing took 1.08s)
DEBUG Running with 16 threads
[00:00:00] ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░       0/0      DEBUG Running epoch 1

site_package_path_source: Interpreter("/home/noten/Documents/GitHub/hukuksuzlugun-bedeli/.venv/bin/python3")}

(.venv) noten@noten:~/Documents/GitHub/hukuksuzlugun-bedeli$ /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/.venv/bin/python3 -c "import sys; print(sys.path)" ['', '/usr/lib/python312.zip', '/usr/lib/python3.12', '/usr/lib/python3.12/lib-dynload', '/home/noten/Documents/GitHub/hukuksuzlugun-bedeli/.venv/lib/python3.12/site-packages']

Notenlish avatar Jun 09 '25 21:06 Notenlish

@anzhi0708 I see. I'll go through and try to make that more clear, since it really is quite confusing to see that when it's really not the true state of things (I was also tripped up until I realized that's what was going on 😅).

And yeah, I've been trying to work on getting the automatic-functionality working, but haven't been able to look at it/fix it in a few weeks. Currently, the way that Pyrefly only uses the MS-python extension to determine the environment in LSP mode before falling back to some system defaults was an intentional design choice at a time where we were more constrained (c.c. @kinto0).

I'm pretty sure I have an idea of how to implement that stuff better now though, so that we can still use our original environment finding logic when an IDE doesn't tell us which interpreter to use, which I'd imagine is how all non-vscode editors would behave. Ideally, after this, you shouldn't need to specify your interpreter path!

connernilsen avatar Jun 09 '25 23:06 connernilsen

Alright, I opened #451 to address the interpreter finding stuff so that we can hopefully be better about finding interpreters (especially local venv interpreters) for non-vscode IDEs. Still need to test once more, but that should be a big improvement there.

Also opened #450 to make our debug logging less confusing about python_interpreter

connernilsen avatar Jun 09 '25 23:06 connernilsen

To add more color, we don't only support vscode for the interpreter selector. Our .vsix is also deployed to openvsx among other places. Other language clients that support the workspace capability can send a workspace/configuration response if they have their own method for selecting Python interpreters. I can add more documentation if that is something people are interested in.

But in this case, the configuration file or .venv detection is likely the easiest solution.

kinto0 avatar Jun 09 '25 23:06 kinto0

For clarification, @Notenlish #451 should address your issue too.

@anzhi0708 @Notenlish I'll do a release later this week (probably tomorrow) once I land another fix that needs to get in, and we can double check to see if your problem is solved.

connernilsen avatar Jun 11 '25 23:06 connernilsen

Glad to hear that! I should have clarified from the beginning: (1) I'm not using the VS Code extension, and (2) my issue is specifically related to LSP mode.

If you could let me know which version I should test and where I can provide feedback, I'd be happy to help.

anzhi0708 avatar Jun 12 '25 14:06 anzhi0708

@anzhi0708 @Notenlish Alright, just released 0.19.1, which should have the changes. Try it out when you get a chance, and let me know if it's working for you!

connernilsen avatar Jun 12 '25 23:06 connernilsen

@connernilsen It’s working fine on my company’s computer now! On Windows 11, I ran uv init hello (to create the virtual environment) and uv add httpx typer (to add dependencies), then edited the files directly with Helix text editor (without running pyrefly init). There were no import errors. Auto-completion works fine too 🎉🎉🎉

anzhi0708 avatar Jun 13 '25 00:06 anzhi0708

It seems to be mostly working but I found a random bug when testing

after running pyrefly check

ERROR Could not import `bcrypt` from `passlib.hash` [missing-module-attribute]
 --> /home/noten/Documents/GitHub/hukuksuzlugun-bedeli/backend/pyag.py:1:26
  |
1 | from passlib.hash import bcrypt
  |                          ^^^^^^
  |

the code is this:

from passlib.hash import bcrypt

password = "123456"
hashed = bcrypt.hash(password)

print(f"Hashed password: {hashed}")

I already installed passlib

Notenlish avatar Jun 13 '25 07:06 Notenlish

I already installed passlib

Thanks for the report! Did you also install bcrypt specifically? Mine doesn't work until I do that:

Traceback (most recent call last):
  File "c:\Users\kylei\projects\pyrefly\test.py", line 6, in <module>
    print(bcrypt.hash("test"))
    self.checksum = self._calc_checksum(secret)
  File "C:\Users\kylei\AppData\Roaming\Python\Python310\site-packages\passlib\handlers\bcrypt.py", line 591, in _calc_checksum
    self._stub_requires_backend()
  File "C:\Users\kylei\AppData\Roaming\Python\Python310\site-packages\passlib\utils\handlers.py", line 2254, in _stub_requires_backend      
    cls.set_backend()
  File "C:\Users\kylei\AppData\Roaming\Python\Python310\site-packages\passlib\utils\handlers.py", line 2176, in set_backend
    raise default_error
passlib.exc.MissingBackendError: bcrypt: no backends available -- recommend you install one (e.g. 'pip install bcrypt')

Pylance works without installing it because it autobundles typeshed stubs which we don't yet support.

kinto0 avatar Jun 13 '25 13:06 kinto0

Oh interesting @kinto0.

@Notenlish We have a task for someone to work on auto including the typeshed stubs that I believe someone is going to pick up soon.

connernilsen avatar Jun 13 '25 16:06 connernilsen

Speaking of stubs, it reminds me that a friend recently told me Django has a similar issue.

Pylance seems to automatically install django-stubs, but both ty and pyrefly throw errors related to some Django stuff (like bad-override error). Could you guys also check that out? Thank you!

anzhi0708 avatar Jun 14 '25 01:06 anzhi0708

Django-stubs has a mypy plugin, so supporting it may be difficult - there's some discussion about plugins in https://github.com/facebook/pyrefly/issues/168 and https://github.com/facebook/pyrefly/discussions/407

Django-types might be an alternative (there are some known gaps in pyrefly that probably prevent it from checking cleanly ATM), but automatically installing dependencies when the typechecker runs may be undesirable for some users.

yangdanny97 avatar Jun 14 '25 07:06 yangdanny97

Did you also install bcrypt specifically? Mine doesn't work until I do that:

I already installed it

Notenlish avatar Jun 14 '25 14:06 Notenlish

@Notenlish it looks like the passlib library you're using does a lot of weird dynamic stuff in it that static analysis likely won't be able to handle. The solution here, since passlib is included in typeshed stubs, is to use that stubs lib automatically, which we have someone working on setting up right now.

Until then, if you want to just get something working in the meantime, I'd recommend the following:

  1. Find your site package path: run python3 import site; print(site.getsitepackages()) (source your venv before if you have one)
  2. Download typeshed
  3. In your Pyrefly config, set site-package-path to [<path to typeshed from step 2>/stubs, <site package path from step 1>]

This should include the stubs in a way Pyrefly can use for you.

Alternatively, if you're sharing this with others, you can check in the project stubs you need from typeshed somewhere into your project and add it as a search-path entry in your config (e.g. copy typeshed/stubs/passlib to something like <your_project>/stubs, then set search-path = [<your project stuff>, "stubs"]). This may cause type checking to behave weirdly in some cases though, so you may have to wait until we fix the typeshed importing stuff to be able to use this.

connernilsen avatar Jun 16 '25 19:06 connernilsen

closing due to inactivity. feel free to open another issue (preferred) or reopen this one if there's other issues

kinto0 avatar Jul 01 '25 17:07 kinto0