Extension fails to recover from pyenv shim without Ruff installed

[binarykitchen]

Hi, thanks for this awesome extension. Ruff is definitely so much faster.

When I run ruff check from the terminal, it shows linting errors.

But not on VS code. Nothing. Now sure why. Using the latest version ruff==0.3.2

VS code settings are

{
  "editor.tabSize": 2,
  "files.trimTrailingWhitespace": true,
  "editor.formatOnSave": true,
  "[python]": {
    "editor.formatOnSave": true,
    "editor.defaultFormatter": "charliermarsh.ruff",
    "editor.codeActionsOnSave": {
      "source.fixAll": "always",
      "source.organizeImports": "always"
    }
  },
}

Any clues why linting errors aren't highlighted when I open the file in VS code or while I'm typing?

[charliermarsh]

Can you click on "Ruff" in the bottom right:

And share some of the output logging?

My guess is there's some kind of error?

[binarykitchen]

Thanks, but I don't see such a ruff button at bottom

[binarykitchen]

Just do double-check that's the Ruff extension I've installed

[binarykitchen]

Oh, I found it in the terminal, under this select box:

[binarykitchen]

Now I'm seeing some weird lines like

2024-03-17 10:51:40.090 [info] [Warn  - 10:51:40] Skipping standard library file: /home/michael-heuberger/.pyenv/versions/3.8.13/lib/python3.8/uuid.py

What's that?

The commands it was running with were:

2024-03-17 10:51:40.218 [info] Using interpreter executable: /home/michael-heuberger/.pyenv/versions/3.8.13/envs/social_media_lambda/bin/ruff
2024-03-17 10:51:40.219 [info] Found ruff 0.3.2 at /home/michael-heuberger/.pyenv/versions/3.8.13/envs/social_media_lambda/bin/ruff
2024-03-17 10:51:40.219 [info] Running Ruff with: /home/michael-heuberger/.pyenv/versions/3.8.13/envs/social_media_lambda/bin/ruff ['check', '--force-exclude', '--no-cache', '--no-fix', '--quiet', '--output-format', 'json', '-', '--stdin-filename', '/home/michael-heuberger/.vscode/extensions/ms-python.python-2024.2.1/pythonFiles/lib/jedilsp/jedi/third_party/typeshed/stdlib/3/builtins.pyi']

[dhruvmanila]

Thank you for providing the logs. The issue arises because we currently ignore any files inside the VS Code extensions directory. This behavior was introduced in this ruff-lsp commit (https://github.com/astral-sh/ruff-lsp/commit/e7f69bc4c009e117db67d74ecd18c538bdc41da6).

Could you please let us know why you're working on a file inside the extensions directory?

However, you can disable this check by setting "ignoreStandardLibrary": false (https://github.com/astral-sh/ruff-vscode#settings).

[binarykitchen]

Interesting. Well, /home/michael-heuberger/.vscode/extensions/ms-python.python-2024.2.1 is where VS Code did install that Python extension, so isn't that sort of expected, pretty much the standard?

I'm not working inside that directory. I think it's more like grabbing builtins.pyi within for better language parsing or something like that?

Regarding that setting ignoreStandardLibrary, I don't see it:

Apart from that, I might have some misconfigurations between pyenv, Python extensions and this one. What's your recommended setup?

[charliermarsh]

What Dhruv is referring to only explains the Skipping standard library file messages. That's actually expected -- if you open a standard library file, we won't show you diagnostics. But your other screenshot shows file paths that are being successfully linted, so I don't know why you're not seeing any diagnostics in the editor itself.

[dhruvmanila]

When I run ruff check from the terminal, it shows linting errors.

Is it possible for you to isolate the code snippet where the diagnostics are being reported when you run from the command-line and share the snippet with us? Totally understand if you cannot. But, if you can, please make sure that the same snippet shows errors when running from the command-line and doesn't show any errors in the VS Code editor. This would be extremely useful in identifying the issue at hand.

Are the diagnostics not being highlighted in the file mentioned in this comment's screenshot (https://github.com/astral-sh/ruff-vscode/issues/426#issuecomment-2002148974) ?

[binarykitchen]

Thanks, guys. Somehow, after having disabled Skipping standard library file (that option I've found under User settings, not workspace settings, Ruff is finally highlighting errors in the IDE and is behaving as expected.

Maybe this deserves to be documented? Note I'm using pyenv if that makes a difference?

[dhruvmanila]

@binarykitchen Is it possible for you to share us the file path for which you faced this problem? It seems that the file you were working on is being detected as being part of standard library. I just want to make sure that there's no bug in the detection logic.

[binarykitchen]

Ummm sorry, which file path @dhruvmanila?

I think it's linting fine now in my project for weird reasons I cannot explain. So I've opened a different project and am now seeing similar errors like I had before:

2024-03-20 17:10:29.468 [info] 
The `ruff' command exists in these Python versions:
  3.8.13/envs/social_media_lambda
  social_media_lambda

Note: See 'pyenv help global' for tips on allowing both
      python2 and python3 to be found.
2024-03-20 17:10:29,289 ERROR Exception occurred in notification: "subprocess.CalledProcessError: Command '['/home/michael-heuberger/.pyenv/shims/ruff', '--version']' returned non-zero exit status 127."
Traceback (most recent call last):
  File "/home/michael-heuberger/.vscode/extensions/charliermarsh.ruff-2024.16.0-linux-x64/bundled/libs/pygls/protocol/json_rpc.py", line 159, in _execute_notification_callback
    raise future.exception()
  File "/home/michael-heuberger/.vscode/extensions/charliermarsh.ruff-2024.16.0-linux-x64/bundled/libs/ruff_lsp/server.py", line 455, in did_open
    diagnostics = await _lint_document_impl(document, settings)
  File "/home/michael-heuberger/.vscode/extensions/charliermarsh.ruff-2024.16.0-linux-x64/bundled/libs/ruff_lsp/server.py", line 605, in _lint_document_impl
    result = await _run_check_on_document(document, settings)
  File "/home/michael-heuberger/.vscode/extensions/charliermarsh.ruff-2024.16.0-linux-x64/bundled/libs/ruff_lsp/server.py", line 1843, in _run_check_on_document
    executable = _find_ruff_binary(settings, VERSION_REQUIREMENT_LINTER)
  File "/home/michael-heuberger/.vscode/extensions/charliermarsh.ruff-2024.16.0-linux-x64/bundled/libs/ruff_lsp/server.py", line 1750, in _find_ruff_binary
    version = _executable_version(path)
  File "/home/michael-heuberger/.vscode/extensions/charliermarsh.ruff-2024.16.0-linux-x64/bundled/libs/ruff_lsp/server.py", line 1825, in _executable_version
    version = utils.version(executable)
  File "/home/michael-heuberger/.vscode/extensions/charliermarsh.ruff-2024.16.0-linux-x64/bundled/libs/ruff_lsp/utils.py", line 96, in version
    output = subprocess.check_output([executable, "--version"]).decode().strip()
  File "/home/michael-heuberger/.pyenv/versions/3.8.13/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/home/michael-heuberger/.pyenv/versions/3.8.13/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/michael-heuberger/.pyenv/shims/ruff', '--version']' returned non-zero exit status 127.
2024-03-20 17:10:29,315 WARNING Cancel notification for unknown message id "1"

2024-03-20 17:10:29.487 [info] [Trace - 17:10:29] Sending notification '$/cancelRequest'.
2024-03-20 17:10:29.497 [info] [Trace - 17:10:29] Sending request 'textDocument/codeAction - (4)'.
2024-03-20 17:10:29.650 [info] 2024-03-20 17:10:29,478 WARNING Cancel notification for unknown message id "2"

2024-03-20 17:10:29.736 [info] 2024-03-20 17:10:29,659 WARNING Cancel notification for unknown message id "3"

2024-03-20 17:10:29.816 [info] 2024-03-20 17:10:29,795 WARNING Cancel notification for unknown message id "4"

2024-03-20 17:10:30.687 [info] 2024-03-20 17:10:30,672 WARNING Cancel notification for unknown message id "6"

Hope this helps?

[binarykitchen]

Oh, another interesting finding. When I went back to the other project, I'm seeing similar errors:

2024-03-20 17:12:52.568 [info] Global settings: {
    "cwd": "/home/michael-heuberger",
    "workspace": "/home/michael-heuberger",
    "path": [],
    "ignoreStandardLibrary": false,
    "interpreter": [],
    "importStrategy": "fromEnvironment",
    "codeAction": {
        "fixViolation": {
            "enable": true
        },
        "disableRuleComment": {
            "enable": true
        }
    },
    "lint": {
        "enable": true,
        "run": "onType",
        "args": []
    },
    "format": {
        "args": []
    },
    "enable": true,
    "organizeImports": true,
    "fixAll": true,
    "showNotifications": "always"
}
2024-03-20 17:12:53.645 [info] [Trace - 17:12:53] Received notification 'window/logMessage'.
2024-03-20 17:12:53.645 [info] Using interpreter executable: /home/michael-heuberger/.pyenv/versions/3.8.13/envs/social_media_lambda/bin/ruff
2024-03-20 17:12:53.694 [info] [Trace - 17:12:53] Received notification 'window/logMessage'.
2024-03-20 17:12:53.694 [info] Inferred version 0.3.2 for: /home/michael-heuberger/.pyenv/versions/3.8.13/envs/social_media_lambda/bin/ruff
2024-03-20 17:12:53.736 [info] [Trace - 17:12:53] Received notification 'window/logMessage'.
2024-03-20 17:12:53.737 [info] Found ruff 0.3.2 at /home/michael-heuberger/.pyenv/versions/3.8.13/envs/social_media_lambda/bin/ruff
2024-03-20 17:12:53.780 [info] [Trace - 17:12:53] Received notification 'window/logMessage'.
2024-03-20 17:12:53.780 [info] Running Ruff with: /home/michael-heuberger/.pyenv/versions/3.8.13/envs/social_media_lambda/bin/ruff ['check', '--force-exclude', '--no-cache', '--no-fix', '--quiet', '--output-format', 'json', '-', '--stdin-filename', '/home/michael-heuberger/code/dexibit/social_media_lambda/dexibitpy/util/vendor_helpers/facebook_integration.py']
2024-03-20 17:12:53.789 [info] [Trace - 17:12:53] Received notification 'textDocument/publishDiagnostics'.
2024-03-20 17:12:53.982 [info] [Trace - 17:12:53] Sending request 'textDocument/codeAction - (1)'.
2024-03-20 17:12:54.027 [info] [Trace - 17:12:54] Sending notification '$/cancelRequest'.
2024-03-20 17:12:54.054 [info] [Trace - 17:12:54] Sending request 'textDocument/codeAction - (2)'.
2024-03-20 17:12:54.061 [info] [Trace - 17:12:54] Received response 'textDocument/codeAction - (1)' in 79ms.
2024-03-20 17:12:54.081 [info] 2024-03-20 17:12:54,072 WARNING Cancel notification for unknown message id "1"

2024-03-20 17:12:54.097 [info] [Trace - 17:12:54] Received response 'textDocument/codeAction - (2)' in 42ms.
2024-03-20 17:12:54.364 [info] [Trace - 17:12:54] Sending request 'textDocument/codeAction - (3)'.
2024-03-20 17:12:54.622 [info] [Trace - 17:12:54] Received response 'textDocument/codeAction - (3)' in 258ms.
2024-03-20 17:12:55.077 [info] [Trace - 17:12:55] Sending request 'textDocument/codeAction - (4)'.
2024-03-20 17:12:55.160 [info] [Trace - 17:12:55] Sending notification '$/cancelRequest'.
2024-03-20 17:12:55.164 [info] [Trace - 17:12:55] Sending request 'textDocument/codeAction - (5)'.
2024-03-20 17:12:55.166 [info] [Trace - 17:12:55] Sending notification '$/cancelRequest'.
2024-03-20 17:12:55.170 [info] [Trace - 17:12:55] Sending request 'textDocument/codeAction - (6)'.
2024-03-20 17:12:55.247 [info] 2024-03-20 17:12:55,181 WARNING Cancel notification for unknown message id "4"
2024-03-20 17:12:55,183 WARNING Cancel notification for unknown message id "5"

2024-03-20 17:12:55.251 [info] [Trace - 17:12:55] Received response 'textDocument/codeAction - (4)' in 174ms.
2024-03-20 17:12:55.299 [info] [Trace - 17:12:55] Received response 'textDocument/codeAction - (5)' in 135ms.
2024-03-20 17:12:55.299 [info] [Trace - 17:12:55] Received response 'textDocument/codeAction - (6)' in 129ms.
2024-03-20 17:12:56.163 [info] 2024-03-20 17:12:55,978 WARNING Cancel notification for unknown message id "7"

Why those warnings at the end?

2024-03-20 17:12:56.163 [info] 2024-03-20 17:12:55,978 WARNING Cancel notification for unknown message id "7"

[dhruvmanila]

Ignore the cancel notification logs.

I think the problem you're facing is similar to the one I've described here https://github.com/astral-sh/ruff-vscode/issues/428#issuecomment-2011642615. It's related to how pyenv shims work. You can see it in your logs:

2024-03-20 17:10:29.468 [info] 
The `ruff' command exists in these Python versions:
  3.8.13/envs/social_media_lambda
  social_media_lambda

Note: See 'pyenv help global' for tips on allowing both
      python2 and python3 to be found.

The server tries to get the version of Ruff through ruff --version command. Here, ruff is a pyenv shim which then tries to find the executable in your configured environments but it doesn't find it so it fails. This makes the subprocess call here throw an error.

I wonder if we should handle that failure gracefully.

[binarykitchen]

Interesting discovery @dhruvmanila - I don't know much about it and hope you can figure a better algorithm for failures like these? LMK if I can help with something.

[charliermarsh]

Could we... detect the error, then fall back to the bundled version? We might just be doing the right thing here though already.

[dhruvmanila]

Could we... detect the error, then fall back to the bundled version?

This might just be an edge case but what if the bundled Ruff version is different than the one installed globally by the user? In this case, the user could possibly see different behavior.

We might just be doing the right thing here though already.

Can you expand on this? I don't think the extension should fail silently. We should at least provide a notification that there's some failure. Although, it might be hard for a user to detect the cause of this failure unless they have some knowledge about pyenv shims.

I think the fallback behavior is a good solution, at least for this specific error.