Add support Poetry (pypoetry)

[theperelomov]

Try to get venv from pypoetry. If not configured pythonPath or set VIRTUAL_ENV.

[theperelomov]

About popularity I don't think it's about popularity, it`s just convenient that everything works out of the box (poetry, pyenv and other tools) Poetry support by default: PyCharm, VsCode Python ext, coc-pyright - this list I use and I know, think the list is bigger. Not difficult for make own configs, but I can get errors, spend time VS solution that is tested and supported by the community.

[mfussenegger]

Added a way to customize to logic: https://github.com/mfussenegger/nvim-dap-python/pull/70

[baggiponte]

Ciao! I know #70 offers a way for nvimmers to customise the python path, but I would like to work on a PR to implement support for the most common python dependency manager - Poetry and PDM (which is newer, but is getting a lot of traction). I don't really know whether this is the right place to discuss, but I wanted to ask you if that was okay with you. It should be fairly easy to implement, but this would be my first contribution in Lua/nvim and would like to ask some help specifically about vim.loop and discuss whether/how we might want to parse toml (the pyproject.toml) to retrieve info about the dependency manager used.

[mfussenegger]

If I understand https://peps.python.org/pep-0582/ right then - assuming the PEP gets accepted - python will look for __pypackages__ automatically. There wouldn't be any extra work necessary in nvim-dap-python or debugpy.

By default, PDM will create .venv in the project root, when doing pdm install on an existing project, as other package managers do.

I could image to add a .venv folder check, similar to the one in the recipe in https://github.com/mfussenegger/nvim-dap/wiki/Debug-Adapter-installation#Python.

I don't intend to merge any code that parses the pyproject.toml file.

[baggiponte]

Thank you for your reply! Indeed, if the PEP gets accepted that's gonna be easy.

I could image to add a .venv folder check, similar to the one in the recipe in https://github.com/mfussenegger/nvim-dap/wiki/Debug-Adapter-installation#Python.

I was also thinking about the .venv folder check - sometimes someone can save the .venv with other names (why tho), so could it be possible to look for a /bin/python folder instead? Rewriting a bit the code you pointed at:

    pythonPath = function()
      -- debugpy supports launching an application with a different interpreter then the one used to launch debugpy itself.
      -- The code below looks for a `venv` or `.venv` folder in the current directly and uses the python within.
      -- You could adapt this - to for example use the `VIRTUAL_ENV` environment variable.
      local cwd = vim.fn.getcwd(env_name)
      if vim.fn.executable(cwd .. '/' .. env_name .. '/' .. '/bin/python') == 1 then
        return cwd .. '/' .. env_name .. '/' .. '/bin/python'
      elseif vim.fn.executable(cwd .. '/venv/bin/python') == 1 then
        return cwd .. '/venv/bin/python'
      elseif vim.fn.executable(cwd .. '/.venv/bin/python') == 1 then
        return cwd .. '/.venv/bin/python'
      else
        return '/usr/bin/python'
      end
    end;
  },
}

Alternatively, it could return the first python on the $PATH, so that it picks up pyenv's python automatically if it is active in the current directory.

About parsing the pyproject: sure! was just a random option, I kinda felt it was unfeasible as there is no builtin lib in lua to parse toml.

[mfussenegger]

I was also thinking about the .venv folder check - sometimes someone can save the .venv with other names (why tho), so could it be possible to look for a /bin/python folder instead? Rewriting a bit the code you pointed at:

Alternatively, it could return the first python on the $PATH, so that it picks up pyenv's python automatically if it is active in the current directory

There are many options to detect it, that's part of the reason I made it customizable - so that people can use the pattern that matches their setups.

I don't intend for nvim-dap-python to attempt to handle all possible scenarios that people can come up with. The goal is to handle stock python variants (VIRTUAL_ENV) and maybe a couple others that are very common and where the check is cheap (e.g. not spawning executables).

[baggiponte]

Hi! Sorry, it's been a while.

I have been trying to write my own function to obtain the python interpreter for pdm first, because that's the tool I know better.

I would like to design the function in this way:

1. Checks for the PDM_PYTHON env variable, which denotes the python executable in use within the project. However, this var is not exported automatically - it is up to the user. As an alternative, the function...
2. ... uses vim.loop, as you do in resolve_python, to read the contents of the .pdm.toml config file, and use a treesitter query to return the path (rather than looping over the rows to find the right key).

I feel a bit stuck, because I could not find lot of details about luv: did you try reading the contents of a file with fs_open() and fs_read()? The official documentation is not super detailed, and I don't really understand the signature of the latter.

Here's the skeleton of it - still need to finalise the parsing, but the query is correct.

local M = {}

local is_windows = function()
    return vim.loop.os_uname().sysname:find("Windows", 1, true) and true
end

M.resolve_pdm = function()
local pdm_python = os.getenv('PDM_PYTHON')

  if pdm_python then
    if is_windows() then
      return pdm_python .. '\\Scripts\\python.exe'
    end
    return pdm_python .. '/bin/python'
  end

  if vim.loop.fs_stat('.pdm.toml') then
    local bits = vim.loop.fs_open('./.pdm.toml', 'r', 438)
    local contents = vim.loop.fs_read()

    local tree = vim.treesitter.get_string_parser(contents, 'toml')

    local pdm_python = vim.treesitter.parse_query(
      'toml',
      [[
        (
          (bare_key) @path (#eq? @path "path")
          (string) @python_path
        )
      ]]
    )

    if is_windows() then
      return pdm_python .. '\\Scripts\\python.exe'
    end
    return pdm_python .. '/bin/python'
  else
    vim.notify('No `.pdm.toml` file was found', 'ERROR')
  end
end

return M

The alternative would be to run pdm config python.path, but this spawns an executable.

With poetry it is a bit more complex. By default, it uses the interpreter on the $PATH. Alternatively, the user can set the interpreter with poetry env use, which can be:

1. An absolute path to an interpreter.
2. A python on the $PATH (not the first one).
3. the name of one of the virtualenvs created with poetry.

This setting is then stored in poetry.toml. While poetry supports env vars as configs, IIUC you cannot export an env var containing the path that poetry will recognise.

I guess that using the treesitter query is technically parsing a file, so you might be against that. However, I dare to say that these tools are becoming more and more used, so supporting them could be a welcome addition.

Finally, covering pyenv is simple if you spawn an executable (python -c 'import sys; print(sys.executable)'), but can still be retrieved by combining the contents of PYENV_ROOT and the local .python-version file.

[mfussenegger]

I feel a bit stuck, because I could not find lot of details about luv: did you try reading the contents of a file with fs_open() and fs_read()

From :h uv.fs_read:

uv.fs_read({fd}, {size} [, {offset} [, {callback}]])              *uv.fs_read()*

                Parameters:
                - `fd`: `integer`
                - `size`: `integer`
                - `offset`: `integer` or `nil`
                - `callback`: `callable` (async version) or `nil` (sync
                  version)
                  - `err`: `nil` or `string`
                  - `data`: `string` or `nil`

That means you'd have to do something like this:

local fd = uv.fs_open(filename, 'r', 438)
if not fd then
  ...
end
local stat = uv.fs_fstat(fd)
local data = uv.fs_read(fd, stat.size, 0)

Or you could use io.open and file:read('*a')

I guess that using the treesitter query is technically parsing a file, so you might be against that

It would also depend on having a parser for toml available, which many users may not have.

However, I dare to say that these tools are becoming more and more used, so supporting them could be a welcome addition.

Part of the reason I do not want to make the logic too complicated is that it becomes harder to explain, and then it becomes more difficult for users to troubleshoot if it doesn't work. Which will then result in more issues.

"Set the VIRTUAL_ENV environment variable" is simple, and as a bonus point language servers pick it up too. Otherwise you have to solve that problem too.