Let poetry use both global/system-wide and venv dependencies

[zkkv]

Issue Kind

Brand new capability

Description

I'd love to be able to let poetry see both packages installed globally on my computer and packages installed via poetry to the project's virtual environment. Perhaps, the global ones should be explicitly mentioned somehow to avoid silent problems with dependencies.

I'm not aware of such functionality in poetry.

Impact

Here's one use case. I want to install a bunch of "regular" dependencies, say numpy and matplotlib for the sake of argument, from PyPI which I can do with poetry. But it is better for some dependencies, like PyTorch, to be left to the user to install on their computer just because it might break at any point. So it would nice to do something like:

poetry install --global=torch,torchvision

And then poetry will install numpy and matplotlib to the venv but it will also be aware that global installations of torch and torchvision should be used.

Given how many issues I've seen with PyTorch mentioned, this could be a solid way to deal with it.

Workarounds

I think you can do some things with --extras flag but I had problems with it. For instance, it tried to install all cuda wheel versions even if I wanted to install just one.

[radoering]

There is an option to make system site-packages available in the venv: https://python-poetry.org/docs/configuration/#virtualenvsoptionssystem-site-packages

[zkkv]

@radoering, ah, that's good to know, thanks. If I want it to apply to a single project, I would need to set it to true and then back to false globally, right? If so, that's a bit inconvenient.

[Secrus]

You can use poetry config --local virtualenvs.options.system-site-packages true for a project-level config

[zkkv]

Thanks. That seems to solve the problem. I'll close the issue.

[zkkv]

I want to reopen this issue because I realized that the existing solution is still not perfect. What if there are conflicting dependencies? Which one takes priority? I couldn't find anything about that in the docs. Ignoring that, I'd rather list system packages for poetry to use explicitly, like I described originally. What do you think, @radoering?

[radoering]

Ignoring that, I'd rather list system packages for poetry to use explicitly, like I described originally. What do you think, @radoering?

I think that such a feature is not requested often and, therefore, it is unlikely that anyone will implement it.

Which one takes priority? I couldn't find anything about that in the docs.

I assume the one in the venv will take priority. Poetry just passes through a virtualenv setting, which is also available in the stdlib venv module. Thus, it should be standard Python behavior.