Enabling PEP 582 globally causes pdm to use local resolvelib

[meop]

• [x] I have searched the issue tracker and believe that this is not a duplicate.

This issue is similar to https://github.com/pdm-project/pdm/issues/685 But in that bug report, users were not installing pdm in an isolated way. However, even installing pdm in a recommended, isolated way, there still seems to be a conflict with enabling PEP 582 globally.

Steps to reproduce

You need to have an older version of resolvelib inside your project's local __pypackages__ directory. This example project that will install resolvelib 0.5.5.. older than pdm's minimum of 0.8.

dependencies = [
    "ansible-core~=2.11.5",
    "passlib~=1.7.4",
]

You can install PDM using a recommended way like Homebrew. Then you can follow the section right after to Enable PEP 582 globally.

Actual behavior

The first run of pdm install -v will work correctly.

Subsequent runs of pdm install -v will end up using the older resolvelib in the project's __pypackages__ now, instead of falling back to the one that Homebrew put in pdm's isolated venv directory.

There will be various API argument errors printed to console.

Expected behavior

pdm install should ideally work even if a project uses an older resolvelib and the user would also like PEP 582 enabled globally. It is not intuitive that the pdm CLI, depending directly on Python itself, would be affected by global PEP 582 even for its own dependencies. It makes the tool a bit fragile, given that enabling global PEP 582 is, I think, one of its main purposes.

Perhaps we could find an alternative way for pdm CLI to exist alongside global PEP 582, but without the current side effects?

Again, thanks for working on the tool!

(A workaround for pdm to work on this project is to not enable PEP 582 globally, by unsetting PYTHONPATH while running pdm, and setting it again afterward.)

Environment Information

PDM version:        1.12.3
Python Interpreter: /Users/me/.asdf/installs/python/3.10.1/bin/python (3.10)
Project Root:       /Users/me/project
Project Packages:   /Users/me/project/__pypackages__/3.10
{
  "implementation_name": "cpython",
  "implementation_version": "3.10.1",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "21.2.0",
  "platform_system": "Darwin",
  "platform_version": "Darwin Kernel Version 21.2.0: Sun Nov 28 20:28:54 PST 2021; root:xnu-8019.61.5~1/RELEASE_X86_64",
  "python_full_version": "3.10.1",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "darwin"
}

[frostming]

I am afraid this has no graceful solution. Not only for PDM itself, enabling PEP 582 can make global tools work with local packages, and there might be conflicting dependencies. If we are to disable PEP 582, this feature is also disabled.

[meop]

So it seems with PEP 582 enabled, running any external Python script, whether it uses venv or not, can be affected by the local __pypackages__ directory overriding the intended packages.

I read PEP 582 to see if it discusses this scenario. It actually does, but only in the case where the external script was also using PEP 582 as well:

"While executing a Python script, it will not consider the __pypackages__ in the current directory, instead if there is a __pypackages__ directory in the same path of the script, that will be used."

This seems to suggest that if the official installation / invocation approaches of PDM were changed to use PEP 582, instead of venv, that PDM would then be protected from other PEP 582 projects!

Although this does not globally solve the __pypackages__ "hijacking" issue, since many other common external Python scripts will likely not be using PEP 582.. it would at least protect PDM.

What do you think?

[frostming]

PEP 582 is rejected and not recommended in PDM, though we still support it. Such problem is due to unpolished standard and behavior.

We are not going to work on this. Close it now.