AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?

[aorumbayev]

Describe the bug

The following repo was specifically created to outline a reproducible environment with the bug https://github.com/aorumbayev/pipx_bug.

I am not sure of the root cause but it is something to do with the way pipx uses shared venv and running pipx installed executable python modules that have copier package as transitive dependency installed for target py >=3.12 while pipx itself is installed with py <3.12.

I also opened this issue on copier repo copier where copier maintainers forwarded me to also open an issue on plumbum transitive dependency that they rely on (and which throws the error) -> available here https://github.com/tomerfiliba/plumbum/issues/678. Yet, still opening the issue on pipx repo as well given that copier maintainers also suspect that something may be wrong with the way shared venv is reused by pipx. Based on logs pipx is somehow able to access packages across multiple venvs. The issue is easily reproduced on github codespaces or github actions (use ubuntu-latest workers). If you have pipx installed locally on your machine and it was installed using 3.12 as a base python - everything works just fine (because the shared venv on pipx side is also using 3.12 in that case).

Logs

Traceback (most recent call last):
  File "/usr/local/py-utils/bin/pipx-bug-cli", line 5, in <module>
    from pipx_bug.cli import hello
  File "/usr/local/py-utils/venvs/pipx-bug/lib/python3.12/site-packages/pipx_bug/cli.py", line 2, in <module>
    from copier.main import Worker
  File "/usr/local/py-utils/venvs/pipx-bug/lib/python3.12/site-packages/copier/__init__.py", line 6, in <module>
    from .main import *  # noqa: F401,F403
    ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/py-utils/venvs/pipx-bug/lib/python3.12/site-packages/copier/main.py", line 33, in <module>
    from plumbum.cli.terminal import ask
  File "/usr/local/py-utils/venvs/pipx-bug/lib/python3.12/site-packages/plumbum/cli/__init__.py", line 1, in <module>
    from .application import Application
  File "/usr/local/py-utils/venvs/pipx-bug/lib/python3.12/site-packages/plumbum/cli/application.py", line 9, in <module>
    from plumbum.cli.i18n import get_translation_for
  File "/usr/local/py-utils/venvs/pipx-bug/lib/python3.12/site-packages/plumbum/cli/i18n.py", line 28, in <module>
    import pkg_resources
  File "/usr/local/py-utils/shared/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2191, in <module>
    register_finder(pkgutil.ImpImporter, find_on_path)
                    ^^^^^^^^^^^^^^^^^^^
AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?

How to reproduce

Refer to readme on https://github.com/aorumbayev/pipx_bug

Expected behavior

Executing packages via pipx on >=3.12 should defer to 3.12 based shared venv

[henryiii]

The problem is:

Python 3.12.2 (main, Apr 23 2024, 13:31:25) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.path)
['', '/home/codespace/.pyenv/versions/3.12.2/lib/python312.zip', '/home/codespace/.pyenv/versions/3.12.2/lib/python3.12', '/home/codespace/.pyenv/versions/3.12.2/lib/python3.12/lib-dynload', '/usr/local/py-utils/venvs/algokit/lib/python3.12/site-packages', '/usr/local/py-utils/shared/lib/python3.10/site-packages']

The 3.10 site-packages is sneaking in there, and I don't see how. It's only in the pipx venv Python, but pyvenv.cfg seems fine, etc. More info at https://github.com/tomerfiliba/plumbum/issues/678.