No module named pip

[pawamoy]

Opening this ticket because I think it could help others. Also, not pipx's fault :+1:!

My distribution recently updated Python from 3.7.x to 3.8.0. It broke every single package I had installed with pipx somehow, and pipx itself.

I was left with these kind of errors:

$ guessit "some string"
Traceback (most recent call last):
  File "/home/pawamoy/.local/bin/guessit", line 5, in <module>
    from guessit.__main__ import main
ModuleNotFoundError: No module named 'guessit'
$ pipx install guessit
Traceback (most recent call last):
  File "/home/pawamoy/.local/bin/pipx", line 6, in <module>
    from pipx.main import cli
ModuleNotFoundError: No module named 'pipx'
$ pip install --user pipx
$ pipx install guessit
'guessit' already seems to be installed. Not modifying existing installation in '/home/pawamoy/.local/pipx/venvs/guessit'. Pass '--force' to force installation.
$ pipx install guessit --force
Installing to existing directory '/home/pawamoy/.local/pipx/venvs/guessit'
/home/pawamoy/.local/pipx/venvs/guessit/bin/python: No module named pip

Not removing existing venv /home/pawamoy/.local/pipx/venvs/guessit because it was not created in this session
'/home/pawamoy/.local/pipx/venvs/guessit/bin/python -m pip install guessit -q' failed
$ guessit
Traceback (most recent call last):
  File "/home/pawamoy/.local/bin/guessit", line 5, in <module>
    from guessit.__main__ import main
ModuleNotFoundError: No module named 'guessit'
$ pipx uninstall guessit
uninstalled guessit! ✨ 🌟 ✨
$ pipx install guessit
/home/pawamoy/.local/pipx/venvs/guessit/bin/python: No module named pip

'/home/pawamoy/.local/pipx/venvs/guessit/bin/python -m pip install guessit -q' failed

Not sure what exactly was happening behind the scenes as it's always hard to figure out with Python: there's the system versions, with their respective site-packages, there's pyenv and its own installed versions and shims, then there's potentially multiple pipx (shims?) and a script in ~/.local/bin (which one is the first in PATH?)... very convoluted but usually working.

Solution

Well, I simply had to explicitly uninstall pipx from the old system Python version with python3.7 -m pip uninstall pipx and then reinstall it with the new system Python version with python -m pip install --user pipx, and reinstall all pipx packages.

[QasimK]

I had the same problem, but was not able to use Python 3.7 to uninstall pipx.

I resolved this by uninstalling pipx, deleting ~/.local/pipx, and reinstalling pipx with the new version of Python.

[pawamoy]

Thanks, in fact I stumbled upon that problem again, and your solution worked.

[uolot]

Removing ~/.local/pipx and reinstalling pipx also worked for me! Thanks.

[metaflow]

Happened to me today too. Are you sure it's not an issue of pipx if it cannot survive python version upgrades?

[itsayellow]

Presently if you still have a functioning pipx but the pipx-installed apps are pointing to a non-existent python, you should be able to execute

pipx reinstall-all

To make things right.

pipx has improved its resilience in this regard since Nov. 2019.

[guihkx]

Arch Linux just updated its python package to v3.9 and pipx borked for me.

First, I tried listing installed packages:

$ pipx list
Traceback (most recent call last):
  File "/home/gui/.local/bin/pipx", line 6, in <module>
    from pipx.main import cli
ModuleNotFoundError: No module named 'pipx'

Then, I reinstalled pipx:

$ pip install --user pipx
Collecting pipx
  Downloading pipx-0.15.6.0-py3-none-any.whl (43 kB)
     |████████████████████████████████| 43 kB 1.1 MB/s 
Collecting userpath>=1.4.1
  Downloading userpath-1.4.1-py2.py3-none-any.whl (14 kB)
Requirement already satisfied: packaging>=20.0 in /usr/lib/python3.9/site-packages (from pipx) (20.7)
Collecting argcomplete<2.0,>=1.9.4
  Downloading argcomplete-1.12.2-py2.py3-none-any.whl (38 kB)
Requirement already satisfied: click in /usr/lib/python3.9/site-packages (from userpath>=1.4.1->pipx) (7.1.2)
Requirement already satisfied: distro; platform_system == "Linux" in /usr/lib/python3.9/site-packages (from userpath>=1.4.1->pipx) (1.5.0)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/lib/python3.9/site-packages (from packaging>=20.0->pipx) (2.4.7)
Installing collected packages: userpath, argcomplete, pipx
Successfully installed argcomplete-1.12.2 pipx-0.15.6.0 userpath-1.4.1

... Tried to list my packages again:

$ pipx list
venvs are in /home/gui/.local/pipx/venvs
apps are exposed on your $PATH at /home/gui/.local/bin
   package jsbeautifier has missing internal pipx metadata.
       It was likely installed using a pipx version before 0.15.0.0.
       Please uninstall and install this package, or reinstall-all to fix.
   package mitmproxy has missing internal pipx metadata.
       It was likely installed using a pipx version before 0.15.0.0.
       Please uninstall and install this package, or reinstall-all to fix.
   package youtube-dl has missing internal pipx metadata.
       It was likely installed using a pipx version before 0.15.0.0.
       Please uninstall and install this package, or reinstall-all to fix.

As suggested by pipx, I tried running pipx reinstall-all:

$ pipx reinstall-all
uninstalled jsbeautifier! ✨ 🌟 ✨
/home/gui/.local/pipx/venvs/jsbeautifier/bin/python: No module named pip

Error installing jsbeautifier.
uninstalled mitmproxy! ✨ 🌟 ✨
/home/gui/.local/pipx/venvs/mitmproxy/bin/python: No module named pip

Error installing mitmproxy.
uninstalled youtube-dl! ✨ 🌟 ✨
/home/gui/.local/pipx/venvs/youtube-dl/bin/python: No module named pip

Error installing youtube-dl.
The following package(s) failed to reinstall: jsbeautifier, mitmproxy, youtube-dl

Having that failed, I tried to manually install an app:

$ pipx install jsbeautifier
/home/gui/.local/pipx/venvs/jsbeautifier/bin/python: No module named pip

Error installing jsbeautifier.

Trying to list my apps again...:

$ pipx list
nothing has been installed with pipx 😴

Did I do something wrong? I haven't deleted my ~/.local/pipx folder yet, by the way, so pipx is still broken.

I'll wait for a response first in case this can help fix something.

[itsayellow]

@guihkx I don't think you did anything wrong. I believe the problem you are experiencing should be fixed in an upcoming release with #554 .

I think what happened was that after reinstalling pipx, the "shared libraries" that pipx uses internally to provide pip and other useful packages (~/.local/pipx/shared) were still stuck at the previous version and therefore unavailable to the new python / pipx. The default behavior in the next release will be to reinstall these shared libraries first before attempting to reinstall all the venvs.

Unfortunately, reinstall-all has uninstalled all of your venvs without being able to install any of them, so your ~/.local/pipx/venvs directory is probably empty. ☹️ Therefore if you want to start over by deleting ~/.local/pipx you will most likely not lose anything. Sorry.

[guihkx]

Don't worry, it's not an issue for me having to delete everything and install it again.

Thanks for the explanation too!

[kidpixo]

I'm having the same error as @guihkx . @itsayellow is right , I believe the only folder we need to delete is ~/.local/pipx/shared/ .

I think pipx will recreate them at the first (re)install, but I cannot prove it because I also ended up with reinstall-all uninstalling everything, but it is no big deal.

[makew0rld]

This occurred to me when upgrading from Debian 10 (Python 3.7) to Debian 11 (Python 3.9).

[saskenuba]

I'm having the same error as @guihkx . @itsayellow is right , I believe the only folder we need to delete is ~/.local/pipx/shared/ .

I think pipx will recreate them at the first (re)install, but I cannot prove it because I also ended up with reinstall-all uninstalling everything, but it is no big deal.

Had the same issue and this solved.

[georgek]

Happened to me when updating from Ubuntu 18.04 to 20.04.

[alecandido]

Therefore if you want to start over by deleting ~/.local/pipx you will most likely not lose anything. Sorry.

This worked for me

[kidpixo]

oops, I did it again after Arch upgraded python 3.9>3.10....

[makew0rld]

Is there a way pipx can prevent this from happening?

[makew0rld]

@itsayellow @pawamoy can this issue be re-opened? pipx still completely breaks when Python version is updated, on Arch at least. Even just running pipx doesn't work. Only the solution of uninstalling and installing pipx, deleting its directory, and then re-installing packages manually works.

[pawamoy]

I'll let the maintainers reopen if they believe a solution can be implemented in pipx itself. After a few days with no answer I'll reopen :slightly_smiling_face:

[pawamoy]

....aaaaaaaaaand misclicked.

[cs01]

I’ll leave it open for visibility, to show it’s a user facing issue, and because there are probably still improvements pipx can make here.

[dracos]

Just to note that I have just had this issue, and the PR mentioned above does not appear to have helped fix it (perhaps I did something wrong). I started off reinstalling pipx (because without it wouldn't run at all):

$ python3 -m pip install --user pipx
Collecting pipx
  Downloading pipx-1.0.0-py3-none-any.whl (54 kB)
     |████████████████████████████████| 54 kB 1.0 MB/s
Collecting userpath>=1.6.0
  Downloading userpath-1.7.0-py2.py3-none-any.whl (14 kB)
Requirement already satisfied: packaging>=20.0 in /usr/lib/python3/dist-packages (from pipx) (20.3)
Collecting argcomplete>=1.9.4
  Downloading argcomplete-2.0.0-py2.py3-none-any.whl (37 kB)
Requirement already satisfied: click in /usr/lib/python3/dist-packages (from userpath>=1.6.0->pipx) (7.0)
Installing collected packages: userpath, argcomplete, pipx
Successfully installed argcomplete-2.0.0 pipx-1.0.0 userpath-1.7.0

The first run of reinstall-all removed all the virtualenvs but didn't install anything, giving lots of e.g.

/home/matthew/.local/pipx/shared/bin/python: No module named pip
pipx.util.PipxError: '/home/matthew/.local/pipx/shared/bin/python -m pip --disable-pip-version-check install -q --upgrade pip setuptools wheel' failed
uninstalled sqlite-utils! ✨ 🌟 ✨
Fatal error from pip prevented installation.

And now a new run of it gives:

$ pipx reinstall-all --verbose
pipx >(setup:729): pipx version is 1.0.0
pipx >(setup:730): Default python interpreter is '/usr/bin/python3'
pipx >(upgrade:91): Upgrading shared libraries in /home/matthew/.local/pipx/shared
upgrading shared libraries...
pipx >(run_subprocess:172): running /home/matthew/.local/pipx/shared/bin/python -m pip --disable-pip-version-check install --upgrade pip setuptools wheel
/home/matthew/.local/pipx/shared/bin/python: No module named pip
pipx >(upgrade:119): Failed to upgrade shared libraries
Traceback (most recent call last):
  File "/home/matthew/.local/lib/python3.8/site-packages/pipx/shared_libs.py", line 113, in upgrade
    subprocess_post_check(upgrade_process)
  File "/home/matthew/.local/lib/python3.8/site-packages/pipx/util.py", line 202, in subprocess_post_check
    raise PipxError(
pipx.util.PipxError: '/home/matthew/.local/pipx/shared/bin/python -m pip --disable-pip-version-check install --upgrade pip setuptools wheel' failed

Looking, ~/.local/pipx/shared/lib/python3.8/site-packages/ is empty, but the old ~/.local/pipx/shared/lib/python3.6/site-packages/ still contains pip. Removing .local/pipx/shared and then it all runs fine (though as above, have to work out what I had installed previously from the log files). So it'd be good if the upgrade of the shared directory could work in this scenario.

[eode]

Happened for me again when upgrading from Ubuntu 20.04 to 22.04. Delete of ~/.local/pipx still works.

[remcoboerma]

Testing the upgrade of ubuntu 22.10 to 23.04 on 2 machines. Both required reinstall-all, one had the problem of not being able to install anything after. Manually removing .local/pipx/shared was required. Maybe a hint towards this solution would be beneficial to accompany the error message. Especially since 23.04 bars installation into system environment, upgrading python while already having a lot of pipx managed environments will be more common.

[remcoboerma]

tested on a new machine. only removing .local/pipx/shared and running pipx list solved it. no reinstall-all required.

[juxuanu]

Arch updated to Python 3.11.x and pipx broke. I deleted ~/.local/pipx and lost all packages. Moreover, pipx install pkg1 pkg2 pkg3 ... is not supported, so I had to install one by one...

[dimon222]

Arch updated to Python 3.11.x and pipx broke. I deleted ~/.local/pipx and lost all packages. Moreover, pipx install pkg1 pkg2 pkg3 ... is not supported, so I had to install one by one...

Technically you dont need to install stuff you can directly pipx run. It will download if needed.

[kidpixo]

@juxuanu same here. I just delete ~/.local/pipx/shared/ and run pipx reinstall-all.

For future reference : I think the problem is that pipx create venvs based on system python. If system python changes, all the references in the venvs are broken, starting with python, pip etc.

[juxuanu]

I think if pipx detects a python version change, it should prompt the user to reinstall all packages (and shared libraries) with the new version.

[kidpixo]

@juxuanu that's a good point! I was wrong in my other comment, the problem is that you always have a python executable, but the pip used to create the environments is different from the current one. Either way, we should able to catch this and offer to rebuild the whole structure.

[pglaser-engineer]

FWIW, I just upated Manjaro, which upgraded Python from 3.10 to 3.11, and had the same problem as @dimon222 and others have reported with Arch (not surprisingly). The only solution for me was to uninstall pipx, delete ~/.local/pipx, and reinstall pipx. That solved the problem.

[dkebler]

Given that the suggestion by @kidpixo works then maybe all that is required is for pipx reinstall-all to overwrite /shared (by default, or with prompt or flag).

Too maybe if pipx install fails from not finding pip module then pipx should prompt to run reinstall-all. Alternatively maybe pipx could store the version of python when it is installed and can check before running to see if it changed. In which case it could prompt for a reinstall. In any case there needs to be something to alert users that their python version changed and thus pipx is broken. why? so we all don't end up here in this issue :).