topgrade
topgrade copied to clipboard
Published
20 hours ago •
r-darwish
r-darwish
pip3 broken?
What did you expect to happen?
Python packages should have updated. Running pip3 list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip3 install -U works without issue.
What actually happened?
―― 09:38:23 - pip3 ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
Requirement already satisfied: pip in ./Library/Python/3.9/lib/python/site-packages (22.2)
Collecting pip
Using cached pip-22.2.1-py3-none-any.whl (2.0 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 22.2
Uninstalling pip-22.2:
Successfully uninstalled pip-22.2
Rolling back uninstall of pip
Moving to /Users/daniel/Library/Python/3.9/bin/
from /Users/daniel/Library/Python/3.9/~in
Moving to /Users/daniel/Library/Python/3.9/lib/python/site-packages/pip-22.2.dist-info/
from /Users/daniel/Library/Python/3.9/lib/python/site-packages/~ip-22.2.dist-info
Moving to /Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/
from /Users/daniel/Library/Python/3.9/lib/python/site-packages/~ip
ERROR: Exception:
Traceback (most recent call last):
File "/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
status = run_func(*args)
File "/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
return func(self, options, args)
File "/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/commands/install.py", line 461, in run
installed = install_given_reqs(
File "/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/req/__init__.py", line 73, in install_given_reqs
requirement.install(
File "/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/req/req_install.py", line 752, in install
scheme = get_scheme(
File "/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/locations/__init__.py", line 244, in get_scheme
from . import _distutils
ImportError: cannot import name '_distutils' from 'pip._internal.locations' (/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/locations/__init__.py)
Retry? (y)es/(N)o/(s)hell/(q)uit%
Additional Details
- Which operating system or Linux distribution are you using? Mac OS 12.5
- How did you install Topgrade? Homebrew
- Which version are you running? 9.0.1
This won't help because I ran it after I manually upgraded the packages:
09:40:59 AM ✦2 » topgrade --dry-run --only pip3
―― 09:41:18 - pip3 ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
Dry running: /opt/homebrew/bin/python3 -m pip install --upgrade --user pip
―― 09:41:18 - Summary ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
pip3: OK
Same here:
2022-07-28T13:41:42.419Z DEBUG topgrade::config > Configuration at /Users/daniel/.config/topgrade.toml
2022-07-28T13:41:42.419Z DEBUG topgrade::config > Loaded configuration: ConfigFile { pre_commands: Some({}), post_commands: None, commands: Some({}), git_repos: None, predefined_git_repos: None, disable: None, ignore_failures: None, remote_topgrades: None, remote_topgrade_path: None, ssh_arguments: None, git_arguments: None, tmux_arguments: None, set_title: None, display_time: None, assume_yes: None, yay_arguments: None, no_retry: None, run_in_tmux: None, cleanup: None, notify_each_step: None, accept_all_windows_updates: None, bashit_branch: None, only: None, composer: Some(Composer { self_update: None }), brew: Some(Brew { greedy_cask: None }), linux: Some(Linux { yay_arguments: None, arch_package_manager: None, show_arch_news: None, trizen_arguments: None, pikaur_arguments: None, dnf_arguments: None, apt_arguments: None, enable_tlmgr: None, redhat_distro_sync: None, rpm_ostree: None, emerge_sync_flags: None, emerge_update_flags: None }), git: Some(Git { max_concurrency: None, arguments: None, repos: None, pull_predefined: None }), windows: Some(Windows { accept_all_updates: None, self_rename: None, open_remotes_in_new_terminal: None, enable_winget: None }), npm: Some(NPM { use_sudo: None }), vim: None, firmware: Some(Firmware { upgrade: None }), vagrant: None, flatpak: Some(Flatpak { use_sudo: None }) }
2022-07-28T13:41:42.419Z DEBUG topgrade > Version: 9.0.1
2022-07-28T13:41:42.419Z DEBUG topgrade > OS: aarch64-apple-darwin
2022-07-28T13:41:42.419Z DEBUG topgrade > Args { inner: ["topgrade", "--only", "pip3", "-v"] }
2022-07-28T13:41:42.419Z DEBUG topgrade > Binary path: Ok("/opt/homebrew/bin/topgrade")
2022-07-28T13:41:42.419Z DEBUG topgrade > Self Update: false
2022-07-28T13:41:42.419Z DEBUG topgrade::utils > Detected "/opt/homebrew/bin/git" as "git"
2022-07-28T13:41:42.420Z DEBUG topgrade::utils > Cannot find "doas"
2022-07-28T13:41:42.420Z DEBUG topgrade::utils > Detected "/usr/bin/sudo" as "sudo"
2022-07-28T13:41:42.420Z DEBUG topgrade::utils > Detected "/usr/local/bin/pwsh" as "pwsh"
2022-07-28T13:41:42.730Z TRACE topgrade::executor > Output of "/usr/local/bin/pwsh" "-NoProfile" "-Command" "Split-Path $profile": Output { status: ExitStatus(unix_wait_status(0)), stdout: "/Users/daniel/.config/powershell\n", stderr: "" }
2022-07-28T13:41:42.730Z DEBUG topgrade::utils > Path "/Users/daniel/.config/emacs" doesn't exist
2022-07-28T13:41:42.730Z DEBUG topgrade::utils > Path "/Users/daniel/.emacs.d" doesn't exist
2022-07-28T13:41:42.730Z DEBUG topgrade::steps::git > /Users/daniel/.ideavimrc does not exists
2022-07-28T13:41:42.730Z DEBUG topgrade::steps::git > /Users/daniel/.intellimacs does not exists
2022-07-28T13:41:42.730Z DEBUG topgrade::steps::git > /Users/daniel/.config/.zshrc is a file. Checking /Users/daniel/.config
2022-07-28T13:41:42.730Z DEBUG topgrade::steps::git > Checking if /Users/daniel/.config is a git repository
2022-07-28T13:41:42.734Z TRACE topgrade::executor > Output of "/opt/homebrew/bin/git" "rev-parse" "--show-toplevel": Output { status: ExitStatus(unix_wait_status(32768)), stdout: "", stderr: "fatal: not a git repository (or any of the parent directories): .git\n" }
2022-07-28T13:41:42.734Z DEBUG topgrade::steps::git > Checking if /Users/daniel/.config/fish is a git repository
2022-07-28T13:41:42.737Z TRACE topgrade::executor > Output of "/opt/homebrew/bin/git" "rev-parse" "--show-toplevel": Output { status: ExitStatus(unix_wait_status(32768)), stdout: "", stderr: "fatal: not a git repository (or any of the parent directories): .git\n" }
2022-07-28T13:41:42.737Z DEBUG topgrade::steps::git > /Users/daniel/Library/Application Support/openbox does not exists
2022-07-28T13:41:42.737Z DEBUG topgrade::steps::git > /Users/daniel/Library/Application Support/bspwm does not exists
2022-07-28T13:41:42.737Z DEBUG topgrade::steps::git > /Users/daniel/Library/Application Support/i3 does not exists
2022-07-28T13:41:42.737Z DEBUG topgrade::steps::git > /Users/daniel/Library/Application Support/sway does not exists
2022-07-28T13:41:42.737Z DEBUG topgrade::runner > Step "pip3"
2022-07-28T13:41:42.737Z DEBUG topgrade::utils > Detected "/opt/homebrew/bin/python3" as "python3"
2022-07-28T13:41:42.876Z TRACE topgrade::executor > Output of "/opt/homebrew/bin/python3" "-m" "pip": Output { status: ExitStatus(unix_wait_status(0)), stdout: "\nUsage: \n /opt/homebrew/opt/[email protected]/bin/python3.9 -m pip <command> [options]\n\nCommands:\n install Install packages.\n download Download packages.\n uninstall Uninstall packages.\n freeze Output installed packages in requirements format.\n inspect Inspect the python environment.\n list List installed packages.\n show Show information about installed packages.\n check Verify installed packages have compatible dependencies.\n config Manage local and global configuration.\n search Search PyPI for packages.\n cache Inspect and manage pip's wheel cache.\n index Inspect information available from package indexes.\n wheel Build wheels from your requirements.\n hash Compute hashes of package archives.\n completion A helper command used for command completion.\n debug Show information useful for debugging.\n help Show help for commands.\n\nGeneral Options:\n -h, --help Show help.\n --debug Let unhandled exceptions propagate outside the\n main subroutine, instead of logging them to\n stderr.\n --isolated Run pip in an isolated mode, ignoring\n environment variables and user configuration.\n --require-virtualenv Allow pip to only run in a virtual environment;\n exit with an error otherwise.\n -v, --verbose Give more output. Option is additive, and can be\n used up to 3 times.\n -V, --version Show version and exit.\n -q, --quiet Give less output. Option is additive, and can be\n used up to 3 times (corresponding to WARNING,\n ERROR, and CRITICAL logging levels).\n --log <path> Path to a verbose appending log.\n --no-input Disable prompting for input.\n --proxy <proxy> Specify a proxy in the form\n scheme://[user:passwd@]proxy.server:port.\n --retries <retries> Maximum number of retries each connection should\n attempt (default 5 times).\n --timeout <sec> Set the socket timeout (default 15 seconds).\n --exists-action <action> Default action when a path already exists:\n (s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort.\n --trusted-host <hostname> Mark this host or host:port pair as trusted,\n even though it does not have valid or any HTTPS.\n --cert <path> Path to PEM-encoded CA certificate bundle. If\n provided, overrides the default. See 'SSL\n Certificate Verification' in pip documentation\n for more information.\n --client-cert <path> Path to SSL client certificate, a single file\n containing the private key and the certificate\n in PEM format.\n --cache-dir <dir> Store the cache data in <dir>.\n --no-cache-dir Disable the cache.\n --disable-pip-version-check\n Don't periodically check PyPI to determine\n whether a new version of pip is available for\n download. Implied with --no-index.\n --no-color Suppress colored output.\n --no-python-version-warning\n Silence deprecation warnings for upcoming\n unsupported Pythons.\n --use-feature <feature> Enable new functionality, that may be backward\n incompatible.\n --use-deprecated <feature> Enable deprecated functionality, that will be\n removed in the future.\n", stderr: "" }
―― 09:41:42 - pip3 ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
2022-07-28T13:41:42.876Z DEBUG topgrade::executor > Running "/opt/homebrew/bin/python3" "-m" "pip" "install" "--upgrade" "--user" "pip"
Requirement already satisfied: pip in /opt/homebrew/lib/python3.9/site-packages (22.2.1)
―― 09:41:43 - Summary ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
pip3: OK
2022-07-28T13:41:43.361Z DEBUG topgrade::terminal > Desktop notification: Topgrade finished successfully
I experienced the same error.
I understand that it is a pip error, but if topgrade would update pip first (python3 -m pip install --upgrade pip), it would prevent the users from experiencing this error in the future.
Requirement already satisfied: pip in ./.local/lib/python3.8/site-packages (22.2)
Collecting pip
Using cached pip-22.2.1-py3-none-any.whl (2.0 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 22.2
Uninstalling pip-22.2:
Successfully uninstalled pip-22.2
Rolling back uninstall of pip
Moving to /home/toshihiro/.local/bin/pip
from /tmp/pip-uninstall-37ltymkz/pip
Moving to /home/toshihiro/.local/bin/pip3
from /tmp/pip-uninstall-37ltymkz/pip3
Moving to /home/toshihiro/.local/bin/pip3.8
from /tmp/pip-uninstall-37ltymkz/pip3.8
Moving to /home/toshihiro/.local/lib/python3.8/site-packages/pip-22.2.dist-info/
from /home/toshihiro/.local/lib/python3.8/site-packages/~ip-22.2.dist-info
Moving to /home/toshihiro/.local/lib/python3.8/site-packages/pip/
from /home/toshihiro/.local/lib/python3.8/site-packages/~ip
ERROR: Exception:
Traceback (most recent call last):
File "/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
status = run_func(*args)
File "/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
return func(self, options, args)
File "/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 461, in run
installed = install_given_reqs(
File "/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/req/__init__.py", line 73, in install_given_reqs
requirement.install(
File "/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 752, in install
scheme = get_scheme(
File "/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/locations/__init__.py", line 244, in get_scheme
from . import _distutils
ImportError: cannot import name '_distutils' from 'pip._internal.locations' (/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/locations/__init__.py)
Are you sure? By my understanding, upgrading pip first would just make it fail sooner. I don't think it's within topgrade's scope to account for unexpected behavior of tools like pip.
Since topgrade is doing carefully rustup before the cargo update, it would be nice to have the same behavior for pip, by updating the pip itself before the pip update. However, I also understand the policy of not addressing it because it is a pip-specific error, as you mentioned.
By my understanding, rustup runs so any following cargo builds don't fail due to an outdated toolchain. If the same can be said about pip (i wouldn't know), that's a valid argument. But this issue is more akin to rustup update failing with a segfault - at which point we run it is not really relevant, it's an upstream issue either way.
In my experience, there have been several times when the latest version of some library failed to install because the pip was out of date. So, in my mind, if pip fails, the right thing to do is to upgrade pip itself.
However, it seems that the upgrade of pip itself can cause problems in a Windows environment, although I don't use Windows very often. https://github.com/pypa/pip/issues/9395
So, I reconsider that it may be a hasty decision to include the upgrade function of pip itself in topgrade as a default behavior.
Until this has been fixed in pip 22.2.2, you can update pip manually using this command (from https://github.com/pypa/pip/issues/11319#issuecomment-1200119836).
python -m pip install --user --upgrade https://github.com/pypa/pip/archive/refs/tags/22.2.1.zip
# python3 -m pip install --user --upgrade https://github.com/pypa/pip/archive/refs/tags/22.2.1.zip
