pipx icon indicating copy to clipboard operation
pipx copied to clipboard
verified publisher icon pypa

Make pipx more automation friendly, like pip

Open gaby opened this issue 1 year ago • 7 comments

Describe the bug

I currently manage a fleet of hosts configured with Ansible, and we’ve recently encountered a new challenge with newer operating systems. We're now required (PEP 668) to install applications globally for multiple users, which is where pipx comes in, using the --global flag. Previously, we could set the version of a Python application as an Ansible variable, and pip would handle the installation smoothly when the version changed. However, after migrating to pipx, this behavior has changed. Pipx won’t allow version updates unless the --force flag is used, which isn't ideal.

If our playbooks run daily, using --force becomes inefficient since it reinstalls the entire application every time. Pipx should ideally support version updates by automatically upgrading the app instead of simply stating that it’s already installed.

The same happens with using pipx inject, there's no way to upgrade an injected app.

How to reproduce

Example of simple pipx install, and changing version:

root@3a7829e7364c:/# pipx --version
1.7.1
root@3a7829e7364c:/# pipx install --global ruff==0.6.8
  installed package ruff 0.6.8, installed using Python 3.12.7
  These apps are now globally available
    - ruff
done! ✨ 🌟 ✨
root@3a7829e7364c:/# pipx install --global ruff==0.6.9
'ruff' already seems to be installed. Not modifying existing installation in
'/opt/pipx/venvs/ruff'. Pass '--force' to force installation.
root@3a7829e7364c:/#

Example pipx inject with issue:

root@3a7829e7364c:/# pipx install --global uv --include-deps
  installed package uv 0.4.20, installed using Python 3.12.7
  These apps are now globally available
    - uv
    - uvx
done! ✨ 🌟 ✨
root@3a7829e7364c:/# pipx inject --global uv ruff==0.6.8 --include-apps --include-deps
  installed package ruff 0.6.8, installed using Python 3.12.7
  These apps are now globally available
    - ruff
    - uv
    - uvx
done! ✨ 🌟 ✨
  injected package ruff into venv uv
done! ✨ 🌟 ✨
root@3a7829e7364c:/# pipx inject --global uv ruff==0.6.9 --include-apps --include-deps
⚠️ ruff already seems to be injected in 'uv'. Not modifying existing installation in '/opt/pipx/venvs/uv'. Pass '--force' to force installation.
root@3a7829e7364c:/#

Now if i try using pipx upgrade, it doesn't support specifying a spec, and doesn't work at all:

root@3a7829e7364c:/# pipx upgrade --global uv ruff==0.6.9
uv is already at latest version 0.4.20 (location: /opt/pipx/venvs/uv)
Package is not installed. Expected to find /opt/pipx/venvs/ruff==0-6-9, but it does not exist.

Expected behavior

Example pip install with a new version, works out of the box.

root@3a7829e7364c:/# pip install ruff==0.6.8
Collecting ruff==0.6.8
  Using cached ruff-0.6.8-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (25 kB)
Using cached ruff-0.6.8-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.7 MB)
Installing collected packages: ruff
Successfully installed ruff-0.6.8
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to                                                            use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
root@3a7829e7364c:/# pip install ruff==0.6.9
Collecting ruff==0.6.9
  Downloading ruff-0.6.9-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (25 kB)
Downloading ruff-0.6.9-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.8/10.8 MB 23.6 MB/s eta 0:00:00
Installing collected packages: ruff
  Attempting uninstall: ruff
    Found existing installation: ruff 0.6.8
    Uninstalling ruff-0.6.8:
      Successfully uninstalled ruff-0.6.8
Successfully installed ruff-0.6.9
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
root@3a7829e7364c:/#

We have spent multiple days trying to figure out how to migrate our automated setup from pip to pipx. It shouldn't be this hard.

tldr

  • Updating installed applications is not user friendly.
  • Updating injected applications is not user friendly.
  • pipx is not consistent across commands in terms of spec. Some commands take a version, some don't.
  • pipx upgrade doesn't support specifying package version.
  • pipx is a nightmare to use with any automation tool that will update versions.

Potential Solutions

  • Adding a --upgrade param to pipx install that will use the spec to auto-upgrade the package. Similar to what pip --upgrade <spec> does.
  • Add support for <spec> to pipx upgrade.
  • Add support for upgrading injected apps via pipx inject --upgrade, similar to point 1 on this list of Potential Solutions

gaby avatar Oct 12 '24 04:10 gaby

Thanks for the feedback, I agree that some parts of the application can be improved to be more user-friendly.

Here are some of the alternative solutions that I would recommend:

Adding a --upgrade param to pipx install that will use the spec to auto-upgrade the package. Similar to what pip --upgrade does.

pipx upgrade --install <spec>

Add support for upgrading injected apps via pipx inject --upgrade, similar to point 1 on this list of Potential Solutions

pipx upgrade --include-injected <package>

dukecat0 avatar Oct 16 '24 12:10 dukecat0

Thanks for the feedback, I agree that some parts of the application can be improved to be more user-friendly.

Here are some of the alternative solutions that I would recommend:

Adding a --upgrade param to pipx install that will use the spec to auto-upgrade the package. Similar to what pip --upgrade does.

pipx upgrade --install <spec>

Add support for upgrading injected apps via pipx inject --upgrade, similar to point 1 on this list of Potential Solutions

pipx upgrade --include-injected <package>

@dukecat0 Your first suggestion does solve one problem. The 2nd suggestion doesnt allow using <spec> which a deal breaker.

My team manages/installs several python applications/packages for a fleet of hosts. We keep a tight boat by controlling which versions are installed via ansible group variables. The playbooks run automatically on each host every day to maintain a consistent fleet of hosts.

Being able to specify versions allows us to revert a deployment in case an application has a bug on their release.

Given my example, I would like to be able to upgrade ruff to a specific version while its injected into uv.

gaby avatar Oct 16 '24 13:10 gaby

Another issue I noticed is that pipx list doesnt show versions of injected packages. There's actually 0 way of finding this information.

This one is a known issue since 2023, https://github.com/pypa/pipx/issues/997

gaby avatar Oct 16 '24 13:10 gaby

pipx is a nightmare to use with any automation tool that will update versions.

I am curious about this. I tried installing and/or upgrading my packages with threading in a custom python script so it can take less time, but I wasn't successful. I think this is due to the shared directory but I am not sure. I ran into odd issues. What sort of automation tools are you using?

powercoconola avatar Oct 17 '24 02:10 powercoconola

pipx is a nightmare to use with any automation tool that will update versions.

I am curious about this. I tried installing and/or upgrading my packages with threading in a custom python script so it can take less time, but I wasn't successful. I think this is due to the shared directory but I am not sure. I ran into odd issues. What sort of automation tools are you using?

Ansible, which has a "pipx" module.

https://docs.ansible.com/ansible/latest/collections/community/general/pipx_module.html

gaby avatar Oct 17 '24 02:10 gaby

For example this won't work with plain pipx and/or ansible:

---
- name: Install tox
  community.general.pipx:
    name: tox==1.0.0

Then running the same thing with ==2.0.0, it just fails. With pip it works by properly upgrading the package.

gaby avatar Oct 17 '24 02:10 gaby

Another issue that makes pipx not friendly is the fact it doesnt use System Certificates like pip does. This causes locally issued certs to not be trusted even though the System and Pip trust them.

This has been extensively documented here https://github.com/pypa/pipx/issues/961

gaby avatar Oct 17 '24 12:10 gaby