uv icon indicating copy to clipboard operation
uv copied to clipboard

Allow creating a `python` shim on `python install`

Open zanieb opened this issue 1 year ago • 14 comments

When installing Python, I should be able to opt-in to adding a python shim to my PATH.

zanieb avatar Aug 20 '24 18:08 zanieb

not sure this was the original intention, but if the shim will work the same as uv run (in terms of environment resolution) it would be very useful

this will allow simpler integration with IDEs

(instead of explicitly providing a venv, they could be pointed to the uv shim that will do the environment resolution)

ophiry avatar Aug 23 '24 09:08 ophiry

This would be very useful, and is also what is missing in uv compared to pyenv.

tux-type avatar Aug 25 '24 09:08 tux-type

Please just 👍 the original post if you want this. If you have commentary on the implementation or behavior of the shim, that's totally welcome but let's keep the noise down for those subscribing to updates.

not sure this was the original intention, but if the shim will work the same as uv run (in terms of environment resolution) it would be very useful

We don't think we can do this by default because people have specific expectations about python behavior but I think an opt-in mode makes sense.

zanieb avatar Aug 26 '24 16:08 zanieb

This would be a great improvement for the entire Python ecosystem user-friendliness. Currently we manage Python projects like this where I work:

  1. Install pyenv.
  2. Install Python versions from source, compiling them (this is a huge pain especially for nontechnical users which get stuck with environment issues and compilation errors).
  3. Use Poetry which automatically detects the correct Python version from the constraint in pyproject.toml.

Having uv install binary version of Python with pyenv-style shims would allow us to entirely drop pyenv and remove the burden of compiling Python from the users... even if we don't switch to uv for package management. 😄

pietrodn avatar Sep 08 '24 11:09 pietrodn

I want to migrate from rye to uv, but the migration is blocked by this issue. This feature is especially wanted by Windows users because Windows doesn't bundle Python. If you run python command without a global Python, an annoying Microsoft Store window is launched.

tats-u avatar Sep 23 '24 12:09 tats-u

I am currently using this workaround to expose uv's Pythons to my shell's PATH:

for dir in $(uv python dir)/*/bin; do PYTHON_PATHS="$dir:$PYTHON_PATHS"; done
export PATH="$PYTHON_PATHS:$PATH"

The problem is that there are multiple python3.10, python3.11, ... executables, only one version for each minor would be exposed. But I typically pick only one version for each Python minor version.

pietrodn avatar Sep 23 '24 12:09 pietrodn

As a workaround, I wrote the following script python3.x:

#!/usr/bin/env bash
set -euo pipefail

SCRIPT_NAME="$(basename "${BASH_SOURCE[0]}")"

if [[ "${SCRIPT_NAME}" =~ ^python3\.[0-9]+$ ]]; then
  PYTHON_VERSION="${SCRIPT_NAME#python}"
else
  2>&1 echo "Error: Invalid script name: ${SCRIPT_NAME}"
  exit 1
fi

exec uv run --no-project --python "${PYTHON_VERSION}" --python-preference only-managed --no-config python "$@"

And I have symlinks python3.x pointing to this script:

~/.local/bin/python3.8  -> ~/.local/share/python3.x
~/.local/bin/python3.9  -> ~/.local/share/python3.x
~/.local/bin/python3.10 -> ~/.local/share/python3.x
~/.local/bin/python3.11 -> ~/.local/share/python3.x
~/.local/bin/python3.12 -> ~/.local/share/python3.x

It's not great, but it works.

vivienm avatar Sep 23 '24 13:09 vivienm

My like was 101, so this feature is in high demand. How can we communicate this to developers?

chaush-server avatar Oct 31 '24 11:10 chaush-server

Checkout #8458 - the functionality is nearly there

garyj avatar Oct 31 '24 11:10 garyj

We agree this is important, we're working on it. There's also a prototype at #7677.

zanieb avatar Oct 31 '24 13:10 zanieb

Note that there's a version of this available in preview mode (i.e. uv python install --preview) via #8458 and there's ongoing work in #8650

zanieb avatar Nov 07 '24 01:11 zanieb

Are there any plans to add PEP 514 support so that the Python executables are discoverable by the Windows py launcher?

weihenglim avatar Nov 07 '24 02:11 weihenglim

@weihenglim yes there are plans for that

zanieb avatar Nov 07 '24 03:11 zanieb

This is so great!

I have use cases that make needing to type uv run every time I want something to use the uv installed Python problematic.

Putting a 'python' executable in $HOME/.local/bin works perfectly for my use cases. Super appreciate the hard work!

(In particular we use poetry for managing our python runtime environments and I couldn't get that working with uv run but it works great with the python installed using --preview --default).

feoh avatar Dec 16 '24 19:12 feoh

We agree this is important, we're working on it. There's also a prototype at #7677.

I'm happy to see this is underway!

In the meantime, is there a reason why a simple alias would be a bad idea?

echo 'alias python="uv run python"'  >> ~/.zshrc

I don't have a system python installation, I'll just use uv to manage it, like I did with pyenv global before switching to uv and uninstalling pyenv.

EDIT:

I think what I want would be better solved by doing e.g.

uv python install 3.12 --preview --default

which installs the executables in my .local/bin directory

~/.local/bin/
├── python -> ~/.local/share/uv/python/...bin/python3.12
├── python3 -> ~/.local/share/uv/python/.../bin/python3.12
└── python3.12 -> ~/.local/share/uv/python/.../bin/python3.12

postylem avatar Feb 26 '25 15:02 postylem

Yeah aliases are tricky because they require a shell.

zanieb avatar Feb 26 '25 16:02 zanieb

not sure this was the original intention, but if the shim will work the same as uv run (in terms of environment resolution) it would be very useful

this will allow simpler integration with IDEs

(instead of explicitly providing a venv, they could be pointed to the uv shim that will do the environment resolution)

I'm not a heavy user, but shims like rye as mentioned above may indeed benefit me. Here's my use case: I have some projects using pyo3 with rust-analyser in vscode. Now using uv, I have to try to reset the global python or start virtualenv manually every time I switch projects.

thanksshu avatar Feb 28 '25 08:02 thanksshu

I'd certainly like this feature! Would it be possible to install packages in this python install?

I often use interactive python sessions, or one-off scripts, where spinning up a new throw-away venv is a bit annoying. It's convenient to have a "general purpose" python install with a few packages for use cases like this. If it ever gets broken for whatever reason you throw it away and start a new one.

I assume it might not be possible since it wouldn't be linked to a venv, but I wanted to throw this use case out there.

auxym avatar Feb 28 '25 15:02 auxym

I often use interactive python sessions, or one-off scripts, where spinning up a new throw-away venv is a bit annoying. It's convenient to have a "general purpose" python install with a few packages for use cases like this.

@auxym have you tried something like uv tool run --with pandas ptpython ? it creates a temporary venv with specified packages installed and start an interactive session in ptpython. Or you just don't want venv to be created each time?

Winand avatar Feb 28 '25 16:02 Winand

👍 uvx --with ... python is the recommendation for that

zanieb avatar Feb 28 '25 16:02 zanieb

I was also a huge fan of the automatic shims with rye and being able to perform only a python in a project was great. You can achieve almost the same with something like mise and with the following mise.toml file in your project:

[env]
_.python.venv = ".venv" # relative to this file's directory

For global tools, like others said, I go with uvx --with ...

clement-tourriere avatar Mar 03 '25 09:03 clement-tourriere

👍 uvx --with ... python is the recommendation for that

To be perfectly clear, it's perfectly fine if uv doesn't support my use case. I understand it might not really fit with the philosophy.

But just for further explanation, uvx --with wouldn't really work for me. First, if I need, say, matplotlib pandas numpy jupyter ipympl nptdms, it's a bit of a hassle to type all that. Second, if I start an interactive ipython session or jupyter, then halfway through realize that I forgot scipy, I have to kill everything and start it back up with scipy. In a normal venv, I can just uv add or pip install scipy then import it without having to restart any process.

I'll probably just keep doing what I'm doing, which is a venv in .local which is added to my path.

But typing all this out... I might consider adding a shell alias that expands to uvx --with ... for all the packages I commonly use.

auxym avatar Mar 03 '25 12:03 auxym

Is the intention to also install a pip shim as well? The current preview only creates a symlink for python.

$ uv --version
uv 0.6.9
$ uv python install 3.12 --default --preview                                
Installed Python 3.12.9 in 6ms
 + cpython-3.12.9-linux-x86_64-gnu (python, python3, python3.12)
$ ls -1 ~/.local/bin | grep -E (pip|python)
python
python3
python3.12

I appreciate the intended workflow for projects is to use venvs where possible, but when using uv to install a "global" Python with no system Python it can be handy to be able to install things like yt-dlp or personal Python programs and have them always be just readily available in interactive shells and scripts.

xM8WVqaG avatar Mar 24 '25 21:03 xM8WVqaG

Therefore you can use uv tool install yt-dlp

mbeijen avatar Mar 24 '25 21:03 mbeijen

Is the intention to also install a pip shim as well? The current preview only creates a symlink for python.

$ uv --version uv 0.6.9 $ uv python install 3.12 --default --preview
Installed Python 3.12.9 in 6ms

  • cpython-3.12.9-linux-x86_64-gnu (python, python3, python3.12) $ ls -1 ~/.local/bin | grep -E (pip|python) python python3 python3.12 I appreciate the intended workflow for projects is to use venvs where possible, but when using uv to install a "global" Python with no system Python it can be handy to be able to install things like yt-dlp or personal Python programs and have them always be just readily available in interactive shells and scripts.

My case is that besides those python projects, I also have a lot of build scripts, crawlers, automation scripts scattering everywhere. They are all very simple and can be written in one file, and they all share the same dependencies like numpy or requests. Before the 3.11 version of python, I could just install dependencies using pip. After that, the system-wide python packages are managed by system package managers like apt or brew. So, if a dependency is not shipped with apt or brew, I have to turn to a virtual environment.

Because they belongs to non-python projects, e.g. build scripts, I don't want to put them in some uv project. They all share the same dependencies, so I don't want to prepare a virtual environment for each of them due to the unnecessary usage of disk spaces. (The size of numpy is around 50M, which is even larger than the data I want to manipulate for most of the cases.)

My solution is to make a global virtual environment by activating it at .zshrc. This method is certainly not good because when I activate another virtual environment and deactivate it, I will also deactivate the global one, and uv also complains about that if I forget to deactivate the global environment.

Previously, I was using pyenv to create python shims, with pip working as before. So, I can manage system-wide python packages by pip. But I also had to install poetry or pipx for project management and python-shipped tools. Besides, pyenv sticks to compiling from source when installing a python, which I think is unnecessary. (This is why I also don't want to compile and install python manually from source.)

In a word, there does exist the need to maintain a global or system-wide python environment (e.g., conda is something like this). I really hope you can think of adding pip to shim and it is then the user's responsibility to resolve correct versions or create virtual environments.

The case for cargo or npm is different, because the outcome of such a project is something other than source code. For example, cargo makes an executable binary/shared-object that doesn't rely on the dependencies of source code. Thus, we don't need a global environment for them, which is not the case for python as we run a python script (source code) mostly.

fduxiao avatar May 06 '25 20:05 fduxiao

@fduxiao

My case is that besides those python projects, I also have a lot of build scripts, crawlers, automation scripts scattering everywhere.

This is major use case for me too. Using python for utilities, shell scripts and other CLI tools.

leejoramo-d51 avatar May 06 '25 22:05 leejoramo-d51

@fduxiao

That's similar to the use case I describe in my previous comments, I believe. What currently do, and works well for me, is a "default" venv similar to what you describe, managed by uv, in my $HOME/.local dir. However, I don't activate it with the normal activate scripts, I just add its bin dir to my PATH in my shell init file. Packages in that venv are easily managed with uv pip .... You can activate other venvs that override your "default" as you normally would with a "system" python install.

auxym avatar May 07 '25 21:05 auxym

Is there a way to undo --preview or --default without completely reinstalling Python or just removing them from .local/bin should do and uv doesn't use preview and default python instances anywhere else?

Andrej730 avatar May 23 '25 18:05 Andrej730