Python: configure venv on a per-subdirectory basis

[spkgyk]

Check for existing issues

• [X] Completed

Describe the bug / provide steps to reproduce it

Zed is unable to find the python installed when using an environment created by uv https://github.com/astral-sh/uv.

I tried giving the python extension a direct path to the python binary inside .venv/bin/ but nothing happened (no error, no message, no bug, just ignored my input).

Environment

Hardware Overview:

Model Name: MacBook Pro Model Identifier: Mac15,7 Chip: Apple M3 Pro Total Number of Cores: 12 (6 performance and 6 efficiency) Memory: 36 GB System Firmware Version: 11881.41.5 OS Loader Version: 11881.41.5

If applicable, add mockups / screenshots to help explain present your vision of the feature

No response

If applicable, attach your Zed.log file to this issue.

Zed.log

[osiewicz]

Can you provide clear repro step? venvs created with uv venv are detected just fine on my Mac. I'm using uv 0.4.20 (Homebrew 2024-10-08)

[spkgyk]

Im using uv 0.5.5 (Homebrew 2024-11-27).

I have a project in a folder (src). I run uv sync and it will create a src/.venv/ folder containing the python binary.

Note that the only python I have installed on my system is the default python that ships with Mac, I use uv to install and handle python versions.

When i select the python interpreter, writing the full path to src/.venv does not work, src/.venv/bin and src/.venv/bin/activate and src/.venv/bin/python also dont work. If I look at the symlink, the actual python binary located at /Users/myusername/.local/share/uv/python/cpython-3.12.6-macos-aarch64-none/bin/python3.12.

[osiewicz]

Yep, it looks like we don't descend into src/ in that case. It works for me when it is created at the root of a worktree tho.

[slckl]

I was trying to use a .venv setup by uv inside of a larger monorepo with multiple languages.

zed can pick up the .venv just fine, if I open a particular python subdirectory as project root, but there doesn't seem to be a way to load a specific .venv for a given subproject?

Simple workaround is to have multiple windows for each subproject, but it's akward to say the least...

[ion-elgreco]

Ran into the same issue while giving Zed a try, .venv's are only recognized when the .venv is in the root of the workspace. This is unpractical because you might have monorepos as @slckl points out.

[hongbo-miao]

Also, I found I cannot manually set my .venv path from uv 😔

[spkgyk]

To add to this problem a bit more, even if I type the exact path of the python executable, it still wont find it. Screenshot:

So it doesn't find my venv in src/data_layer/.venv.

Next, I try to type in the full path to my python executable:

No matches.

I understand that maybe you don't want to ripgrep subdirs for performance reasons, but not being able to choose a python path that exists even when I manually type it in is quite frustrating. It is quite common to be working in a code monolith with different python paths and subdirs as projects. I really think this support should be added.

Note that uv uses system links:

ls -lah /Users/cvaxh/Code/fateview/src/data_layer/.venv/bin/python lrwxr-xr-x 1 cvaxh staff 84B Mar 28 17:23 /Users/cvaxh/Code/fateview/src/data_layer/.venv/bin/python -> /Users/cvaxh/.local/share/uv/python/cpython-3.12.6-macos-aarch64-none/bin/python3.12

But BOTH paths do not work. I have to open a new window inside src/data_layer in order to use the environment I want.

I think this in parallel with the poor syntax highlighting for python and the difficulty in setting up the python auto-formatting/goto definition is really poor ergonomics. Hope this can be improved in the future!

[sebasv]

I have the same problem. I've been trying to configure the .venv location by setting the path in pyrightconfig.json or .zed/settings.json in one of the subdirectories but the env simply is not recognized. Only if I configure the path in a settings file in the project root do I get the env to be recognized, leaving me to have to constantly update my settings file. When will it be possible to configure a per-directory python venv?

[matthew-robb]

I have the same problem. I've been trying to configure the .venv location by setting the path in pyrightconfig.json or .zed/settings.json in one of the subdirectories but the env simply is not recognized. Only if I configure the path in a settings file in the project root do I get the env to be recognized, leaving me to have to constantly update my settings file. When will it be possible to configure a per-directory python venv?

I'm encountering the same issue. Would love to give zed a shot but need a fix for this

[ion-elgreco]

@osiewicz the release notes mentions this resolves this ticket, but it doesn't seem to work for me, it doesn't recognize any venvs in mono repos

[osiewicz]

Hey @ion-elgreco, I am sorry to say this was temporarily reverted in https://github.com/zed-industries/zed/pull/29676 as it caused issues for some of our users; I plan to re-land it next week.

[kallepyorala]

@osiewicz any news on this? Now that I got my debugger access this is the last thing making zed difficult to use as daily driver 😀

[ion-elgreco]

I think there is one more thing missing, @kallepyorala I am still switching back to VSCode frequently just for the jupyter support ^^

[strangemonad]

@osiewicz thanks so much for working on this. It's definitely one of the main issues preventing me from using zed more regularly (single zed workspace with multiple python projects).

A few thoughts:

• I'm curious if this is cleanly achievable without some broader support for multi-root workspaces? Specifically when the projects are in subdirectories of one of the workspace roots? One potentially workable temporary approach might be to use a .zed/settings.json in each location that has a pyproject.toml / python project root and allow manually configuring the the venv path. Not ideal, but unblocks things. Ideally the toolchain provider discovers all pyproject.toml roots.
• I took a look at #29676. I'm curious if setting the PET workspaces might work better https://github.com/zed-industries/zed/blob/c57e6bc784d116cbad0e040963db0c8243ea34c5/crates/languages/src/python.rs#L699 I believe this is the functionality vscode uses.

That being said, I appreciate just how messy python toolchain discovery and context is and until you have editor venv, linter config, type checking config, lsp config etc all working, the python experience feels broken to the end user. Given how complex this is in python (vs say rust with cargo), my hope is that there could be a standard crate used across python tooling that PythonContextProvider and PythonToolchainProvider could more or less just proxy to. PET, as it currently stands, isn't quite complete enough. I've filed related issues with astral https://github.com/astral-sh/uv/issues/2813 and https://github.com/astral-sh/ty/issues/526. There seems to be interest but it's a matter of priority. UV has also defined a consistent approach to interpreter discovery https://github.com/astral-sh/uv/issues/2386. I'd hope this common logic doesn't depend on a project using directly being configured for uv, just standard PEP-compliant pyproject.toml (dependencies and build backend are probably the 2 main PEPs to anchor on but there might be a way to have hooks for non-compliant dependency formats like poetry).

I'd imaging a flow to be more or less like the following:

• given a path to a file or dir, return Maybe[PythonContext + PythonToolchain] | Err.

This could handle cases like

• files (py, ipynb) with self-contained dependencies in yaml front-matter or metadata
• directories where you find the first parent pyproject.toml or .venv
• fallback to listing the system python versions

I'm not very fluent in rust, nor do I deeply understand the direction zed's going in for mono-repos so that's prevented me from just jumping in and sketching out code for this.

[strangemonad]

also, as a shorter term workaround, if there could be some way to explicitly pick a venv in an arbitrary workspace subfolder you could at least pick the right toolchain manually as you move from project to project in the workspace.

[osiewicz]

@strangemonad thanks for a thorough feedback. I kinda dropped the ball here, but I was knee-deep in a debugger/rustc work. As for your points:

1. We already have a multi-root workspace support (sorta) - each language can define how to divide a single Zed workspace into subprojects by finding "manifest files". In case of Python that'd be pyproject.toml.
2. Yes, admittedly we should augment workspace_folders as passed to PET with the path to the current project subroot. As for the tooling situation; yes, it's not ideal with Python, though I'd argue that PET is already a great step forward, as prior to that we've had very scarce support for anything Python-related.
3. As for manually picking a toolchain, I believe @probably-neb might've worked on something like this? Not sure.

[probably-neb]

I have not worked on manually selecting a toolchain, however, I believe it is something we should do, at the very least to give people an escape hatch when running into issues like have been described in this chain.

[strangemonad]

We already have a multi-root workspace support (sorta) - each language can define how to divide a single Zed workspace into subprojects by finding "manifest files". In case of Python that'd be pyproject.toml.

@osiewicz hmm, is there a way to debug what project context's / worktree manifests are being discovered and provided? I don't think this has every worked for me even in the simplest of setups (eg single workspace with 2 sibling dirs each containing a pyproject).

[osiewicz]

Not at the moment, no; I can add logs though.

[strangemonad]

@osiewicz related request / question (and probably a separate issue if you think it's worth having). I think any ability to debug the state of the workspace would come in handy e.g. I tried running an echo task echo $ZED_RELATIVE_FILE and echo $ZED_WORKTREE_ROOT. And those point to the workspace's root but there doesn't seem to be any alternative to infer project relative or project manifest root? I could see possibly like exposing the server_tree's InnerTreeNode info in the dev keystroke window (which already shows a ton of context info) and more broadly, being able to reference the project manifest root in settings (eg setting a python path as something like $PROJECT_ROOT/.venv/bin/python).

Maybe some of this is already there and just not documented so i'm missing it? (sorry for the eagerness, I just really want to figure out how to get zed minimally functional for my python workflow)

[osiewicz]

In case of Python there's ZED_CUSTOM_PYTHON_ACTIVE_ZED_TOOLCHAIN env variable that you can use to query the path to an active toolchain. Right now we don't expose path to what we think is a project subroot - while this would be vital, exposing things as task variables is a bit of an API commital (we don't want to add/remove task variables at will if we're not sure about the stability of the origin of data). TL;DR: yes, we could expose these in task variables, but we (well, I) want to be sure that the current API is good enough to not pull a rug under people who'd depend on these variables.

As for the better inspector though, sure, that'd be lovely.

[sygi]

Do I understand this thread correctly that settings multiple venvs should be now supported by setting them up in separate pyproject.toml? Can you show an example on how this would be expressed? Fwiw, I'm only interested in setting up a single venv but haven't managed to make it work for a non-trivial path (outside of pyright etc).

[sbj-ee]

I was attempting to point zed to my python venv (outside of my project directory) using an absolute path and it wouldn't use it. I got enough clues from above and just created a soft link to a .venv at the root of my project and used the path to that soft link and viola. Simple enough to do, but it wasn't obvious that the venv needed to be at the root of my project.