Support `pip` packages for `--no-build-isolation`

[calebho]

Problem description

Currently, only Conda dependencies can be used to define the build environment when no build isolation is desired. One common use case for disabling build isolation is for building PyTorch CUDA extensions which implies projects with such extensions must install PyTorch with Conda. However, PyTorch 2.5 is the last version to officially support the pytorch Conda channel. While there are pytorch-{cpu,gpu} packages on conda-forge,

please do note that not all gaps, like full Windows support, will be closed and should not be viewed in the short run as a 1:1 offering with our current Anaconda offering.

In the interest of using the best supported PyTorch builds, would it be possible to use pip dependencies to define the build environment as well?

[nichmor]

Hey! Thanks for opening this issue!

In my opinion, we could support this. We need to think about how to declare the pip build environment and how to handle it—whether together or separately from other dependencies like PyTorch. We should also consider whether they need to be locked together.

Maybe @tdejager could help with this topic

[tdejager]

Indeed, to support this we would require a two step environment creation as the PyPI installation depends on the conda installation. This is related: #1340

[pablovela5620]

Just wanted to add to this, I'm in the process of needing to use the pip version as I'm install torchcodec and its recommended to avoid using conda-forge version of pytorch when packages are pypi only (as torchcodec is) https://pixi.sh/latest/features/pytorch/#installing-from-pypi

I get the error that torch is not installed in build system requirements so I have to manually install thing through a task using pip. This is what my pixi.toml looks like and the error I get

[project]
authors = ["pablovela5620 <[email protected]>"]
channels = ["conda-forge"]
name = "MASt3R-SLAM"
platforms = ["linux-64"]
version = "0.1.0"

[system-requirements]
cuda = "12.0"

[tasks]
install-mast3r = { cmd = "pip install -e thirdparty/mast3r" }
install-thirdparty-in3d = { cmd = "pip install -e thirdparty/in3d" }
install = { cmd = "pwd", depends-on = [
    "install-mast3r",
    "install-thirdparty-in3d",
] }

[pypi-options]
no-build-isolation = ["MAST3R-SLAM", "lietorch", "curope", "MAST3R"]

[pypi-dependencies]
torch = { version = "==2.4.1", index = "https://download.pytorch.org/whl/cu124" }
torchvision = { version = "==0.19.1", index = "https://download.pytorch.org/whl/cu124" }
torchcodec = { version = "==0.1.0", index = "https://download.pytorch.org/whl/cu124" }
MAST3R-SLAM = { path = ".", editable = true }
lietorch = { git = "https://github.com/princeton-vl/lietorch.git" }
simplecv = { git = "https://github.com/pablovela5620/simplecv.git" }
MAST3R = { path = "thirdparty/mast3r", editable = true }
in3d = { path = "thirdparty/in3d", editable = true }

[dependencies]
python = "3.11.*"
numpy = "<2"
cuda-version = "12.4"
ffmpeg = ">=7.1.0,<8"
pip = ">=25.0.1,<26"
tyro = ">=0.9.1,<0.10"
rerun-sdk = ">=0.22.0,<0.23"
beartype = ">=0.20.0,<0.21"
jaxtyping = ">=0.2.36,<0.3"

hint: This error likely indicates that `curope @ file:///home/pablo/0Dev/personal/MASt3R-SLAM/
      thirdparty/mast3r/dust3r/croco/models/curope` depends on `torch`, but doesn't declare it as a
      build dependency. If `curope @ file:///home/pablo/0Dev/personal/MASt3R-SLAM/thirdparty/mast3r/
      dust3r/croco/models/curope` is a first-party package, consider adding `torch` to its `build-
      system.requires`. Otherwise, `uv pip install torch` into the environment and re-run with
      `--no-build-isolation`.

I made a reproducible version in the following branch of a project I'm working on https://github.com/pablovela5620/MASt3R-SLAM/tree/pixi-install-fail

[tdejager]

Yes @pablovela5620 what a great example! Would you as a user object to defining a PyPI build environment in some way? So that we can explicitly do a two-phase instantiation. I don't see this working without some user input though (i.e some specification in the toml) and we always thought this might be somewhat complex. What are your thoughts? Also to the OP @calebho?

[pablovela5620]

I'm personally fine with including something in the toml so long as there are good error messages + documentation links as to why something fails (if say missing from the toml). I previously had problems where things wouldn't resolve and without asking about it on the discord/gh issues I would have never figured it out.

[calebho]

An explicit section in the config file is preferred

[xiaosq2000]

I wonder if https://docs.astral.sh/uv/concepts/resolution/#dependency-metadata is a solution for this?

[roger-]

I’m having the same issue. Is there a workaround that doesn’t require pip? Can the internal uv be called somehow?

I’m worried about calling uv / pip directly and creating some conflict with conda packages.

[pablovela5620]

@roger- What I'm doing is first installing all packages that don't require building, using pixi install then uncommenting/adding the package that needs building. This works seems to work for me without requiring uv/pip being called directly

[tdejager]

@pablovela5620 and you are using that with no-builld-isolation for the packages that do require building?

[pablovela5620]

yes @tdejager here is a minimal pixi.toml to reproduce

[workspace]
authors = ["pablo vela <[email protected]>"]
channels = ["conda-forge"]
name = "sample-build"
platforms = ["linux-64"]
version = "0.1.0"


[pypi-options]
extra-index-urls = ["https://download.pytorch.org/whl/nightly/cu128"]
no-build-isolation = ["xformers"]

[pypi-dependencies]
torch = { version = ">=2.5.1", index = "https://download.pytorch.org/whl/nightly/cu128" }
torchvision = { version = ">=0.20.1", index = "https://download.pytorch.org/whl/nightly/cu128" }
# cu128 version of xformers not yet available, needs to be built from source
xformers = { git = "https://github.com/facebookresearch/xformers.git", rev = "main" }


[tasks]

[dependencies]
python = "3.11.*"
ninja = ">=1.12.1,<2"
cuda = "12.8.*"
cuda-nvcc = "12.8.*"
cudnn = ">=9.8.0.87,<10"
setuptools = ">=78.1.0,<79"

if I just run pixi install on this I'll get the following error

pablo@pablo-dl-server:~/0Dev/personal/sample-build$ pixi install
Error:   × Failed to update PyPI packages for environment 'default'
  ├─▶ Failed to prepare distributions
  ├─▶ Failed to build `xformers @ git+https://github.com/facebookresearch/xformers.git?rev=main#c5c0720c0c628539affbcd5332ccc18f813ccafd`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta:__legacy__.build_wheel` failed (exit status: 1)
      
      [stderr]
      Traceback (most recent call last):
        File "<string>", line 11, in <module>
        File "/home/pablo/0Dev/personal/sample-build/.pixi/envs/default/lib/python3.11/site-packages/setuptools/build_meta.py", line 435, in build_wheel
          return _build(['bdist_wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/pablo/0Dev/personal/sample-build/.pixi/envs/default/lib/python3.11/site-packages/setuptools/build_meta.py", line 426, in _build
          return self._build_with_temp_dir(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/pablo/0Dev/personal/sample-build/.pixi/envs/default/lib/python3.11/site-packages/setuptools/build_meta.py", line 407, in _build_with_temp_dir
          self.run_setup()
        File "/home/pablo/0Dev/personal/sample-build/.pixi/envs/default/lib/python3.11/site-packages/setuptools/build_meta.py", line 522, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/home/pablo/0Dev/personal/sample-build/.pixi/envs/default/lib/python3.11/site-packages/setuptools/build_meta.py", line 320, in run_setup
          exec(code, locals())
        File "<string>", line 24, in <module>
      ModuleNotFoundError: No module named 'torch'
      
      hint: This error likely indicates that `xformers @ git+https://github.com/facebookresearch/xformers.git?rev=main#c5c0720c0c628539affbcd5332ccc18f813ccafd` depends on `torch`, but doesn't
      declare it as a build dependency. If `xformers @ git+https://github.com/facebookresearch/xformers.git?rev=main#c5c0720c0c628539affbcd5332ccc18f813ccafd` is a first-party package, consider
      adding `torch` to its `build-system.requires`. Otherwise, `uv pip install torch` into the environment and re-run with `--no-build-isolation`.

if I comment out xformers

# xformers = { git = "https://github.com/facebookresearch/xformers.git", rev = "main" }

run pixi install and then uncomment out xformers and run pixi install again, xformers builds no problem and things work.

This isn't really a good solution though because it would require users to go and manually comment and uncomment deps that required building. Just a hacky workaround

[tdejager]

Right, but I do think you are on to something interesting :) There might be something we could do here, I'll discuss with the others later.

[sidmadala]

I am having a similar issue and discussed in #3730. If there's been any progress on how to make this installation easier, would much appreciate it!

[tdejager]

Let me get back to you once I know more, my plan is for now to kind of do it in stages and first make an ordering so no-build-isolation packages are installed first. And then move on to more sophisticated things.

Edit: Should be last instead.

[pablovela5620]

Let me get back to you once I know more, my plan is for now to kind of do it in stages and first make an ordering so no-build-isolation packages are installed first. And then move on to more sophisticated things.

@tdejager Just wanted to check if there was any updates on this, I'm bumping up against no build isolation issues with pypi again. I'm on a 5090 and since cuda 12.8 is not available via conda forge (only for pytorch and not torchvision) I'm forced to use the pypi torch version for now (which leads to the no build isolation annoyance)

[tdejager]

Hey @pablovela5620, really sorry to hear that, unfortunately I did not get the time to work on it yet. However, let me see if I can get me or someone else to look at it soon.

[pablovela5620]

No problem, thank you for taking a look!

[alberthli]

Agreed, this would be very helpful. I'm also interested in the case of flash-attn!

[sidmadala]

Let me get back to you once I know more, my plan is for now to kind of do it in stages and first make an ordering so no-build-isolation packages are installed first. And then move on to more sophisticated things.

I don't know if no-build-isolation packages should be installed first. For example, flash-attn requires no-build-isolation because it has to import the version of torch you have hopefully already installed. Maybe they should go last or users should have some way to specify what they'd like?

It appears that uv has a relatively simple way to state what packages should not have build isolation. Unsure what order it places them in though.

[tdejager]

Yeah you are right I misspoke. I meant they have to be installed last. I think the easiest heuristic to implement is.

1. Install no-build-isolation packages last.
2. Install them in-order that they are defined in the no-build-isolation array. So taking an actual example

[project]
authors = [{ name = "Tim de Jager", email = "[email protected]" }]
dependencies = [
  "detectron2 @ git+https://github.com/facebookresearch/detectron2.git@bcfd464d0c810f0442d91a349c0f6df945467143",
]
description = "Build detectron2 from source and installs into pypi environment"
name = "detectron2-from-source"
requires-python = ">= 3.11"
version = "0.1.0"

[build-system]
build-backend = "hatchling.build"
requires = ["hatchling"]

[tool.pixi.project]
channels = ["conda-forge"]
platforms = ["osx-arm64", "osx-64", "linux-64"]

[tool.pixi.dependencies]
cxx-compiler = "1.7.*"
ninja = ">=1, <=2"
pytorch = ">=2.4.0, <=3.0.0"
setuptools = ">=72, <=73"

[tool.pixi.pypi-options]
# Will build detectron2 and fvcore without isolation
# in the default environment
no-build-isolation = ["fvcore", "detectron2"]

This would install all detectron2 dependencies first and then:

0. ... all other deps ...
1. fvcore
2. detectron

Makes sense?

[JeanPhilippeMonteuuis]

Facing the same issue with xformers as described by @pablovela5620

@tdejager : I like the proposed approach

[smokestacklightnin]

Agreed, this would be very helpful. I'm also interested in the case of flash-attn!

+1 on this

[ivandariojr]

+1 on not being able to integrate flash-attn.

[kywch]

Hello, I ran the minimal xformers example by @pablovela5620 (https://github.com/prefix-dev/pixi/issues/3223#issuecomment-2797022870) using the pixi 0.52.0 and still getting the same missing-torch error.

@tdejager Can you please provide the working example for no-build-isolation? Thank you!

kywch@pop-os:/workspace/tests/test-pixi$ pixi shell
Error:   × failed to solve the pypi requirements of environment 'default' for platform 'linux-64'
  ├─▶ Failed to build `xformers @ git+https://github.com/facebookresearch/xformers.git@main`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta.prepare_metadata_for_build_wheel` failed (exit status: 1)
      
      [stderr]
      Traceback (most recent call last):
        File "<string>", line 14, in <module>
        File "/workspace/tests/test-pixi/.pixi/envs/default/lib/python3.11/site-packages/setuptools/build_meta.py", line 377, in
      prepare_metadata_for_build_wheel
          self.run_setup()
        File "/workspace/tests/test-pixi/.pixi/envs/default/lib/python3.11/site-packages/setuptools/build_meta.py", line 320, in
      run_setup
          exec(code, locals())
        File "<string>", line 24, in <module>
      ModuleNotFoundError: No module named 'torch'
      
      hint: This error likely indicates that `xformers @ git+https://github.com/facebookresearch/xformers.git@main` depends
      on `torch`, but doesn't declare it as a build dependency. If `xformers @ git+https://github.com/facebookresearch/
      xformers.git@main` is a first-party package, consider adding `torch` to its `build-system.requires`. Otherwise, `uv pip install
      torch` into the environment and re-run with `--no-build-isolation`.

kywch@pop-os:/workspace/tests/test-pixi$ pixi -V
pixi 0.52.0

[tdejager]

Strange indeed, let me check it out later!

[tdejager]

Okay, so @Hofer-Julian and myself have figured it out, this indeed is not working because the fix as we envisioned is not encompassing enough. I've created the related issue: https://github.com/prefix-dev/pixi/issues/4392. Let's use that as the discussion point for now, it is already focused on a solution though but feel free to leave other thoughts.

[the-xentropy]

Since some people commented specifically about flash-attn. I was able to install/build it successfully (and it works fine) by using pixi run -- python -m pip install flash-attn --no-build-isolation . Alternatively, it looks like you could try to set CUDA_HOME and CUDACXX env vars to avoid using the --no-build-isolation flag per https://github.com/Dao-AILab/flash-attention/issues/1247#issuecomment-2642118210.

I was worried about Pixi breaking the build process or the --no-build-isolation flag escaping the environment somehow, but neither happened. It would be 10x better to allow Pixi to handle this more conveniently, but I'll take it at the moment.

[sync-fly]

Hi. For those looking for a solution.

The 'no-build-isolation' option has been added to [pypi-options].

See this: https://github.com/prefix-dev/pixi/pull/3657#issue-3010970687