Tracking issue: PyPI dependencies pixi can't manage yet.

[ruben-arts]

There are a few PyPI packages pixi can't install yet, where pip can.

Please paste your examples in this issue so we have a list of known packages we can track, test and benchmark with along the way.

Information we would like:

1. What did you run and what was the outcome? e.g. pixi add --pypi packagex If it doesn't recreate in an empty environment please share your pixi.toml that recreates the issue.

2. What error did pixi return? e.g.

   × RECORD file doesn't match wheel contents: missing hash for mediapipe/version.txt (expected sha256=-fE2KU)
   

3. Can pip install the package? Does pip install packagex work?

4. What platform are you on? e.g. linux-64

5. Did you find a workaround, if so please explain. e.g. build it into a conda package, using a custom fork, etc.

Your input would greatly help us improve the pixi's experience! Thanks in advance! :heart:

[pablovela5620]

Mediapipe

1. pixi add --pypi mediapipe
2. × RECORD file doesn't match wheel contents: missing hash for mediapipe/version.txt (expected sha256=-fE2KU)
3. python -m pip install mediapipe does work
4. platform osx-arm64
5. work around was just to use pip to install using pixi tasks

[project]
name = "ipdscan"
version = "0.1.0"
description = "Add a short description here"
authors = ["pablovela5620 <[email protected]>"]
channels = ["conda-forge"]
platforms = ["osx-arm64"]

[tasks]
mp-install = "python -m pip install mediapipe"

[dependencies]
python = "3.11.*"
pip = ">=23.3.2,<23.4"
rerun-sdk = ">=0.12.0,<0.13"
requests = ">=2.31.0,<2.32"
tqdm = ">=4.66.1,<4.67"

[pypi-dependencies]
imutils = "*"

[tylerjw]

I wanted to use this library that is packaged in pypi: https://github.com/spirali/elsie

Here is the error I was seeing when trying to use pixi add:

 WARN rattler_installs_packages::index::package_database: errors while processing source distributions:
  × failed to resolve `pypi-dependencies`, due to underlying error
  ╰─▶ No metadata could be extracted for the following available artifacts:
      	- lxml-4.6.5.tar.gz

Error:   × error while processing source distribution 'lxml-4.6.5.tar.gz':
  │  could not build wheel: <string>:67: DeprecationWarning: pkg_resources is deprecated as an API. See https://
  │ setuptools.pypa.io/en/latest/pkg_resources.html
  │ 
  help: Probably an error during processing of source distributions. Please check the error message above.

Posting in the discord channel got it working with this fix:

pixi add python lxml
pixi add --pypi elsie

The reason for this is:

there seems to be an issue with one of the lxml source distributions. Meaning you have to locally build the package but that is not working because of the error it is giving you. pixi reports back on errorous packages where pip probably continues trying other versions.

[liquidcarbon]

pixi add duckdb ❌ pixi add --pypi duckdb ❌ vs pixi run pip install duckdb ✅ on Windows

PS C:\code> pixi init duckdb-pip
✔ Initialized project in C:\code\duckdb-pip
PS C:\code> cd .\duckdb-pip\
PS C:\code\duckdb-pip> pixi add python=3.11
✔ Added python=3.11
PS C:\code\duckdb-pip> pixi add duckdb     
  × could not determine any available versions for duckdb on win-64. Either the package could not be found or version constraints on other
  │ dependencies result in a conflict.
  ╰─▶ Cannot solve the request because of: No candidates were found for duckdb *.


PS C:\code\duckdb-pip> pixi add --pypi duckdb
  × could not build wheel: warning: no files found matching '*.h' under directory 'duckdb'
  │ warning: no files found matching '*.hpp' under directory 'duckdb'
  │ warning: no files found matching '*.cpp' under directory 'duckdb'
  │ warning: no files found matching '*.h' under directory 'src'
  │ warning: manifest_maker: MANIFEST.in, line 6: 'recursive-include' expects <dir> <pattern1> <pattern2> ...
  │
  │ C:\Users\AKISLU~1\AppData\Local\Temp\.tmpkvrxrc\venv\Lib\site-packages\setuptools\command\build_py.py:207: _Warning: Package 'duckdb-      
  │ stubs.value' is absent from the `packages` configuration.
  │ !!
  │
  │         ********************************************************************************
  │         ############################
  │         # Package would be ignored #
  │         ############################
  │         Python recognizes 'duckdb-stubs.value' as an importable package[^1],
  │         but it is absent from setuptools' `packages` configuration.
  │
  │         This leads to an ambiguous overall configuration. If you want to distribute this
  │         package, please make sure that 'duckdb-stubs.value' is explicitly added
  │         to the `packages` configuration field.
  │
  │         Alternatively, you can also rely on setuptools' discovery methods
  │         (for example by using `find_namespace_packages(...)`/`find_namespace:`
  │         instead of `find_packages(...)`/`find:`).
  │
  │         You can read more about "package discovery" on setuptools documentation page:
  │
  │         - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  │
  │         If you don't want 'duckdb-stubs.value' to be distributed and are
  │         already explicitly excluding 'duckdb-stubs.value' via
  │         `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
  │         you can try to use `exclude_package_data`, or `include-package-data=False` in
  │         combination with a more fine grained `package-data` configuration.
  │
  │         You can read more about "package data files" on setuptools documentation page:
  │
  │         - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  │
  │
  │         [^1]: For Python, any directory (with suitable naming) can be imported,
  │               even if it does not contain any `.py` files.
  │               On the other hand, currently there is no concept of package data
  │               directory, all directories are treated like packages.
  │         ********************************************************************************
  │
  │ !!
  │   check.warn(importable)
  │ C:\Users\AKISLU~1\AppData\Local\Temp\.tmpkvrxrc\venv\Lib\site-packages\setuptools\command\build_py.py:207: _Warning: Package 'duckdb-      
  │ stubs.value.constant' is absent from the `packages` configuration.
  │ !!
  │
  │         ********************************************************************************
  │         ############################
  │         # Package would be ignored #
  │         ############################
  │         Python recognizes 'duckdb-stubs.value.constant' as an importable package[^1],
  │         but it is absent from setuptools' `packages` configuration.
  │
  │         This leads to an ambiguous overall configuration. If you want to distribute this
  │         package, please make sure that 'duckdb-stubs.value.constant' is explicitly added
  │         to the `packages` configuration field.
  │
  │         Alternatively, you can also rely on setuptools' discovery methods
  │         (for example by using `find_namespace_packages(...)`/`find_namespace:`
  │         instead of `find_packages(...)`/`find:`).
  │
  │         You can read more about "package discovery" on setuptools documentation page:
  │
  │         - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  │
  │         If you don't want 'duckdb-stubs.value.constant' to be distributed and are
  │         already explicitly excluding 'duckdb-stubs.value.constant' via
  │         `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
  │         you can try to use `exclude_package_data`, or `include-package-data=False` in
  │         combination with a more fine grained `package-data` configuration.
  │
  │         You can read more about "package data files" on setuptools documentation page:
  │
  │         - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  │
  │
  │         [^1]: For Python, any directory (with suitable naming) can be imported,
  │               even if it does not contain any `.py` files.
  │               On the other hand, currently there is no concept of package data
  │               directory, all directories are treated like packages.
  │         ********************************************************************************
  │
  │ !!
  │   check.warn(importable)
  │ C:\Users\AKISLU~1\AppData\Local\Temp\.tmpkvrxrc\venv\Lib\site-packages\setuptools\command\build_py.py:207: _Warning: Package
  │ 'duckdb.experimental' is absent from the `packages` configuration.
  │ !!
  │
  │         ********************************************************************************
  │         ############################
  │         # Package would be ignored #
  │         ############################
  │         Python recognizes 'duckdb.experimental' as an importable package[^1],
  │         but it is absent from setuptools' `packages` configuration.
  │
  │         This leads to an ambiguous overall configuration. If you want to distribute this
  │         package, please make sure that 'duckdb.experimental' is explicitly added
  │         to the `packages` configuration field.
  │
  │         Alternatively, you can also rely on setuptools' discovery methods
  │         (for example by using `find_namespace_packages(...)`/`find_namespace:`
  │         instead of `find_packages(...)`/`find:`).
  │
  │         You can read more about "package discovery" on setuptools documentation page:
  │
  │         - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  │
  │         If you don't want 'duckdb.experimental' to be distributed and are
  │         already explicitly excluding 'duckdb.experimental' via
  │         `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
  │         you can try to use `exclude_package_data`, or `include-package-data=False` in
  │         combination with a more fine grained `package-data` configuration.
  │
  │         You can read more about "package data files" on setuptools documentation page:
  │
  │         - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  │
  │
  │         [^1]: For Python, any directory (with suitable naming) can be imported,
  │               even if it does not contain any `.py` files.
  │               On the other hand, currently there is no concept of package data
  │               directory, all directories are treated like packages.
  │         ********************************************************************************
  │
  │ !!
  │   check.warn(importable)
  │ error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/      
  │ visual-cpp-build-tools/
  │

PS C:\code\duckdb-pip> pixi add pip          
✔ Added pip
PS C:\code\duckdb-pip> pixi run pip install duckdb
Collecting duckdb
  Downloading duckdb-0.9.2-cp311-cp311-win_amd64.whl.metadata (798 bytes)
Downloading duckdb-0.9.2-cp311-cp311-win_amd64.whl (10.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.3/10.3 MB 21.8 MB/s eta 0:00:00
Installing collected packages: duckdb
Successfully installed duckdb-0.9.2

[liquidcarbon]

5. work around was just to use pip to install using pixi tasks

@pablovela5620 I am also finding this necessary quite often, probably more often than @ruben-arts would like :)

In this pattern the environment definition is fragmented between pixi.toml tasks and pixi.lock. It could work, esp with pinned pip installs in tasks, but is it the intention?

[ruben-arts]

In this pattern the environment definition is fragmented between pixi.toml tasks and pixi.lock. It could work, esp with pinned pip installs in tasks, but is it the intention?

@liquidcarbon This is indeed not what we want as UX but we simply need to develop more to support all PyPI packages. Its a weird bunch of requirements we have to support to be equivalent to pip. So please keep posting non working packages!

[ruben-arts]

@liquidcarbon the duckdb on conda-forge has been continued on python-duckdb which should be available on windows. We'll keep the example for pypi to test more!

[tdejager]

@liquidcarbon I cannot reproduce this, neither on OSX or on windows. It's strange it does not select the .whl for some reason.

[tdejager]

@tylerjw actually it also fails for the same version in pip on apple silicon, like pip we error out when we cannot build the first source distribution.

[tdejager]

@pablovela5620 so it seems mediapipe 10.9 is a package with an invalid RECORD file, I manually checked it and it's incorrect.

This is mentioned in the PyPa

During extraction, wheel installers verify all the hashes in RECORD against the file contents. Apart from RECORD and its signatures, installation will fail if any file in the archive is not both mentioned and correctly hashed in RECORD.

Mediapipe has a version.txt that is not mentioned in the RECORD for the 10.9 release.

Which in this case triggers the error, I'm unsure why pip does no do this. But I feel its good to adhere to the standard here.

In any case, mediapipe 10.8 does seem to work, you could use that instead.

Also see: https://github.com/google/mediapipe/issues/5025

[liquidcarbon]

@liquidcarbon I cannot reproduce this, neither on OSX or on windows. It's strange it does not select the .whl for some reason.

@tdejager Just tried on Win10, original comment was on Win11 same thing:

PS C:\Users\a\Desktop\code\duckdb-pip> pixi add python=3.11 ✔ Added python 3.11.* PS C:\Users\a\Desktop\code\duckdb-pip> pixi add duckdb × could not determine any available versions for duckdb on win-64. Either the package could not be found or version │ constraints on other dependencies result in a conflict. ╰─▶ Cannot solve the request because of: No candidates were found for duckdb *.

I was on Pixi 0.9.1, upped to 0.13.0 - same thing

But pixi add --pypi duckdb works. 🤷‍♂️

[ruben-arts]

The conda package you add with pixi add duckdb should be pixi add python-duckdb

https://prefix.dev/channels/conda-forge/packages/python-duckdb

[liquidcarbon]

@ruben-arts noted -- I just wasn't sure which part @tdejager was trying to reproduce

[awray3]

Tensorflow metal on Apple silicon MacOS 14.2.1:

pixi init tf-metal && cd tf-metal
pixi add "python>=3.11" "tensorflow>=2.13" pip
pixi add --pypi tensorflow-metal
> × failed to resolve `pypi-dependencies`, due to underlying error
  ╰─▶ The following packages are incompatible
      └─ tensorflow-metal * cannot be installed because there are no viable options:
         └─ tensorflow-metal 0.1.0 | 0.1.1 | 0.1.2 | 0.2.0 | 0.3.0 | 0.4.0 | 0.5.0 | 0.5.1 | 0.6.0 | 0.7.0 | 0.7.1
      | 0.8.0 | 1.0.0 | 1.0.1 | 1.1.0 is excluded because none of the artifacts are compatible with the Python
      interpreter or glibc version and there are no supported sdists

pixi run pip install tensorflow-metal
> Successfully installed tensorflow-metal-1.1.0

[baszalmstra]

Most likely you are missing a system requirement: https://pixi.sh/latest/configuration/#the-system-requirements-table

Most likely macos=12.0

[awray3]

Nice, that fixed it. ~~But would the generated pixi.toml work on a linux machine? I guess I would have to mark tensorflow-metal as platform dependent somehow. My goal is to have a pixi.toml that installs tensorflow metal on a mac and tensorflow with gpu support on linux.~~

EDIT: Nevermind, I found the example demonstrating how to do this. Thank you!

[roaldarbol]

Installing opencv-python-headless with conda works (through python -m pip install opencv-python-headless), but not with pixi. I'm on osx-64.

(base) ➜  idtracker pixi add --pypi opencv-python-headless
  × failed to resolve `pypi-dependencies`, due to underlying error
  ╰─▶ No metadata could be extracted for the following available artifacts:
        - opencv-python-headless-4.9.0.80.tar.gz

Error:   × error while processing source distribution 'opencv-python-headless-4.9.0.80.tar.gz':
  │  could not build wheel: Traceback (most recent call last):
  │   File "/var/folders/rh/y9ws4bgj7x50twypcwtk2ypr0000gp/T/.tmpvjLm8l/build_frontend.py", line 124, in <module>
  │     get_requires_for_build_wheel(backend, work_dir)
  │   File "/var/folders/rh/y9ws4bgj7x50twypcwtk2ypr0000gp/T/.tmpvjLm8l/build_frontend.py", line 58, in get_requires_for_build_wheel
  │     result = f()
  │              ^^^
  │   File "/var/folders/rh/y9ws4bgj7x50twypcwtk2ypr0000gp/T/.tmpvjLm8l/venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in
  │ get_requires_for_build_wheel
  │     return self._get_build_requires(config_settings, requirements=['wheel'])
  │            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  │   File "/var/folders/rh/y9ws4bgj7x50twypcwtk2ypr0000gp/T/.tmpvjLm8l/venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
  │     self.run_setup()
  │   File "/var/folders/rh/y9ws4bgj7x50twypcwtk2ypr0000gp/T/.tmpvjLm8l/venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 487, in run_setup
  │     super().run_setup(setup_script=setup_script)
  │   File "/var/folders/rh/y9ws4bgj7x50twypcwtk2ypr0000gp/T/.tmpvjLm8l/venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
  │     exec(code, locals())
  │   File "<string>", line 10, in <module>
  │ ModuleNotFoundError: No module named 'skbuild'
  │ 
  help: Probably an error during processing of source distributions. Please check the error message above.

[baszalmstra]

@roaldarbol The error message is absolutely terrible but if you add:

[system-requirements]
macos = "11.0"

It should work.

[roaldarbol]

It does indeed! Thanks!

[jacobbieker]

Trying to install torch-geometric-temporal with pixi add. It fails as it cannot find torch, which had already been install previously

1. pixi add --pypi torch-geometric-temporal
2. python -m pip install torch-geomtric-temporal does work, after the other dependencies are install with pixi
3. platform linux-64
4. work around was just to use pip to install using pixi tasks Configuration:

[project]
name = "graph_weather"
version = "0.1.0"
description = "Add a short description here"
authors = ["Jacob Bieker <[email protected]>"]
channels = ["pyg", "nvidia", "conda-forge", "pytorch"]
platforms = ["linux-64"]

[tasks]
tinstall = "python -m pip install torch-geometric-temporal"

[dependencies]
python = "3.11.*"
torchvision = ">=0.16.1,<0.17"
pytorch-cluster = ">=1.6.3,<1.7"
pytorch-scatter = ">=2.1.2,<2.2"
pytorch-cuda = "12.1.*"
xarray = ">=2024.2.0,<2024.3"
pytorch-spline-conv = ">=1.2.2,<1.3"
pytorch = ">=2.1"
pandas = ">=2.2.1,<2.3"
h3-py = ">=3.7.6,<3.8"
numcodecs = ">=0.12.1,<0.13"
scipy = ">=1.12.0,<1.13"
zarr = ">=2.17.0,<2.18"
pyg = ">=2.5.0,<2.6"
tqdm = ">=4.66.2,<4.67"
pytorch-sparse = ">=0.6.18,<0.7"
lightning = ">=2.2.0.post0,<2.2.1"
einops = ">=0.7.0,<0.8"
fsspec = ">=2024.2.0,<2024.3"
datasets = ">=2.18.0,<2.19"
pip = ">=24.0,<25"


[pypi-dependencies]
pytest = "*"  # This means any version (this `*` is custom in pixi)
pre-commit = "*"
pysolar = "*"

[ruben-arts]

Trying to install torch-geometric-temporal with pixi add. It fails as it cannot find torch, which had already been install previously

It does indeed not work. Testing the repro gives me this error:

  × failed to solve the pypi requirements of 'default' 'linux-64'
  ├─▶ failed to resolve pypi dependencies
  ├─▶ Failed to download and build: torch-scatter==2.1.2
  ├─▶ Failed to build: torch-scatter==2.1.2
  ╰─▶ Build backend failed to determine extra requires with `build_wheel()`:
      --- stdout:
      
      --- stderr:
      Traceback (most recent call last):
        File "<string>", line 14, in <module>
        File "/home/rarts/.cache/rattler/cache/uv-cache/.tmpvVNUWW/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in
      get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/rarts/.cache/rattler/cache/uv-cache/.tmpvVNUWW/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 295, in
      _get_build_requires
          self.run_setup()
        File "/home/rarts/.cache/rattler/cache/uv-cache/.tmpvVNUWW/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 487, in
      run_setup
          super().run_setup(setup_script=setup_script)
        File "/home/rarts/.cache/rattler/cache/uv-cache/.tmpvVNUWW/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in
      run_setup
          exec(code, locals())
        File "<string>", line 8, in <module>
      ModuleNotFoundError: No module named 'torch'
      ---

In both the uv and rip workflow although import torch in the pixi run python does work for this pixi.toml.

Thanks for the info @jacobbieker

[tdejager]

Trying to install torch-geometric-temporal with pixi add. It fails as it cannot find torch, which had already been install previously

It does indeed not work. Testing the repro gives me this error:

  × failed to solve the pypi requirements of 'default' 'linux-64'
  ├─▶ failed to resolve pypi dependencies
  ├─▶ Failed to download and build: torch-scatter==2.1.2
  ├─▶ Failed to build: torch-scatter==2.1.2
  ╰─▶ Build backend failed to determine extra requires with `build_wheel()`:
      --- stdout:
      
      --- stderr:
      Traceback (most recent call last):
        File "<string>", line 14, in <module>
        File "/home/rarts/.cache/rattler/cache/uv-cache/.tmpvVNUWW/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in
      get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/rarts/.cache/rattler/cache/uv-cache/.tmpvVNUWW/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 295, in
      _get_build_requires
          self.run_setup()
        File "/home/rarts/.cache/rattler/cache/uv-cache/.tmpvVNUWW/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 487, in
      run_setup
          super().run_setup(setup_script=setup_script)
        File "/home/rarts/.cache/rattler/cache/uv-cache/.tmpvVNUWW/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in
      run_setup
          exec(code, locals())
        File "<string>", line 8, in <module>
      ModuleNotFoundError: No module named 'torch'
      ---

In both the uv and rip workflow although import torch in the pixi run python does work for this pixi.toml.

Thanks for the info @jacobbieker

Would be kind-of useful if we can keep the build environments for uv as well, so it's easier to debug these things.

[liblaf]

What did you run and what was the outcome?

pixi add --pypi -vv "trimesh[all]"

What error did pixi return?

Log

 INFO pixi::lock_file::outdated: the pypi dependencies of environment 'default' for platform linux-64 are out of date because the requirement 'trimesh[all]' could not be satisfied (required by '<environment>')
 INFO pixi::lock_file::update: updated conda packages in the 'default' prefix in 5ms 378us 132ns
 INFO resolve_pypi{group=default platform=linux-64}: pixi::lock_file::resolve: the following python packages are assumed to be installed by conda: libexpat 2.6.1, xz 5.2.6, readline 8.2, openssl 3.2.1, ld-impl-linux-64 2.40, libsqlite 3.45.1, libuuid 2.38.1, libffi 3.4.2, ncurses 6.4, bzip2 1.0.8, libzlib 1.2.13, tk 8.6.13, tzdata 2024a0, libxcrypt 4.4.36, libnsl 2.0.1, libgcc-ng 13.2.0, python 3.11.8, ca-certificates 2024.2.2, libgomp 13.2.0
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: root @ 0a0.dev0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: trimesh[all] @ 4.2.0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: trimesh @ 4.2.0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: trimesh[test] @ 4.2.0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: trimesh[easy] @ 4.2.0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: trimesh[recommend] @ 4.2.0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: numpy @ 1.26.4    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: pytest-cov @ 4.1.0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: coveralls @ 3.3.1    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: mypy @ 1.9.0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: ezdxf @ 1.2.0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: pytest @ 8.1.1    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: pymeshlab @ 2022.2.post3    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: pyinstrument @ 4.6.2    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: matplotlib @ 3.8.3    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: ruff @ 0.3.2    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: typeguard @ 4.1.5    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: colorlog @ 6.8.2    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: mapbox-earcut @ 1.0.1    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: chardet @ 5.2.0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: lxml @ 5.1.0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: jsonschema @ 4.21.1    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: networkx @ 3.2.1    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: svg-path @ 6.3    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: pycollada @ 0.8    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: setuptools @ 69.1.1    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: shapely @ 2.0.3    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: xxhash @ 3.4.1    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: rtree @ 1.2.0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: httpx @ 0.27.0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: scipy @ 1.12.0    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: Start conflict resolution because incompat satisfied:
   embreex <2.17.7 | >2.17.7, <2.17.7.post1 | >2.17.7.post1, <2.17.7.post2 | >2.17.7.post2, <2.17.7.post3 | >2.17.7.post3, <2.17.7.post4 | >2.17.7.post4 is forbidden    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: prior cause: embreex <2.17.7.post1 | >2.17.7.post1, <2.17.7.post2 | >2.17.7.post2, <2.17.7.post3 | >2.17.7.post3, <2.17.7.post4 | >2.17.7.post4 is forbidden    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: prior cause: embreex <2.17.7.post2 | >2.17.7.post2, <2.17.7.post3 | >2.17.7.post3, <2.17.7.post4 | >2.17.7.post4 is forbidden    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: prior cause: embreex <2.17.7.post3 | >2.17.7.post3, <2.17.7.post4 | >2.17.7.post4 is forbidden    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: prior cause: embreex <2.17.7.post4 | >2.17.7.post4 is forbidden    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: backtrack to DecisionLevel(5)    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: Start conflict resolution because incompat satisfied:
   embreex ==2.17.7.post4 is forbidden    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: prior cause: embreex * is forbidden    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: prior cause: trimesh[easy] ==4.2.0 is forbidden    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: backtrack to DecisionLevel(2)    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: not adding trimesh[easy] @ 4.1.8 because of its dependencies    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: not adding trimesh[easy] @ 4.1.7 because of its dependencies    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: not adding trimesh[easy] @ 4.1.6 because of its dependencies    
...
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: not adding trimesh[easy] @ 1.9.15 because of its dependencies    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: not adding trimesh[easy] @ 1.9.14 because of its dependencies    
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: not adding trimesh[easy] @ 1.9.13 because of its dependencies    
 INFO resolve_pypi{group=default platform=linux-64}:process_request{request=Metadata trimesh==1.9.12}:get_or_build_wheel_metadata{dist=trimesh==1.9.12}:build_source_dist_metadata:setup_build{package_id="trimesh==1.9.12" subdirectory=None}:solve: pubgrub::internal::partial_solution: add_decision: root @ 0a0.dev0    
 INFO resolve_pypi{group=default platform=linux-64}:process_request{request=Metadata trimesh==1.9.12}:get_or_build_wheel_metadata{dist=trimesh==1.9.12}:build_source_dist_metadata:setup_build{package_id="trimesh==1.9.12" subdirectory=None}:solve: pubgrub::internal::partial_solution: add_decision: setuptools @ 69.1.1    
 INFO resolve_pypi{group=default platform=linux-64}:process_request{request=Metadata trimesh==1.9.12}:get_or_build_wheel_metadata{dist=trimesh==1.9.12}:build_source_dist_metadata:setup_build{package_id="trimesh==1.9.12" subdirectory=None}:solve: pubgrub::internal::partial_solution: add_decision: wheel @ 0.43.0    
  × failed to solve the pypi requirements of 'default' 'linux-64'
  ├─▶ failed to resolve pypi dependencies
  ├─▶ Failed to download and build: trimesh==1.9.12
  ├─▶ Failed to build: trimesh==1.9.12
  ╰─▶ Build backend failed to determine extra requires with `build_wheel()` with exit status: 1
      --- stdout:
      
      --- stderr:
      Traceback (most recent call last):
        File "<string>", line 9, in <module>
      ModuleNotFoundError: No module named 'pypandoc'
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "<string>", line 14, in <module>
        File "/home/liblaf/.cache/rattler/cache/uv-cache/.tmpc8Jzlx/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in
      get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/liblaf/.cache/rattler/cache/uv-cache/.tmpc8Jzlx/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/home/liblaf/.cache/rattler/cache/uv-cache/.tmpc8Jzlx/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 487, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/home/liblaf/.cache/rattler/cache/uv-cache/.tmpc8Jzlx/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 12, in <module>
      FileNotFoundError: [Errno 2] No such file or directory: 'README.md'
      ---

Can pip install the package? Does pip install "trimesh[all]" work?

Yes.

What platform are you on?

linux-64

The above log shows that the problem seems to be with embreex, so I tried adding it directly, and got the following error:

$ pixi add --pypi -vv embreex
 INFO pixi::lock_file::outdated: the pypi dependencies of environment 'default' for platform linux-64 are out of date because the requirement 'embreex' could not be satisfied (required by '<environment>')
 INFO pixi::lock_file::update: updated conda packages in the 'default' prefix in 6ms 587us 9ns
 INFO resolve_pypi{group=default platform=linux-64}: pixi::lock_file::resolve: the following python packages are assumed to be installed by conda: libxcrypt 4.4.36, readline 8.2, libgomp 13.2.0, setuptools 69.1.1, xz 5.2.6, libnsl 2.0.1, tzdata 2024a0, libuuid 2.38.1, wheel 0.42.0, libgcc-ng 13.2.0, bzip2 1.0.8, ca-certificates 2024.2.2, libsqlite 3.45.1, ncurses 6.4, pip 24.0, openssl 3.2.1, ld-impl-linux-64 2.40, libzlib 1.2.13, libffi 3.4.2, libexpat 2.6.1, python 3.11.8, tk 8.6.13
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::partial_solution: add_decision: root @ 0a0.dev0
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: Start conflict resolution because incompat satisfied:
   embreex <2.17.7 | >2.17.7, <2.17.7.post1 | >2.17.7.post1, <2.17.7.post2 | >2.17.7.post2, <2.17.7.post3 | >2.17.7.post3, <2.17.7.post4 | >2.17.7.post4 is forbidden
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: prior cause: embreex <2.17.7.post1 | >2.17.7.post1, <2.17.7.post2 | >2.17.7.post2, <2.17.7.post3 | >2.17.7.post3, <2.17.7.post4 | >2.17.7.post4 is forbidden
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: prior cause: embreex <2.17.7.post2 | >2.17.7.post2, <2.17.7.post3 | >2.17.7.post3, <2.17.7.post4 | >2.17.7.post4 is forbidden
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: prior cause: embreex <2.17.7.post3 | >2.17.7.post3, <2.17.7.post4 | >2.17.7.post4 is forbidden
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: prior cause: embreex <2.17.7.post4 | >2.17.7.post4 is forbidden
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: prior cause: embreex * is forbidden
 INFO resolve_pypi{group=default platform=linux-64}:solve: pubgrub::internal::core: prior cause: root ==0a0.dev0 is forbidden
  × failed to solve the pypi requirements of 'default' 'linux-64'
  ├─▶ failed to resolve pypi dependencies
  ╰─▶ Because only the following versions of embreex are available:
          embreex==2.17.7
          embreex==2.17.7.post1
          embreex==2.17.7.post2
          embreex==2.17.7.post3
          embreex==2.17.7.post4
      and embreex==2.17.7 is unusable because no wheels are available with a matching platform, we can conclude that embreex<2.17.7.post1 cannot be used.
      And because embreex==2.17.7.post1 is unusable because no wheels are available with a matching platform, we can conclude that embreex<2.17.7.post2 cannot be used.
      And because embreex==2.17.7.post2 is unusable because no wheels are available with a matching platform and embreex==2.17.7.post3 is unusable because no wheels are
      available with a matching platform, we can conclude that embreex<2.17.7.post4 cannot be used.
      And because embreex==2.17.7.post4 is unusable because no wheels are available with a matching platform and you require embreex, we can conclude that the
      requirements are unsatisfiable.

[evetion]

What did you run and what was the outcome?

pixi add --platform osx-arm64 --pypi "meshkernel==4.1"

What error did pixi return?

  × failed to resolve `pypi-dependencies`, due to underlying error
  ╰─▶ The following packages are incompatible
      └─ meshkernel ==4.1 cannot be installed because there are no viable options:
         └─ meshkernel 4.1.0 is excluded because none of the artifacts are compatible with the Python interpreter or glibc version and there are no supported sdists

Can pip install the package?

pixi run pip install "meshkernel==4.1" works on the osx-arm64 platform.

What platform are you on?

This happens on the osx-64 and osx-arm64 platforms (it works on win-64 and linux-64). I expect this to be some mismatching in the name/metadata of/in the .whl files for the macos specific ones.

Did you find a workaround, if so please explain.

Not yet, it would help if the resolver can be more verbose (-vvv has no effect on it) how it matches the platform/interpreter/glibc version and what's incompatible. Possibly renaming the wheels on the meshkernel side would be enough to fix it.

Encountered in https://github.com/Deltares/Ribasim/pull/1137

[wolfv]

@evetion that error often means that the system-requirements are not set high enough. I can see that wheels are only available for macos = 14.0 (for arm64). We should do a better job at explaining this.

Unfortunately, setting that in my pixi.toml didn't work just now, so we might also still have to do some more debugging of the uv integration! I'll take a look :)

[wolfv]

Sorry, my bad. meshkernel actually does work fine when you add the following:

[system-requirements]
macos = "14.0"

We should make the error more actionable and figure out a better default behavior.

[wolfv]

@liblaf I wonder if for you it's a similar issue and adding glibc 2.28 or higher would fix it.

e.g.

[system-requirements]
glibc = "2.28"

[liblaf]

@liblaf I wonder if for you it's a similar issue and adding glibc 2.28 or higher would fix it.

e.g.

[system-requirements]
glibc = "2.28"

@wolfv thx! pixi add --pypi "trimesh[all]" now works for me with the following config:

[system-requirements]
libc = "2.39"

[wolfv]

Good to hear, thanks @liblaf! We should make this easier for users, and maybe use a more recent glibc / macos version as default.

[evetion]

Sorry, my bad. meshkernel actually does work fine when you add the following:

[system-requirements]
macos = "14.0"

We should make the error more actionable and figure out a better default behavior.

Thanks for the quick fix! This does indeed work. I'm looking forward to further improvements to pixi.

[ruben-arts]

The pypi support seems to be much better now. I'm closing this issue, feel free to open when you have a related issue.