Environment with old versions for NumPy/Numba fails to resolve

[hameerabbasi]

Checks

• [X] I have checked that this issue has not already been reported.

• [X] I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

pixi.toml:

[project]
authors = ["Hameer Abbasi <[email protected]>"]
channels = ["conda-forge"]
description = "Add a short description here"
name = "repro"
platforms = ["osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
python = ">=3.10"

[pypi-dependencies]
numpy = ">=1.17"
numba = ">=0.49"

then pixi shell. Output:

  × failed to solve the pypi requirements of 'default' 'osx-arm64'
  ╰─▶ Build backend failed to determine metadata through `prepare_metadata_for_build_wheel` with exit status: 1
      --- stdout:
      
      --- stderr:
      Traceback (most recent call last):
        File "<string>", line 14, in <module>
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmp9XQk1M/lib/python3.12/site-packages/setuptools/build_meta.py", line 373, in
      prepare_metadata_for_build_wheel
          self.run_setup()
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmp9XQk1M/lib/python3.12/site-packages/setuptools/build_meta.py", line 503, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmp9XQk1M/lib/python3.12/site-packages/setuptools/build_meta.py", line 318, in run_setup
          exec(code, locals())
        File "<string>", line 354, in <module>
        File "<string>", line 87, in get_ext_modules
      ModuleNotFoundError: No module named 'numpy.distutils'
      ---

Issue description

I have found variants of this issue as well, please let me know if I should file separate issues for those.

win-64 sporadic issue

Adding `win-64` to platforms without removing `osx-arm64` gives:

  × failed to solve the pypi requirements of 'default' 'win-64'
  ╰─▶ Build backend failed to determine metadata through
      `prepare_metadata_for_build_wheel` with exit status: 1
      --- stdout:
      
      --- stderr:
      Traceback (most recent call last):
        File "<string>", line 14, in <module>
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/
      builds-v0/.tmpF43R1i/lib/python3.12/site-packages/setuptools/
      build_meta.py", line 373, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/
      builds-v0/.tmpF43R1i/lib/python3.12/site-packages/setuptools/
      build_meta.py", line 503, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/
      builds-v0/.tmpF43R1i/lib/python3.12/site-packages/setuptools/
      build_meta.py", line 318, in run_setup
          exec(code, locals())
        File "<string>", line 354, in <module>
        File "<string>", line 87, in get_ext_modules
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/
      builds-v0/.tmpF43R1i/lib/python3.12/site-packages/numpy/
      __init__.py", line 98, in <module>
          _delvewheel_patch_1_8_0()
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/
      builds-v0/.tmpF43R1i/lib/python3.12/site-packages/numpy/
      __init__.py", line 95, in _delvewheel_patch_1_8_0
          os.add_dll_directory(libs_dir)
          ^^^^^^^^^^^^^^^^^^^^
      AttributeError: module 'os' has no attribute
      'add_dll_directory'
      ---

osx-64 sporadic issue

Also, switching `win-64` to `osx-64` sometimes gives:

  × failed to solve the pypi requirements of 'default' 'osx-64'
  ╰─▶ Build backend failed to determine metadata through `prepare_metadata_for_build_wheel` with exit status: 1
      --- stdout:
      
      --- stderr:
      Traceback (most recent call last):
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/_core/__init__.py", line 23, in <module>
          from . import multiarray
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/_core/multiarray.py", line 10, in <module>
          from . import overrides
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/_core/overrides.py", line 8, in <module>
          from numpy._core._multiarray_umath import (
      ImportError: dlopen(/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/_core/_multiarray_umath.cpython-
      312-darwin.so, 0x0002): tried: '/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/_core/
      _multiarray_umath.cpython-312-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64')), '/System/Volumes/Preboot/
      Cryptexes/OS/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/_core/_multiarray_umath.cpython-
      312-darwin.so' (no such file), '/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/_core/
      _multiarray_umath.cpython-312-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64'))
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/__init__.py", line 114, in <module>
          from numpy.__config__ import show as show_config
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/__config__.py", line 4, in <module>
          from numpy._core._multiarray_umath import (
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/_core/__init__.py", line 49, in <module>
          raise ImportError(msg)
      ImportError:
      
      IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
      
      Importing the numpy C-extensions failed. This error can happen for
      many reasons, often due to issues with your setup or how NumPy was
      installed.
      
      We have compiled some common reasons and troubleshooting tips at:
      
          https://numpy.org/devdocs/user/troubleshooting-importerror.html
      
      Please note and check the following:
      
        * The Python version is: Python3.12 from "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/bin/python"
        * The NumPy version is: "2.1.0"
      
      and make sure that they are the versions you expect.
      Please carefully study the documentation linked above for further help.
      
      Original error was: dlopen(/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/_core/
      _multiarray_umath.cpython-312-darwin.so, 0x0002): tried: '/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/
      numpy/_core/_multiarray_umath.cpython-312-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64')), '/System/Volumes/
      Preboot/Cryptexes/OS/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/_core/_multiarray_umath.cpython-
      312-darwin.so' (no such file), '/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/_core/
      _multiarray_umath.cpython-312-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64'))
      
      
      The above exception was the direct cause of the following exception:
      
      Traceback (most recent call last):
        File "<string>", line 14, in <module>
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/setuptools/build_meta.py", line 373, in
      prepare_metadata_for_build_wheel
          self.run_setup()
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/setuptools/build_meta.py", line 503, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/setuptools/build_meta.py", line 318, in run_setup
          exec(code, locals())
        File "<string>", line 354, in <module>
        File "<string>", line 87, in get_ext_modules
        File "/Users/habbasi/Library/Caches/rattler/cache/uv-cache/builds-v0/.tmpbLdG4X/lib/python3.12/site-packages/numpy/__init__.py", line 119, in <module>
          raise ImportError(msg) from e
      ImportError: Error importing numpy: you should not try to import numpy from
              its source directory; please exit the numpy source tree, and relaunch
              your python interpreter from there.
      ---

It seems to me like there are three issues at play here:

1. Building from source for osx-arm64 when wheels exist (this issue).
2. Building from source for win-64 (you can't on an ARM Mac).
3. Mixing architectures for NumPy (x86_64) and Python (arm64e).

Also, when all three platforms are present; which error shows up first is non-deterministic.

Expected behavior

The environment should (always) solve.

[tdejager]

As extra information:

• Switching the numpy and numba depencies seems to select different versions. And the solve succeeds.
• I've tested with uv and pyproject.toml and did not encounter these problems.