Packages being built even if flagged as "only-binary"

[arnaldojvg]

I have this minimal pyproject.toml

[project]
name = "my-project"
version = "0.2.0"
description = "Cool project"
requires-python = ">=3.10"
dependencies = [ "pyarrow==17.0.0" ]

[tool.uv]
[tool.uv.pip]
only-binary = ["pyarrow"]

(venv) (base) ➜  test_uv  uv cache clean
Clearing cache at: /Users/arnaldovarela/.cache/uv
Removed 1770 files (200.6MiB)
(venv) (base) ➜  test_uv  uv self update
info: Checking for updates...
success: You're on the latest version of uv (v0.4.25)
(venv) (base) ➜  test_uv  uv sync       
warning: `VIRTUAL_ENV=venv` does not match the project environment path `.venv` and will be ignored
Using CPython 3.10.4
Creating virtual environment at: .venv
Resolved 3 packages in 0.92ms
   Built numpy==2.1.2                                                                                                                                                                                                                                               error: Failed to prepare distributions
  Caused by: Failed to download and build `pyarrow==17.0.0`
...

And the packages are built instead of using the binaries. This is a problem as to build pyarrow some apache dependencies and cmake are required to be installed.

I'm running this on a Mac M1 running 14.5 (23F79).

When installing with Pipfile:

[[source]]
url = "https://pypi.org/simple"
name = "pypi"
verify_ssl = true

[packages]
pyarrow = "==17.0.0"


[requires]
python_version = "3.10"

it takes just a few seconds and no need to build:

(venv) (base) ➜  test_uv  pipenv install
Loading .env environment variables...
Courtesy Notice:
Pipenv found itself running within a virtual environment,  so it will automatically use that environment, instead of  creating its own for any project. You can set
PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create  its own instead.
You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [dev-packages] dependencies...
Updated Pipfile.lock (9f6dbf61cecaca8a7db4e36abb151730ab51fadc17e8c779b2687808a27abe20)!
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
Installing dependencies from Pipfile.lock (7abe20)...

Also i think is a bug that it tries to build the package as, according to the docs

but operations that require building distributions will exit with an error.

And it isnt, it is trying to build and the errors depend on the build phase error:

1. cmake missing
2. apache-arrow missing
3. Some arrow build flag missing

[zanieb]

Looks like you're using uv sync but configuring things at the tool.uv.pip level. See https://docs.astral.sh/uv/configuration/files/#configuring-the-pip-interface

[arnaldojvg]

Okok that in fact makes sense @zanieb thanks a lot.

Do you see anyway, then, for me to specify the package as dependency and not build it but taking the built wheel directly using sync? 🙏

I feel there is something off, as with pipenv it definitely finds the built binary (the example in the original post ran in a few seconds)

but i just ran:

uv sync --no-build-package pyarrow
Resolved 3 packages in 281ms
error: Distribution `pyarrow==17.0.0 @ registry+https://pypi.org/simple` can't be installed because it is marked as `--no-build` but has no binary distribution

[zanieb]

Hm, interesting yeah this works fine for me

❯ echo pyarrow==17.0.0 | uv pip compile --only-binary pyarrow -
Resolved 2 packages in 5ms
# This file was autogenerated by uv via the following command:
#    uv pip compile --only-binary pyarrow -
numpy==2.1.2
    # via pyarrow
pyarrow==17.0.0

Does it work if you configure tool.uv.no-build-package instead of setting it on the CLI?

[arnaldojvg]

I get the same issue of it not finding the binary 😭

[zanieb]

Does it work with uv pip compile as in my example?

[arnaldojvg]

Nope, same error:

(venv) (base) ➜  pipfile_to_uv  uv cache clean                                               
Clearing cache at: /Users/arnaldovarela/.cache/uv
Removed 10 files (2.9MiB)
(venv) (base) ➜  pipfile_to_uv  uv venv                                                      
warning: `VIRTUAL_ENV=venv` does not match the project environment path `.venv` and will be ignored
Using CPython 3.10.4
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
(venv) (base) ➜  pipfile_to_uv  source .venv/bin/activate                                    
(pipfile_to_uv) (base) ➜  pipfile_to_uv  echo pyarrow==17.0.0 | uv pip compile --only-binary pyarrow -
  × No solution found when resolving dependencies:
  ╰─▶ Because pyarrow==17.0.0 has no usable wheels and building from source is disabled and you require pyarrow==17.0.0, we can conclude that your requirements are unsatisfiable.

[zanieb]

Then there must be something different about your platform. Can you provide verbose logs (i.e. -v)?

[arnaldojvg]

This is what came out:

(pipfile_to_uv) (base) ➜  pipfile_to_uv  echo pyarrow==17.0.0 | uv pip compile -v --only-binary pyarrow -
DEBUG uv 0.4.25 (97eb6ab4a 2024-10-21)
DEBUG Starting Python discovery for a default Python
DEBUG Looking for exact match for request a default Python
DEBUG Searching for default Python interpreter in managed installations or system path
DEBUG Found `cpython-3.10.4-macos-aarch64-none` at `/Users/arnaldovarela/Documents/repos/pipfile_to_uv/.venv/bin/python3` (active virtual environment)
DEBUG Using Python 3.10.4 interpreter at .venv/bin/python3 for builds
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.10.4
DEBUG Solving with target Python version: >=3.10.4
DEBUG Adding direct dependency: pyarrow==17.0.0
DEBUG Found stale response for: https://pypi.org/simple/pyarrow/
DEBUG Sending revalidation request for: https://pypi.org/simple/pyarrow/
DEBUG Found not-modified response for: https://pypi.org/simple/pyarrow/
DEBUG Searching for a compatible version of pyarrow (==17.0.0)
  × No solution found when resolving dependencies:
  ╰─▶ Because pyarrow==17.0.0 has no usable wheels and building from source is disabled and you require pyarrow==17.0.0, we can conclude that your requirements are unsatisfiable.

[arnaldojvg]

I think is related to the cpython version being chosen + the wheels available. Seems like the "aarch" versions don't properly map for mac, being "manylinux" in the pyarrow wheels.

After a uv uv python list --all-versions I found that cpython-3.10.15-macos-x86_64-none was available.

I changed my commands to:

uv venv --python 3.10.15
source .venv/bin/activate
echo pyarrow==17.0.0 | uv pip compile -v --only-binary pyarrow -
uv sync
uv cache clean
uv venv --python 3.10.15
source .venv/bin/activate
uv sync

And it worked 🎉

I think this kinda solves my problem, but i wonder if the manylinux wheels should or shouldn't have been mapped to my cpython version 🤔

[zanieb]

Sweet!

You can't use manylinux wheels on macOS. Are you suggesting something else?

[bluss]

https://pypi.org/project/pyarrow/#files should have your explanation.

You have macos and Python 3.10, these are the available wheels:

• pyarrow-17.0.0-cp310-cp310-macosx_11_0_arm64.whl (27.2 MB view hashes)

Uploaded Jul 16, 2024 CPython 3.10 macOS 11.0+ ARM64

• pyarrow-17.0.0-cp310-cp310-macosx_10_15_x86_64.whl (29.0 MB view hashes)

Uploaded Jul 16, 2024 CPython 3.10 macOS 10.15+ x86-64

The first one fits your arm64 configuration but it requires macos version 11, so it can't be used for you and I think that's the reason. (Screenshot says you have macos 10.16)

[arnaldojvg]

Ok, turns out i'm actually on macos 15.0.1 but there is a configuration when they went from macos 10 to 11 to show a legacy version number. Most recent wheels for mac are buing built for 11+ on arm64. Setting the environment variable SYSTEM_VERSION_COMPAT=0 changed the macos version determined by python to: macOS-15.0.1-arm64-arm-64bit

with this my install problems have been solved so far.

thanks both!

[arnaldojvg]

I know this is closed already, but I made an article at my job on migrating Pipenv to uv as there were no other guides. I included the macOS issue and a minimal script to migrate from Pipfile to pyproject.toml

In case you care: https://medium.com/clarityai-engineering/migrating-from-pipenv-pipfile-to-uv-59ba2846636f

[ara-incom]

Struggled installing pyarrow in a similar fashion (failing to find pyarrow 17 binary, on macos). Turned out I had a .python-version specifying 3.13, which seemingly made uv use 3.13 for resolution, even though I had used uv venv -p 3.11 to setup venv. Setting 3.11 in .python-version solved the issue.

[tomalf2]

Struggled installing pyarrow in a similar fashion (failing to find pyarrow 17 binary, on macos). Turned out I had a .python-version specifying 3.13, which seemingly made uv use 3.13 for resolution, even though I had used uv venv -p 3.11 to setup venv. Setting 3.11 in .python-version solved the issue.

Following the answer of @ara-incom I managed to install pyarrow also with python 3.13.2. In my case, what's in .python-version seems to be unimportant. The trick is to run uv venv -p 3.13.2 before uv pip install pyarrow. It works also with python 3.11.11, and might work with other python versions too, but not the 3.12.2.

What's really important here is the -p <python-version>. Creating an empty dir and the file .python-version with the version 3.13.2 (or alternatively running uv python pin 3.13.2) before uv venv without additional options, didn't work for me. The cause of the error is that uv pip install pyarrow tries to build pyarrow and fails with error: command 'cmake' failed: No such file or directory.