uv icon indicating copy to clipboard operation
uv copied to clipboard

support uv on manylinux_2_17_aarch64

Open njzjz opened this issue 1 year ago • 3 comments

While uv on other architectures only requires glibc 2.17, uv on aarch64 requires glibc 2.28.

image

Installing uv in a glibc 2.17 environment is not possible.

    Downloading uv-0.1.40.tar.gz (787 kB)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 787.0/787.0 kB 21.3 MB/s eta 0:00:00
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'error'
    error: subprocess-exited-with-error
    
    × Preparing metadata (pyproject.toml) did not run successfully.
    │ exit code: 1
    ╰─> [6 lines of output]
        
        Cargo, the Rust package manager, is not installed or is not on PATH.
        This package requires Rust and Cargo to compile extensions. Install it through
        the system's package manager or via https://rustup.rs/
        
        Checking for Rust toolchain....
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: metadata-generation-failed

njzjz avatar May 07 '24 21:05 njzjz

We can try reducing the glibc requirement. I think we had trouble compiling on aarch manylinux_2_17 for unknown reasons.

charliermarsh avatar May 07 '24 23:05 charliermarsh

Yeah not sure, some error compiling ring -- see here: https://github.com/astral-sh/uv/pull/3444

charliermarsh avatar May 08 '24 00:05 charliermarsh

Yeah not sure, some error compiling ring -- see here: #3444

Got it. I found the root cause is https://github.com/briansmith/ring/issues/1728

njzjz avatar May 08 '24 00:05 njzjz

Note that in the meantime, using the standalone installer (curl -LsSf https://astral.sh/uv/install.sh | sh) should work; it will fallback to the statically compiled (musl) version, which works independent of glibc version.

konstin avatar May 08 '24 09:05 konstin

If the installer falls back to using the statically compiled version, is there any interest in providing also a dynamically link (gnu) version ? The fact that the 2 variants are provided is misleading: one would assume it only works on one or the other no ?

If the statically compiled musl version works everywhere, then why not use this one for the wheel ?

On aarch64 at least, the binary is smaller with musl (despite the fact that libc is statically linked) so it would be a win across the board (unless there are also performance implications): less wheels (less storage on PyPI) & better compatbility as it could be tagged manylinux_2_17_aarch64.musllinux_1_1_aarch64

mayeut avatar May 09 '24 08:05 mayeut

Separately, this is now supported: https://github.com/astral-sh/uv/pull/3444

charliermarsh avatar May 09 '24 13:05 charliermarsh

We had to revert this as it led to SSL failures (see #3576)

charliermarsh avatar May 14 '24 14:05 charliermarsh

What about trying https://github.com/astral-sh/uv/issues/3439#issuecomment-2102159802?

This will block https://github.com/pypa/manylinux/pull/1614 (though reverting for now is the correct short term fix).

henryiii avatar May 14 '24 15:05 henryiii

@konstin - wdyt?

charliermarsh avatar May 14 '24 16:05 charliermarsh

This should be really easy to test later, I just need to use my AS machine and try it.

henryiii avatar May 14 '24 17:05 henryiii

I hope i'm understanding the suggestion correctly, but i think we can add manylinux tags to the musllinux wheels, as long as they are lower than the main manylinux image, the should only be used in this small range.

We could also go further and only build statically linked wheels and tag them both musllinux and manylinux, dropping the "real" manylinux image with dynamically linked glibc altogether.

konstin avatar May 14 '24 17:05 konstin

Yes, it works:

pipx run wheel tags --platform-tag +manylinux_2_17_aarch64.manylinux2014_aarch64 uv-0.1.43-py3-none-musllinux_1_1_aarch64.whl

And the following Dockerfile:

ARG PYTHON_VERSION=3.11
FROM python:${PYTHON_VERSION}-slim as base

RUN apt-get update
RUN apt-get install -y curl build-essential cmake

ADD ./uv-0.1.43-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.musllinux_1_1_aarch64.whl \
      uv-0.1.43-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.musllinux_1_1_aarch64.whl
RUN pip install ./uv-0.1.43-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.musllinux_1_1_aarch64.whl

ADD ./scripts/requirements/compiled/black.txt /requirements.txt

ENV RUST_LOG=trace
RUN uv pip install -r /requirements.txt --system -v

Works, while the previous dynamic wheel did not. This would reduce the number of wheels built and hosted, too.

Also checked with the manylinux image:

FROM quay.io/pypa/manylinux2014_aarch64 as base

ADD ./uv-0.1.43-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.musllinux_1_1_aarch64.whl uv-0.1.43-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.musllinux_1_1_aarch64.whl
RUN pipx install ./uv-0.1.43-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.musllinux_1_1_aarch64.whl

ADD ./scripts/requirements/compiled/black.txt /requirements.txt

ENV RUST_LOG=trace
RUN uv pip install --python python3.12 -r /requirements.txt -v

henryiii avatar May 14 '24 18:05 henryiii