cp38-macosx_arm64 produces a x86_64 wheel

[ns-yanghsing]

Description

I'm building a library wrapping c++ code on macos with Github Action. In the cp38-macosx_arm64 (os: macos-14) build, the "Building wheel" step went through, but the "Repairing wheel" step failed with

  delocate.libsana.DelocationError: Failed to find any binary with the required architecture: 'x86_64'

The issue only occurs on cp38 where cp39-macosx_arm64 went through without problems.

Here's the log for "building wheel", which passed but seemed to produce a x86_64 wheel for an arm64 build:

+ python -m pip wheel /Users/runner/work/py_jwt_cpp/py_jwt_cpp --wheel-dir=/private/var/folders/g6/rgtlsw6n123b0gt5483s5_cm0000gn/T/cibw-run-b894xoq9/cp38-macosx_arm64/built_wheel --no-deps
  Processing /Users/runner/work/py_jwt_cpp/py_jwt_cpp
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Building wheels for collected packages: py_jwt_cpp
    Building wheel for py_jwt_cpp (pyproject.toml): started
    Building wheel for py_jwt_cpp (pyproject.toml): still running...
    Building wheel for py_jwt_cpp (pyproject.toml): finished with status 'done'
    Created wheel for py_jwt_cpp: filename=py_jwt_cpp-0.1.0-cp38-cp38-macosx_14_0_x86_64.whl size=66406 sha256=3a67c0ae39221cb61d1971c5e9e082752298ebf34dd2abe2a18b95f34350f363
    Stored in directory: /private/var/folders/g6/rgtlsw6n123b0gt5483s5_cm0000gn/T/pip-ephem-wheel-cache-cankgeva/wheels/24/a5/8d/764d337a233749b5593943775402d69ea2e[128](https://github.com/ns-yanghsing/py_jwt_cpp/actions/runs/11810912790/job/32903660483#step:6:132)85755535c006
  Successfully built py_jwt_cpp

And here's the "Repairing" log looking for an x86_64 one for an arm64 build:

+ delocate-wheel --require-archs arm64 -w /private/var/folders/g6/rgtlsw6n123b0gt5483s5_cm0000gn/T/cibw-run-b894xoq9/cp38-macosx_arm64/repaired_wheel -v /private/var/folders/g6/rgtlsw6n123b0gt5483s5_cm0000gn/T/cibw-run-b894xoq9/cp38-macosx_arm64/built_wheel/py_jwt_cpp-0.1.0-cp38-cp38-macosx_14_0_x86_64.whl
  INFO:delocate.delocating:Copying library /opt/homebrew/Cellar/openssl@3/3.3.2/lib/libssl.3.dylib to py_jwt_cpp/.dylibs/libssl.3.dylib
  INFO:delocate.delocating:Copying library /opt/homebrew/Cellar/openssl@3/3.3.2/lib/libcrypto.3.dylib to py_jwt_cpp/.dylibs/libcrypto.3.dylib
  INFO:delocate.delocating:Modifying install name in py_jwt_cpp/jwt_cpp.cpython-38-darwin.so from /opt/homebrew/opt/openssl@3/lib/libssl.3.dylib to @loader_path/.dylibs/libssl.3.dylib
  INFO:delocate.delocating:Modifying install name in py_jwt_cpp/jwt_cpp.cpython-38-darwin.so from /opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib to @loader_path/.dylibs/libcrypto.3.dylib
  INFO:delocate.delocating:Modifying install name in py_jwt_cpp/.dylibs/libssl.3.dylib from /opt/homebrew/Cellar/openssl@3/3.3.2/lib/libcrypto.3.dylib to @loader_path/libcrypto.3.dylib
  Fixing: /private/var/folders/g6/rgtlsw6n123b0gt5483s5_cm0000gn/T/cibw-run-b894xoq9/cp38-macosx_arm64/built_wheel/py_jwt_cpp-0.1.0-cp38-cp38-macosx_14_0_x86_64.whl
  Traceback (most recent call last):
    File "/private/var/folders/g6/rgtlsw6n123b0gt5483s5_cm0000gn/T/cibw-run-b894xoq9/cp38-macosx_arm64/build/venv/bin/delocate-wheel", line 8, in <module>
      sys.exit(main())
    File "/private/var/folders/g6/rgtlsw6n123b0gt5483s5_cm0000gn/T/cibw-run-b894xoq9/cp38-macosx_arm64/build/venv/lib/python3.8/site-packages/delocate/cmd/delocate_wheel.py", line 116, in main
      copied = delocate_wheel(
    File "/private/var/folders/g6/rgtlsw6n123b0gt5483s5_cm0000gn/T/cibw-run-b894xoq9/cp38-macosx_arm64/build/venv/lib/python3.8/site-packages/delocate/delocating.py", line 1090, in delocate_wheel
      out_wheel_fixed = _check_and_update_wheel_name(
    File "/private/var/folders/g6/rgtlsw6n123b0gt5483s5_cm0000gn/T/cibw-run-b894xoq9/cp38-macosx_arm64/build/venv/lib/python3.8/site-packages/delocate/delocating.py", line 914, in _check_and_update_wheel_name
      new_name, problematic_files = _calculate_minimum_wheel_name(
    File "/private/var/folders/g6/rgtlsw6n123b0gt5483s5_cm0000gn/T/cibw-run-b894xoq9/cp38-macosx_arm64/build/venv/lib/python3.8/site-packages/delocate/delocating.py", line 848, in _calculate_minimum_wheel_name
      raise DelocationError(
  delocate.libsana.DelocationError: Failed to find any binary with the required architecture: 'x86_64'
  Error: Command delocate-wheel --require-archs arm64 -w /private/var/folders/g6/rgtlsw6n123b0gt5483s5_cm0000gn/T/cibw-run-b894xoq9/cp38-macosx_arm64/repaired_wheel -v /private/var/folders/g6/rgtlsw6n123b0gt5483s5_cm0000gn/T/cibw-run-b894xoq9/cp38-macosx_arm64/built_wheel/py_jwt_cpp-0.1.0-cp38-cp38-macosx_14_0_x86_64.whl failed with code 1. 

And here's the github flow config:

name: Build

on: [push, pull_request]

jobs:
  build_wheels:
    name: Build wheels on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [macos-14]
        # os: [ubuntu-latest, macos-13, macos-14]

    steps:
      - uses: actions/checkout@v4
        with:
          submodules: 'true'

      # Used to host cibuildwheel
      - uses: actions/setup-python@v5

      - name: Install cibuildwheel
        run: python -m pip install cibuildwheel==2.21.3

      - name: Set macOS deployment target
        if: startsWith(matrix.os, 'macos')
        run: echo "MACOSX_DEPLOYMENT_TARGET=$(sw_vers -productVersion | cut -d '.' -f 1-2)" >> $GITHUB_ENV

      - name: Build wheels
        run: python -m cibuildwheel --output-dir wheelhouse
        env:
          CIBW_BEFORE_BUILD_LINUX: |
            if [ -f "/etc/alpine-release" ]; then
              apk add --no-cache openssl-dev
            else
              yum install -y openssl openssl-devel
            fi
          CIBW_BUILD: "cp39*"
          CIBW_SKIP: "pp*"
      - uses: actions/upload-artifact@v4
        with:
          name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
          path: ./wheelhouse/*.whl

Please see the build link here for details. Thanks!

Build log

No response

CI config

No response

[patrikhuber]

@ns-yanghsing Did you ever solve the issue or find a workaround? I think I'm seeing a related error in #2111.

[Czaki]

@patrikhuber you need to setup python 3.8 using github actions setup-python action to setup python 3.8 on macos arm runner.

Official installer for python 3.8 don not support macos arm runner.

[long2ice]

So any solutions?

[Czaki]

@long2ice follow the documentation https://cibuildwheel.pypa.io/en/stable/faq/#macos-building-cpython-38-wheels-on-arm64 and install python 3.8 using actions/setup-python@v5

[MichaIng]

Related to this, which macOS version is targeted by default for which cbuildwheel targets?

Based on the Python packages names pulled I get the impression that it is macOS 10.9 for Python 3.8 wheels, and macOS 11 for all others. Hence, instead of e.g. hardcoding CMAKE_OSX_DEPLOYMENT_TARGET for cmake builds, I though skipping Python 3.8 wheels should solve things, and at the same time this issue: https://github.com/catchorg/Catch2/issues/2779

The macOS ARM build went trough indeed, but the error from mismatching expectations related std::uncaught_exceptions remained for macOS x86_64 builds, which was successfully solved with CMAKE_OSX_DEPLOYMENT_TARGET=10.12 before. So I am unsure now whether the targeted macOS version is still 10.9, or whether this is only the case for the particular dependency build. I would like to avoid hardcoding macOS 10.12 or 11.0 if it would be a higher target for higher Python versions 🤔.

[henryiii]

Cibuildwheel will bump it up to the minimum supported version if necessary. You shouldn’t have to do anything except set your actual minimum version.

You can do it, but since 3.8 is passed EoL, I highly recommend dropping it, it actually makes support worse if your update breaks someone else who has already dropped 3.8. :)

[MichaIng]

But how do I know which minimum it is in which case? Obviously, there are two macOS related issues since the mimimum versions chosen by cibuildwheel are too low, at least in certain host <> target combinations. So while I would love to leave that to cibuildwheel and not enforce anything, I need to do it to solve the issues.

I did drop Python 3.8 wheels already, assuming that would fix both issues, but one remained, as Python 3.9 wheels fail for the same reason, while enforcing even just macOS 10.12 solves it. I will enforce macOS 11.0 target now, but I would prefer to do that only for those wheels/Python version where it is really needed, and for this I would need to know which macOS version cibuilwheel targets in which case.

[henryiii]

If you require more than 10.9, like if you use C++17, you need to set the minimum to whatever you require. Cibuildwheel will increase it when required, like to 11 when building for ARM, or 10.13 when building for 3.12+ on Intel.

I believe it sounds like you should set yours to 10.12. There’s no need to split it up, cibuildwheel will do the correct thing for you.

[MichaIng]

Do I understand it right that there is absolutely no downside to target older macOS versions (as long as generally supported by toolchain/environment), like performance downsides if the compiler needs to use legacy methods or so?

We do not really need wheels for macOS 10.y in this case, so we'll leave it like that now, aligning all macOS wheels with Big Sur target.

I found the table with minimum version compatibility: https://cibuildwheel.pypa.io/en/stable/platforms/#macos-version-compatibility So for ARM builds, macOS 11 was target already, as you said, which explains why we did not run into this std::uncaught_exceptions() availability for ARM builds.

[Czaki]

Do I understand it right that there is absolutely no downside to target older macOS versions (as long as generally supported by toolchain/environment), like performance downsides if the compiler needs to use legacy methods or so?

There might be. As you mentioned, the older and slower API may be selected by a compiler.

We do not really need wheels for macOS 10.y in this case, so we'll leave it like that now, aligning all macOS wheels with Big Sur target.

It needs to write a querry to data from Google BigQuery to check the popularity version of macOS. To see how many users are using outdated operating systems.

[MichaIng]

The project did not provide wheels via PyPI at all previously, and it is generally something more tech/dev oriented, so pretty unlikely that someone who uses our new wheels is using them on a macOS version that is EOL since 2022. So going with an aligned macOS 11 minimum should be pretty fine. But I know now the background and defaults, and we could quickly adjust if ever needed. Thanks for the information and insights.