github/workflows/cibuildwheel.yml: Add ubuntu aarch64

[rbange]

Build a wheel for linux on arm to enable e.g. dockerized usage on Apple Silicon an SBCs.

closes #158

[CLAassistant]

All committers have signed the CLA.

[tkralphs]

Looks like something went wrong with the runner, but it's not clear to me what the issue is.

[rbange]

Also the runner seems to be caught in an infinite loop due to the error. Can you cancel the run manually? Ill have a look later

[tkralphs]

Poking at this a bit, I'm not sure where the issue is. The error is from running an image that is not compatible with the platform, but it is not clear if this is a Github issue or a cibuildwheel issue. It could be the latter. Maybe try what's recommended here?

[mkoeppe]

Yes, the docker/setup-qemu-action is needed if one wants to build without setting things up properly for cross

[rbange]

Unfortunately, Linux ARM runners are still in private beta so that will be the way to go. I am a bit confused that macos can build for arm on amd64, but linux cannot. I will implement the recommended qemu setup.

[tkralphs]

I am a bit confused that macos can build for arm on amd64, but linux cannot

Actually, MacOS can't do this either. macos-14 runners are using Apple Silicon. Before those were introduced, we couldn't build wheels for those platforms either.

[tkralphs]

Looks like two issues:

• Looks like pkg-config is not available on the docker image in the qemu setup.
• For some reason, it is trying to upload artifacts with the same name, I didn't check it very carefully.

[mkoeppe]

• For some reason, it is trying to upload artifacts with the same name, I didn't check it very carefully.

The attempted upgrade of the upload-artifact action here causes this. Changes like this are best not mixed in with the proposed changes for the purpose of this PR.

upload-artifact v4 is not compatible with our use of the action.

[rbange]

Hi everyone,

yes the upload/download artifact version bumping is on me. However this needs to be addressed as well as the current used version is deprecated. Can / Should be addressed in another PR that is true.

However the regarding the actual error for ubuntu aarch64 I think the error lies in the before_all step of cibuildwheels. It seems like the custom script located https://github.com/coin-or/CyLP/blob/efe4314f7c76f393a2aa8e76e1a01a7aa18bd372/pyproject.toml#L13 cannot install cbc and fails silently:

  UNAME_MACHINE = aarch64
  UNAME_RELEASE = 6.5.0-1016-azure
  UNAME_SYSTEM  = Linux
  UNAME_VERSION = #16~22.04.1-Ubuntu SMP Fri Feb 16 15:42:02 UTC 2024
  configure: error: cannot guess build type; you must specify one
  checking build system type... ignoring errors

In contrast to the builds for other os/archs it does not even start compiling. However, I am not that deep into that setup.

[tkralphs]

Coincidentally, we just ran into this very same thing in a Cbc conda build on the same architecture. It seems our config.guess and config.sub are out of date (this is known). The fix in the conda build was to update the config.guess and config.sub with a version that was sitting on the build machine, but an easier fix is probably just to specify --build=<...>, but I don't know what the correct string should be.

[tkralphs]

I would try this.

[rbange]

I reverted my changes regarding the action version bump to get rid of the failing pipelines for the other architectures. However, it seems like you fix did not resolve the missing library in the last run: https://github.com/coin-or/CyLP/actions/runs/8880693060/job/24381324439

[tkralphs]

I should have done this before now, but I finally just tried building on a Raspberry Pi and it works fine. So I guess maybe this is just back to where we started, something is going wrong because we are not actually on an ARM runner. I have to admit that I don't really have a good understanding of what's going on here. I just set up one final try, using the build platform specification that my Raspberry Pi reports when I configure there. We'll see.

[tkralphs]

Well, OK, that seems to have worked!

[tkralphs]

Something's weird. All the builds seem to be working, but I hard-coded the build specification to be aarch64-unknown-linux-gnu, just to test whether this works, so the builds on other platforms should have failed I thought...

[rbange]

For whatever reason it also took a very long time to build all of the wheels. Maybe due to the virtualisation layer.

[tkralphs]

It seems that the environment variable CIBW_ARCHS is not set inside the before-all block and so config_args is also not being set and thus, the build platform is not being passed to configure. I don't really know how the environment variables that are set in Github Actions get passed through to the cibuildwheel environment inside before-all. I tried explicitly setting the variables in pyproject.toml, but it seems the environment variables don't get inherited into pyproject.toml either.

I'm a bit out of my depth here. I got the building of wheels working. If we can just get the environment variables set up properly inside before-all, everything should work. It seems like this was already broken before, but we didn't notice. @mkoeppe, I think you set up the before-all block and modified it for macos. Any idea what's going on here?

[mkoeppe]

@tkralphs Here's something that seems to work.

[tkralphs]

The failure in integration testing with CVXPY was already present in master and is being resolved in #200 so I will go ahead and merge this.