Support ARM architectures as a host archicture

[junaruga]

Is this a bug report, feature (enhancement) request or question? (leave only one on its own line)

/kind enhancement

Description:

There are tickets that has an error because the executed host architecture is not x86_64. Right now only x86_64 are supported. Because we only use qemu-user-static RPM x86_64 package in .travis.yml.

https://github.com/multiarch/qemu-user-static/blob/master/.travis.yml#L25

- PACKAGE_URI="https://kojipkgs.fedoraproject.org/packages/qemu/4.0.0/5.fc31/x86_64/qemu-user-static-4.0.0-5.fc31.x86_64.rpm"

To support other architectures, we need to add new piple line to .travis.yml, downloading the arch's qemu-user-static as input.

I think a similar project might support ARM architectures as a host architecture. I am not sure. https://github.com/dbhi/qus

Additional information optionally:

Related tickets: #8 , #36 ,

[umarcor]

I think a similar project might support ARM architectures as a host architecture. I am not sure. dbhi/qus

Yes, in dbhi/qus seven host architectures are supported with the Debian variant: amd64, i386, arm64v8, arm32v7, arm32v6, s390x and ppc64le. With the Fedora variant, only six are supported; arm32v6 is not.

For aarch64, the following images can be used:

• aptman/qus:arm64v8-d3.1
• aptman/qus:arm64v8-f4.0.0

To register any or all of the nine supported client architectures.

[junaruga]

@umarcor thanks for the sharing! How are you building the each host arch's aptman/qus container image? Because Travis CI is x86_64 host. Maybe is it executed through the .travis.yml pipe line?

[umarcor]

Yes, everything is executed in Travis. The most important feature of dbhi/qus is that everything is public. There is no step at all that maintainers need to do offline.

It is a x86_64 host, but the project is about executing foreign arcuitectures, so we use it as a kind of 'inception'. For example, to build the aarch64 host image:

• Download qemu-user-static package for x86 hosts and extract the binary for aarch64 targets only.
• Register the binary for aarch64-on-amd64 in the host (Travis VM), either using the register script or a previous version of the images.
• Download qemu-user-static package for aarch64 hosts and extract the binaries for all the targets.
• Execute/build an image based on arm64v8/busybox and copy all the *arch-on-aarch64 binaries in it.
• Push image aptman/qus:VERSION-aarch64 to dockerhub. The same image is also tagged aptman/qus:aarch64.
• aptman/qus is just a manifest that 'redirects' to all the host images. So, when on aarch64, the image built in the previous step is downloaded when aptman/qus is used.

All of it is defined in the travis.yml file and in the run.sh script. Precisely, images are built in the first stage: https://github.com/dbhi/qus/blob/master/.travis.yml#L45-L70. As you see, there are two jobs, one for images based on binaries provided by Debian and another one for images based on binaries provided by Fedora. You can focus on the latter only: https://travis-ci.com/dbhi/qus/jobs/224057179. The 'block' corresponding to each host is shown in magenta, and the steps in each of them have blue headers. All of it is triggered with run.sh -a: https://github.com/dbhi/qus/blob/master/run.sh#L641-L658.

Hope this is useful to you! Do not hesitate to ask further questions.

[junaruga]

@umarcor thank you! It's very helpful! I will try to understand, taking a look at the source code!

[junaruga]

Good news. Travis started supporting ARM 64-bit. https://blog.travis-ci.com/2019-10-07-multi-cpu-architecture-support

[junaruga]

But I also love Drone CI supporting ARM 64/32-bit. Ref: https://bugs.ruby-lang.org/issues/16234

[junaruga]

@umarcor I am reading qus source code again as I have a time now. I am very slow to do it :-)

Register the binary for aarch64-on-amd64 in the host (Travis VM), either using the register script or a previous version of the images.

It seems this logic is run.sh - do_register function.

Execute/build an image based on arm64v8/busybox and copy all the *arch-on-aarch64 binaries in it.

I see you are building qus arch container on qemu. It's really great.

[umarcor]

First off, note that GitHub Actions are used now, apart from Travis: https://github.com/dbhi/qus/blob/master/docs/dev.md

It seems this logic is run.sh - do_register function.

do_register is used during tests. When building docker images getAndRegisterSingleQemuUserStatic is used instead. The difference is that do_register uses either docker images or binaries from https://github.com/dbhi/qus/releases. However, getAndRegisterSingleQemuUserStatic uses binaries from http://ftp.debian.org/debian/pool/main/q/qemu/qemu-user-static_* or https://kojipkgs.fedoraproject.org/packages/qemu/*.

I see you are building qus arch container on qemu. It's really great.

Yes. getAndRegisterSingleQemuUserStatic is used to extract 'target_host_arch on amd64' QEMU binaries, apart from 'all_target_client_arch on target_host_arch' binaries (which are the ones copied into the image). That's why I said that it is some kind of 'inception'.

[junaruga]

Thanks for the explanation! I love the metaphor 'inception'. We might also be living on QEMU ? :-)

[junaruga]

@moul in this project's case, shall we use Travis (arch: arm64) for aarch64 pipeline for now?

[junaruga]

For you information. Apple announced the own ARM based CPU 2 days ago. Supporting aarch64 as a host architecture is more important than ever.

https://www.apple.com/mac/m1/ https://www.theverge.com/2020/11/10/21550892/apple-arm-silicon-event-macbook-air-pro-mini-mac-big-sur-biggest-announcements

[umarcor]

Be aware that QEMU's scripts might not support running 32 bit ARM binaries on aarch64-only devices (yet). See dbhi/qus#4.

[natcl]

Hello, was curious to know if there was an update on this ? I'd like to execute x86 containers on my M1 Mac. thanks !

[umarcor]

@natcl, in dbhi/qus host architectures other than amd64 are supported.

[natcl]

Thanks will give it a try!