arm64 docker images

[morganchristiansson]

Having ansible image that run on arm64 would be real nice. I'm running CI on arm64/raspberry pi. And arm is becoming popular with AWS Graviton2 instances.

So I've forked the repo and have trying some things out for multi arch...

First.. Building on arm64 is much slower due to running in qemu in github actions. If workers running on arm64 were available it would run as-fast as amd64.

There's some 3x different ways to build images:

• docker build per platform
• docker buildx per platform
• docker buildx multiple platforms

The benefit of running per-platform is that the very slow arm64 builds run in separate GH actions. It will require a final step to creates multi-arch image manifest that point to amd64/arm64 specific images.

docker buildx build with image registry for cache has really helped speed up the github actions. I'm testing using github container registry for cache and final image sto docker hub. I think this will be important for arm64 builds to run at reasonable speed.

There's also some 350+ actions to run on PR after adding below to github workflow matrix.

matrix:
  platform:
    - amd64
    - arm64

Also kubectl and oc need to be fetched from different source that provide arm64 go binaries.

Also there's some duplication in Makefile and github workflows which is making the work to add arm64 a bit extra tedious.

I'm happy to keep trying things out in my fork .. would like to upstream at some point and wondering what everyone thinks..

There's probably better place than github actions to run arm64 builds .. but in interest of a future PR it may be best place? CircleCI and TravisCI have native arm64 workers.

[morganchristiansson]

I got working build on amd64+arm64 in github actions here: https://github.com/morganchristiansson/docker-ansible/pull/2

Which has pushed images with -arm64 and -amd64 suffix to https://hub.docker.com/r/morganchristiansson/ansible/tags

[cytopia]

That is some awesome work :-)

[cytopia]

Which has pushed images with -arm64 and -amd64 suffix to https://hub.docker.com/r/morganchristiansson/ansible/tags

Can't AMD and ARM have the same name and tag? I thought the client will auto-figure out what arch to pull.

[morganchristiansson]

Yeah I mentioned the join step. it creates a manifest that that points to the real image.. when using docker buildx build --platform x,y these images are untagged and not visible.

Now that we have built our images and pushed them, we are able to reference them all in a manifest list using the docker manifest command.

$ docker manifest create
your-username/multiarch-example:manifest-latest
--amend your-username/multiarch-example:manifest-amd64
--amend your-username/multiarch-example:manifest-arm32v7
--amend your-username/multiarch-example:manifest-arm64v8

Once the manifest list has been created, we can push it to Docker Hub.

$ docker manifest push your-username/multiarch-example:manifest-latest

From https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/

(article goes on to decsribe "the simple way with buildx".. I tried this approach in another branch.. but the arm64 builds were really slowing the amd64 images down. Building the images separately and joining them after seems better given the buildtimes).

I'm using the morganchristiansson/ansible:2.10-tools-arm64v8 right now and works great. I don't really need all these images was hoping it can be merged. I think my work is mostly working .. just missing some fixes for kubectl and oc binary.. and handling older kops versions that have failing builds because they did not release arm64 binaries.

[morganchristiansson]

arm64 multi-arch images are available at https://hub.docker.com/r/morganchristiansson/ansible/tags

built from my fork https://github.com/morganchristiansson/docker-ansible/ using cytopia/ansible for amd64 docker images I only build arm64