docker-glibc-builder
docker-glibc-builder copied to clipboard
sgerrand
Enable Arm64 images by docker buildx
In this patch, it enables the arm64 images by docker buildx tools. Firstly, it adds a build scripts which used for building multi-arch images. Secondly, modify the circle script file for building multi-arch images.
Signed-off-by: Jingzhao Ni [email protected]
Thanks for the patch. Have you tested that CircleCI support your changes, specifically:
- running Docker in privileged mode; and
- providing a version of Docker that runs
buildxcommands?
I'm not sure that they will. Hopefully that's not the case.
Hi, I manually test those scripts. But I don't test it on the CircleCI environment since it is still lack of "DOCKERHUB_USERNAME" and "DOCKERHUB_PASSWORD". I think you could add environment variables for it. And from docker version 19.03, buildx feature has been supported.
I don't test it on the CircleCI environment since it is still lack of "DOCKERHUB_USERNAME" and "DOCKERHUB_PASSWORD". I think you could add environment variables for it.
There's no need to login because the Docker image is never pushed. It's only used to build glibc. Why is the image being pushed to a remote repository? It's not clear from your changes.
To answer the questions I had about CircleCI's support, I've pushed your branch to my remote to trigger a build: https://app.circleci.com/pipelines/github/sgerrand/docker-glibc-builder?branch=Jingzhao123%2FEnableArm64Images
Perhaps you could also configure your fork to run on CircleCI as well. There's no cost to you involved.
@Jingzhao123, after reviewing your changes I can't see how they enable building glibc for multiple architectures. You've changed the Docker image to run as different architectures but no change was made to the glibc build. As building glibc is the sole purpose of this repository, I imagine that you've updated the Docker image to support building glibc for linux/arm64. How do you envisage making this additional step?
Something else worth noting is that CircleCI only support running privileged containers (e.g. docker run --privileged) using the machine operator: https://circleci.com/docs/2.0/executor-types/#docker-benefits-and-limitations
One final thing: I've tweaked your original changes to get them in a form where they can be run on CircleCI. You can see my additional commits here: https://github.com/sgerrand/docker-glibc-builder/compare/tweak-enable-arm64
Unfortunately, as you can see in this failed build, it appears that CircleCI do not support Docker runtimes for other architectures:
[+] Building 4.6s (7/10)
=> [internal] booting buildkit 3.2s
=> => pulling image moby/buildkit:buildx-stable-1 2.2s
=> => creating container buildx_buildkit_multi-builder0 1.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 50B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 462B 0.0s
=> [linux/arm64 internal] load metadata for *********/library/ubuntu:20. 1.1s
=> [linux/amd64 internal] load metadata for *********/library/ubuntu:20. 1.2s
=> CANCELED [internal] load build context 0.0s
=> => transferring context: 0.1s
=> [linux/amd64 1/4] FROM *********/library/ubuntu:20.04@sha256:52259450 0.0s
=> => resolve *********/library/ubuntu:20.04@sha256:52259450119427dab05c 0.0s
failed to solve: rpc error: code = Unknown desc = failed to load LLB: runtime execution on platform linux/arm64 not supported
Exited with code exit status 1
CircleCI received exit code 1
The crux of my question still holds: how do these changes help with building glibc for multiple architectures?
I look forward to your thoughts on this.
The crux of my question still holds: how do these changes help with building
glibcfor multiple architectures?I look forward to your thoughts on this.
Hi, I have updated my patches for it. It will generate the multi-arch glibc packages on host. Please check it. Thanks.
Thanks for pushing these additional changes. In particular, seeing your evolved approach to the Dockerfile has made how you are planning to build glibc for other architectures much clearer to me.
Thanks for pushing these additional changes. In particular, seeing your evolved approach to the
Dockerfilehas made how you are planning to buildglibcfor other architectures much clearer to me.
Hi, the qemu tool also supports other architectures, e.g. ppc, armv7. So, I think it could use the same way to build glibc packages after the aarch64 build successful.
Hi, @sgerrand I notice that there is a docker image repository in dockerhub. Could it directly build and push images to dockerhub account with buildx?
I notice that there is a docker image repository in dockerhub. Could it directly build and push images to dockerhub account with buildx?
Well spotted. Images there are automatically built directly on Docker Hub: https://hub.docker.com/r/sgerrand/glibc-builder/builds
I assume from your question that you're proposing to change where images are built, i.e. to use CircleCI instead of Docker Hub. I don't think there's much point in doing so. Perhaps you can suggest how that would be an improvement?
I notice that there is a docker image repository in dockerhub. Could it directly build and push images to dockerhub account with buildx?
Well spotted. Images there are automatically built directly on Docker Hub: https://hub.docker.com/r/sgerrand/glibc-builder/builds
I assume from your question that you're proposing to change where images are built, i.e. to use CircleCI instead of Docker Hub. I don't think there's much point in doing so. Perhaps you can suggest how that would be an improvement?
Hi, I am not familiar with dockerhub auto build. But, as far as I know, it does not well support multi-arch build. Please refer to https://github.com/docker/roadmap/issues/109. So, just a suggestion, if we want to enable multi-arch images, why not use the circleCI directly build and push those images?
My apologies for not having reviewed this when I said I would. I've been busier than expected for the past week and my extracurricular activities have suffered as a result. I'll have time tomorrow to do this.
@sgerrand Hi!, I've been following the cross-platform issues / prs and a while ago forked the repository to make these builds myself. (although I am using github actions instead of CircleCI as I am more familiar with it)
It is indeed possible to compile glibc via buildx for lots more architectures! (I'm doing x86_64 arm64 arm armhf s390x ppc64le and buildx will also support riscv) This can be achieved with minimal changes to the build script.
I'd also like to say that you can run qemu on the buildx host (will allow docker to run dockerfiles as / from another arch) - that would also limit the amount of changes needed to the dockerfile.
Many thanks for passing on your experience with building for multiple architectures, @Prouser123. The insights from you and @Jingzhao123 have been invaluable.
My main concern at this stage is finding a way to validate the built library works on the target architecture, ideally using a host which will support a wide range of architectures as to avoid rework of an integration test setup.
My main concern at this stage is finding a way to validate the built library works on the target architecture, ideally using a host which will support a wide range of architectures as to avoid rework of an integration test setup.
We could just use qemu-static and docker, and run tests that way? If we do it on the host we can use the exact same dockerfile between different architectures, no platform-specific stuff needed - We can just do docker run [..] on an image / tag for another platform!
Although this does have some limitations, namely with qemu and glibc >2.27 readdir() returns NULL (errno=EOVERFLOW) for 32-bit user-static qemu on 64-bit host (launchpad bugtracker), so physical hardware may be preferred. However, there is a possible workaround if this becomes an issue.
Personally I use my builds upstream to make multiarch openjdk images, and have found that running java and jlink (to create custom jvm images) has rooted out quite a few issues for me.