[INFRA/eyes] Provide multi-architecture Docker images

[fgksgf]

Search before asking

• [X] I had searched in the issues and found no similar feature requirement.

Description

We should provide multi-architecture Docker images of skywalking-eyes, just like other SkyWalking sub-projects.

Use case

No response

Related issues

No response

Are you willing to submit a PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[ghost]

Hi @fgksgf I am new to open source. I know java and basic docker technology. I am interested in this project and want to make my first contribution .How should I start?

[Superskyyy]

Hi @fgksgf I am new to open source. I know java and basic docker technology. I am interested in this project and want to make my first contribution .How should I start?

Hello there @00TanmaySinha00, and welcome to the Apache SkyWalking community! It's a good practice to get familiar with the ecosystem project first, in this case SkyWalking-Eyes. Then you should take a detailed look at the building script (Makefile) for building and pushing docker images and go from there to support cross-platform build by docker buildx, you may also need some minor naming changes to the GitHub Actions workflow.

You can take a reference from https://github.com/apache/skywalking/pull/8141

[Superskyyy]

@00TanmaySinha00 Hello, please update the status, shall we assign this issue to you?

[gcamach0]

Hi @fgksgf I was taking a look at the project and I would like to contribute, I already saw in the comments the links with which to familiarize myself with the project and with the issue. I can take a look and update you when I've read it, so you can assign me the issue if that's ok with you.

[Superskyyy]

@gcamach0 Hi I assigned to you, please feel free to start digging and take steps to try implement this.

[gcamach0]

Thanks, I'll let you know any update.

[gcamach0]

I'm getting an error when trying to run the docker build command, with or without buildx. So I was wondering if maybe there's something I missed or if you have faced the same error before.

[kezhenxu94]

I'm getting an error when trying to run the docker build command, with or without buildx. So I was wondering if maybe there's something I missed or if you have faced the same error before.

Hi, I never saw this error, and we have a workflow in GitHub to build the Docker for every commit so this should not be a problem, did you check your network is OK?

[fgksgf]

I'm getting an error when trying to run the docker build command, with or without buildx. So I was wondering if maybe there's something I missed or if you have faced the same error before.

Are you using any proxy? I searched a similar issue, hope it can help: https://github.com/gliderlabs/docker-alpine/issues/191#issuecomment-433450655

[gcamach0]

Yeah it seems that my network is the problem, I just wanted to check if nothing similar had ever happened before. I think the problem is that I have an application that uses a proxy to route traffic and apply security policies.

I have already requested help from the IT department where I work and I hope tomorrow I will have an answer so that I can test my build command.

[wu-sheng]

Sometimes, you could check on your fork's GitHub action task, which doesn't have network issue.

[gcamach0]

Oh I didn't know that was possible, thanks. I'll try it that way.

[mohammedtabish0]

Hey, it's been many days since @gcamach0 picked it up. I can't see any new commits in your forked branch so let me know if you are still working on this or I can pick it up. Thanks

[Superskyyy]

Hey, it's been many days since @gcamach0 picked it up. I can't see any new commits in his forked branch so let me know if you are still working on this or I can pick it up. Thanks

I guess the work hasn't continued on @gcamach0's side, so feel free to take it and proceed. Thank you. Please acknowledge and I will assign it to you instead.

[mohammedtabish0]

Sure, I will pick this up.

[Superskyyy]

Sure, I will pick this up.

Cool! Assigned to you.

[mohammedtabish0]

@kezhenxu94 I was trying to create multi platform images through docker buildx command locally using the skywalking-eyes repo's context(DockerFile) as we cannot use --load option in buildx for multiplatforms I tried --push command to push the images to my local docker registry. The command I used was docker buildx build --push --platform linux/amd64,linux/arm64 -t 127.0.0.1:5000/multi-arch:latest . I can see images getting created for amd64 and arm64 but when I open the registry I can find only one image(attached below). Am I doing something wrong? Should I be asking these kind of questions when contributing to open-source?

Output of the above command

[+] Building 522.8s (27/27) FINISHED
 => [internal] booting buildkit                                                                                    9.5s
 => => pulling image moby/buildkit:buildx-stable-1                                                                 8.6s
 => => creating container buildx_buildkit_mybuilder0                                                               0.9s
 => [internal] load build definition from Dockerfile                                                               0.1s
 => => transferring dockerfile: 1.35kB                                                                             0.0s
 => [internal] load .dockerignore                                                                                  0.1s
 => => transferring context: 2B                                                                                    0.0s
 => [linux/amd64 internal] load metadata for docker.io/library/alpine:3                                            5.8s
 => [linux/arm64 internal] load metadata for docker.io/library/alpine:3                                            5.5s
 => [linux/arm64 internal] load metadata for docker.io/library/golang:1.18-alpine                                  5.9s
 => [linux/amd64 internal] load metadata for docker.io/library/golang:1.18-alpine                                  5.3s
 => [linux/amd64 bin 1/5] FROM docker.io/library/alpine:3@sha256:bc41182d7ef5ffc53a40b044e725193bc10142a1243f395e  1.4s
 => => resolve docker.io/library/alpine:3@sha256:bc41182d7ef5ffc53a40b044e725193bc10142a1243f395ee852a8d9730fc2ad  0.1s
 => => sha256:213ec9aee27d8be045c6a92b7eac22c9a64b44558193775a1a7f626352392b49 2.81MB / 2.81MB                     0.8s
 => => extracting sha256:213ec9aee27d8be045c6a92b7eac22c9a64b44558193775a1a7f626352392b49                          0.4s
 => [linux/amd64 build 1/4] FROM docker.io/library/golang:1.18-alpine@sha256:8e45e2ef37d2b6d98900392029db2bc88f4  25.4s
 => => resolve docker.io/library/golang:1.18-alpine@sha256:8e45e2ef37d2b6d98900392029db2bc88f42c0f2a9a8035fa7da90  0.1s
 => => sha256:3a421caacf35538ce671745cb9396eea38bf42cd3093eaf78dbcbe8bf5f83f82 156B / 156B                         0.5s
 => => sha256:af72e8bb74dba7b7a51456e414ac4999ff55a6a68e532a7cd1b6f32707be625f 115.31MB / 115.31MB                18.2s
 => => sha256:1cab0e43db0af1d30e55de347bd78df438344691f8fb379f631a07e2c8a80f0a 155B / 155B                         0.3s
 => => sha256:5299e6f7860564fd4df7e9a224f3d05dc26d0e855fb26ac7e9d9e156cf1422b2 284.73kB / 284.73kB                 0.4s
 => => extracting sha256:5299e6f7860564fd4df7e9a224f3d05dc26d0e855fb26ac7e9d9e156cf1422b2                          0.2s
 => => extracting sha256:1cab0e43db0af1d30e55de347bd78df438344691f8fb379f631a07e2c8a80f0a                          0.0s
 => => extracting sha256:af72e8bb74dba7b7a51456e414ac4999ff55a6a68e532a7cd1b6f32707be625f                          6.6s
 => => extracting sha256:3a421caacf35538ce671745cb9396eea38bf42cd3093eaf78dbcbe8bf5f83f82                          0.0s
 => [internal] load build context                                                                                  2.3s
 => => transferring context: 6.85MB                                                                                2.2s
 => [linux/arm64 build 1/4] FROM docker.io/library/golang:1.18-alpine@sha256:8e45e2ef37d2b6d98900392029db2bc88f4  27.8s
 => => resolve docker.io/library/golang:1.18-alpine@sha256:8e45e2ef37d2b6d98900392029db2bc88f42c0f2a9a8035fa7da90  0.1s
 => => sha256:316bf57efd3b0cdd0835f8d3393e898cd27ed09707eff97189df62f132805299 125B / 125B                         0.3s
 => => sha256:cea4debaad2864fd0099780f6208841dc3a80054b81f7531944737f0f633e22a 110.32MB / 110.32MB                20.6s
 => => sha256:4392115cee3dde58f6f650f24b78fb0a4dd12537cca1b3eec0be6ffb470055aa 154B / 154B                         0.3s
 => => sha256:817a8a8f634c3a78b1a3b55a03335868971f2378932d3454ece4e3bfc5931675 284.52kB / 284.52kB                 0.5s
 => => extracting sha256:817a8a8f634c3a78b1a3b55a03335868971f2378932d3454ece4e3bfc5931675                          0.2s
 => => extracting sha256:4392115cee3dde58f6f650f24b78fb0a4dd12537cca1b3eec0be6ffb470055aa                          0.0s
 => => extracting sha256:cea4debaad2864fd0099780f6208841dc3a80054b81f7531944737f0f633e22a                          5.9s
 => => extracting sha256:316bf57efd3b0cdd0835f8d3393e898cd27ed09707eff97189df62f132805299                          0.0s
 => [linux/arm64 bin 1/5] FROM docker.io/library/alpine:3@sha256:bc41182d7ef5ffc53a40b044e725193bc10142a1243f395e  1.3s
 => => resolve docker.io/library/alpine:3@sha256:bc41182d7ef5ffc53a40b044e725193bc10142a1243f395ee852a8d9730fc2ad  0.1s
 => => sha256:9b18e9b68314027565b90ff6189d65942c0f7986da80df008b8431276885218e 2.71MB / 2.71MB                     0.7s
 => => extracting sha256:9b18e9b68314027565b90ff6189d65942c0f7986da80df008b8431276885218e                          0.4s
 => [linux/amd64 build 2/4] WORKDIR /license-eye                                                                   1.2s
 => [linux/amd64 build 3/4] COPY . .                                                                               0.3s
 => [linux/amd64 build 4/4] RUN apk add --no-cache make curl && make linux                                        19.8s
 => [linux/arm64 build 2/4] WORKDIR /license-eye                                                                   0.2s
 => [linux/arm64 build 3/4] COPY . .                                                                               0.2s
 => [linux/arm64 build 4/4] RUN apk add --no-cache make curl && make linux                                       246.8s
 => [linux/amd64 bin 2/5] COPY --from=build /license-eye/bin/linux/license-eye /bin/license-eye                    0.1s
 => [linux/amd64 bin 3/5] COPY --from=build /usr/local/go/bin/go /usr/local/go/bin/go                              0.1s
 => [linux/amd64 bin 4/5] RUN apk add --no-cache bash gcc musl-dev npm cargo                                     150.1s
 => [linux/amd64 bin 5/5] WORKDIR /github/workspace/                                                               0.2s
 => [linux/arm64 bin 2/5] COPY --from=build /license-eye/bin/linux/license-eye /bin/license-eye                    0.1s
 => [linux/arm64 bin 3/5] COPY --from=build /usr/local/go/bin/go /usr/local/go/bin/go                              0.1s
 => [linux/arm64 bin 4/5] RUN apk add --no-cache bash gcc musl-dev npm cargo                                     156.8s
 => [linux/arm64 bin 5/5] WORKDIR /github/workspace/                                                               0.1s
 => exporting to image                                                                                            38.2s
 => => exporting layers                                                                                           36.9s
 => => exporting manifest sha256:d7209175b5c4c3d59331f52c957b248b095336337ca0a531d437297a787d56a9                  0.0s
 => => exporting config sha256:cfbf3cdb3884edf8c914604fafde5e865b0d23c5f4157b7228a4c66ef55c9699                    0.0s
 => => exporting manifest sha256:0302c880b41778569b26e7c6c11fb478f6f1fbac504503acebd0a2183e0ccd5d                  0.0s
 => => exporting config sha256:6b7bf366b0cbafbeca666d737b9fb0232b45ba4a8b30be0cd8ff9cd314170f1a                    0.0s
 => => exporting manifest list sha256:731f4e42578534d9e1a5e990ba2d00ca2d97e0c2015d412c3873b1918e44a6e6             0.0s
 => => pushing layers                                                                                              1.2s
 => => pushing manifest for 127.0.0.1:5000/multi-arch:latest@sha256:731f4e42578534d9e1a5e990ba2d00ca2d97e0c2015d4  0.0s

[kezhenxu94]

@kezhenxu94 I was trying to create multi platform images through docker buildx command locally using the skywalking-eyes repo's context(DockerFile) as we cannot use --load option in buildx for multiplatforms I tried --push command to push the images to my local docker registry. The command I used was docker buildx build --push --platform linux/amd64,linux/arm64 -t 127.0.0.1:5000/multi-arch:latest . I can see images getting created for amd64 and arm64 but when I open the registry I can find only one image(attached below). Am I doing something wrong? Should I be asking these kind of questions when contributing to open-source?

Just feel free to ask any question you have. I don't have much experience with local registry UI so I don't know whether Crane Operator displays all architectures of a Docker images, usually when I test I just push the images to my own account and check the OS/Arch like this

[mohammedtabish0]

Hey @kezhenxu94, Apologies for the delay in raising the PR. I had a few questions to ask sorry if these are too dumb

1. How will I test the created docker image? The docker images are getting created using the buildx command and I am able to run it but is there any command to run in the docker container to run the test cases?
2. Do we need to really use buildx command to build the docker image and not just when pushing it? As the --load keyword will not create multiplatform images, so we can just use the normal build command as we will not be creating a builder then removing it at the end,
3. In the MakeFile the docker-release recipe already contains the docker build and push command then why is there another tag and push command? Can't just docker-release : docker docker-push work?
4. Do we need to change architecture for build recipe too? As I can see we are creating ARCH variable but then overriding it by amd64 value.

[kezhenxu94]

1. How will I test the created docker image? The docker images are getting created using the buildx command and I am able to run it but is there any command to run in the docker container to run the test cases?

No, the final docker image is for end user' use so there is no test case in that image anymore. Just feel free to run the license-eye version and if it succeeds it's ok enough.

2. Do we need to really use buildx command to build the docker image and not just when pushing it? As the --load keyword will not create multiplatform images, so we can just use the normal build command as we will not be creating a builder then removing it at the end,

We don't have to. Using buildx for both loading and pushing is just for simplifying the script if you find it simpler to use plain docker build for local build and buildx for publishing, just do it!

4. In the MakeFile the docker-release recipe already contains the docker build and push command then why is there another tag and push command? Can't just docker-release : docker docker-push work?

docker-release : docker docker-push just works. As a convention, a Docker image will be also tagged as latest (if it's latest) so users can just run docker run apache/skywalking-eye ... without adding an explicit version, i.e. docker run apache/skywalking-eye:0.6.0 ....

5. Do we need to change architecture for build recipe too? As I can see we are creating ARCH variable but then overriding it by amd64 value.

The first ARCH is just for determining the OS and the second one is real ARCH. Sorry they have the same name to confuse. You can also rename one of them in your PR.

I think there is a mistake in the build recipe as it only builds one arch variation for macOS, which I expect it should generate for both amd and arm. Do you want to fix that first?

[mohammedtabish0]

I think there is a mistake in the build recipe as it only builds one arch variation for macOS, which I expect it should generate for both amd and arm. Do you want to fix that first?

Let's raise a PR for this docker one then will create an issue for the build one and pick it as I have very less idea in golang and will need some time.