Provide a multi-arch Docker Image

[dduportal]

This image is currently targeted on a Linux container with x86_64 CPU architecture.

Docker galaxy recently saw the "manifest 2.2" landing, enabling multi-arch support on a single image name.

In our case, it means that the command docker pull asciidoctor/docker-asciidoctor would have the same outcomes on any supported platform.

The CPU architectures armv6 and armv7 would be easy to start with (using quemu it will be very easy to build and test within TravisCI).

x86 (32 bits) might work.

The Windows container version could be also easy to build, I am less sure on the test harness (we would need a Window server for this).

zOS: I don't know :)

Here is a list of useful links about this capability, to help to start with:

• Manifest v2.2 definition on Docker's Docs: https://docs.docker.com/registry/spec/manifest-v2-2/
• Phil Estes (a Docker Captain working for IBM) wrote this tool: https://github.com/estesp/manifest-tool
• An example from another Docker Captain, Adrian Mouat: http://container-solutions.com/multi-arch-docker-images/

[sdavids]

This would be useful for people using Apple Silicon.

$ docker run -it -v "$(pwd):/documents/" asciidoctor/docker-asciidoctor
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

[sdavids]

$ docker buildx build --platform linux/amd64,linux/arm64

might do the trick.

https://docs.docker.com/buildx/working-with-buildx/#build-multi-platform-images

[mojavelinux]

I'm watching this issue with great interest because I often get asked about this for other images I maintain. Is it really as simple as using buildx, or is there a hardware requirement for the machine on which Docker is run?

[dduportal]

That would really be simple yes, as GitHub action provides a Docker Engine, and we use buildx already because we have a docker bake file. It means editing https://github.com/asciidoctor/docker-asciidoctor/blob/main/docker-bake.hcl to ensure that there is an arm64 platform, and that should be all: next release would have an arm64 version (I'm configdent since we did the same with Jenkins Docker images a few month ago with the bake file: https://github.com/jenkinsci/docker/blob/master/docker-bake.hcl#L206

The only "hard" part is Windows which is another scope.

@sdavids Would you be interested into adding your contribution (with mentoring and help if you want it)? Otherwise I would do it my self, but anyone is welcome to try :)

[p-]

@mojavelinux @dduportal We are interested in a multi-arch Docker Image as well (especially for Arm64). (Testing on the Apple Silicon M1 macs showed that CPU-emulation with QEMU might work in some cases but is rather slow)

[gounthar]

For the time being, the cabal package which is needed is not available for aarch64, so we're kind of stuck until it becomes available.

[rhmic]

How could we motivate / accelerate the cabal team providing their package for aarch64 ? I built an image without cabal and the depending functionalities and it works fine for my use cases. But that's clearly only a workaround. :-)

[gounthar]

I managed to build an aarch64 cabal package thanks to RubiksCraft efforts. I think the way to go is to make a pull request to Alpine directly.

[Weltraumschaf]

@barthel Just did it https://hub.docker.com/r/uwebarthel/docker-asciidoctor/tags

[dduportal]

@barthel Just did it https://hub.docker.com/r/uwebarthel/docker-asciidoctor/tags

Hi @Weltraumschaf could you share the source (Dockerfile) and/or help us by contributing? I'm interested in how did you build the erd project for ARM CPUs?

[barthel]

@dduportal This is not yet solved and I've not tried yet. I remove erd when building because I build this container mainly for my raspberry pi and erd is only available/buildable for x86 and aarch64.

The Dockerfile is the original one. The multi arch build is done by the Circle CI configuration: https://github.com/barthel/docker-asciidoctor-base/blob/main/.circleci/config.yml

[gounthar]

What Raspberry Pi do you use @barthel ? Mine is a 3B+, and it's already an aarch64.

[barthel]

Raspberry Pi Zero W (armv6l) and Raspberry Pi 3B (armv7l).

Not tried aarch64.

[gounthar]

I should definitely retry all that thanks to a GitHub action with Docker multiplatform. 🤔

[barthel]

@dduportal I solved the issue with erd in multi platform build. Before the build starts I replace erd (https://github.com/BurntSushi/erd) by erd-go (https://github.com/kaishuu0123/erd-go/). I'm use the original Dockerfile from here and do the modification via two sed commands in my CI description (https://github.com/barthel/docker-asciidoctor-base/blob/8ff997140bae42b8f919099a58eeb50532a7e105/.circleci/config.yml#L51)

This could be handled in the original Docker image by CI or, even better, Asciidoctor switch over to erd-go and adapt the Dockerfile directly.

Unmodified, expect the erd change, Asciidoctor Docker for arm/v7, arm64 and amd64 are available in Docker hub: https://hub.docker.com/r/uwebarthel/asciidoctor-base/tags

I'm open to share my knowledge to provide an official multi-arch Docker Image of Asciidoctor.

[dduportal]

@dduportal I solved the issue with erd in multi platform build. Before the build starts I replace erd (https://github.com/BurntSushi/erd) by erd-go (https://github.com/kaishuu0123/erd-go/). I'm use the original Dockerfile from here and do the modification via two sed commands in my CI description (https://github.com/barthel/docker-asciidoctor-base/blob/8ff997140bae42b8f919099a58eeb50532a7e105/.circleci/config.yml#L51)

This could be handled in the original Docker image by CI or, even better, Asciidoctor switch over to erd-go and adapt the Dockerfile directly.

Unmodified, expect the erd change, Asciidoctor Docker for arm/v7, arm64 and amd64 are available in Docker hub: https://hub.docker.com/r/uwebarthel/asciidoctor-base/tags

I'm open to share my knowledge to provide an official multi-arch Docker Image of Asciidoctor.

Hello @barthel , many thanks for this work and for sharing the information!

It's clearly worthwile asking for erd-go support in https://github.com/asciidoctor/asciidoctor-diagram as ERD is a dependency of this sub-project (introduced in https://github.com/asciidoctor/asciidoctor-diagram/issues/118 for reference).

As soon as erd-go is advertised as supported by asciidoctor-diagram, then we'll be able to switch in the official image. Until then, we are, alas, stuck with the Haskell ERD. But your image is a really nice alternative to be used, and it paves the way for the upcoming officials!

[barthel]

Both erd-go (https://github.com/asciidoctor/asciidoctor-diagram/issues/402) and also mscgen-js (https://github.com/asciidoctor/asciidoctor-diagram/issues/404) are now official supported by asciidoctor-diagram.

One big step forward to multi-platform Docker image.

[barthel]

Let's go on with the multi-platform Docker Image. I'm not aware with github actions but could provide my knowledge and insights here.

[dduportal]

With GitHub actions, the runner is a VM with `dockerù already installed and running.

You might want to add https://github.com/docker/setup-qemu-action to the workflow as a preliminary step, to ensure the qemu bindings are loading inside that VM's kernel. Then, the usual Docker buildx action will take over (see https://github.com/docker/setup-buildx-action).

The alternative is to write down the steps (it is the case today but might be causing too much glue code) in a script or Makefile. It makes things reproducible to contributors machines.

[barthel]

This issue is currently only related to

1. the multi-arch build pipeline with GH actions and
2. the availability of Java 11+ for Alpine (musl) on armv7.

Does anyone have ideas or can support with one or both of these issues?

My goal is to successfully get this item finally done after almost 6 years.

[dduportal]

We can already get started with arm64 and eventually others.

To be honest, amrv7 is quite the pain to manage, but if you are willing to help for maintenance, that would be nice.

WDYT?

[gounthar]

the availability of Java 11+ for Alpine (musl) on armv7.

As far as I know, there is no armv7 (or even aarch64) Java 11+ for Alpine.

[barthel]

Sounds good @dduportal . For example, let`s make all silicon (M1...) users happy and support more official platforms.

I will try to get the armv7 part. Maybe we swallow the bitter pill and use Java 8 on armv7 as long as no other version is available. And at the time we raise the main version, we could start with a Debian based and less complicated armv7.

[spkane]

Please at least add pre-built images for arm64. There are a lot of developer and cloud systems that are using arm64 nowadays, and the image seems to build just fine.

This is an example of a GitHub workflow that can basically handle this for you.

name: Build Container Image

on:
  push:
    branches:
      - 'main'

jobs:
  buildx:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Set up QEMU
        uses: docker/setup-qemu-action@v2
      - name: Set up Docker Buildx
        id: buildx
        uses: docker/setup-buildx-action@v2
      - name: Available platforms
        run: echo ${{ steps.buildx.outputs.platforms }}
      - name: Login to DockerHub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      - name: Build and push
        id: docker_build
        uses: docker/build-push-action@v3
        with:
          context: .
          file: ./Dockerfile
          platforms: linux/amd64,linux/arm64
          push: true
          tags: asciidoctor/docker-asciidoctor:latest
      - name: Image digest
        run: echo ${{ steps.docker_build.outputs.digest }}

[dduportal]

Please at least add pre-built images for arm64. There are a lot of developer and cloud systems that are using arm64 nowadays, and the image seems to build just fine.

This is an example of a GitHub workflow that can basically handle this for you.

name: Build Container Image

on:
  push:
    branches:
      - 'main'

jobs:
  buildx:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Set up QEMU
        uses: docker/setup-qemu-action@v2
      - name: Set up Docker Buildx
        id: buildx
        uses: docker/setup-buildx-action@v2
      - name: Available platforms
        run: echo ${{ steps.buildx.outputs.platforms }}
      - name: Login to DockerHub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      - name: Build and push
        id: docker_build
        uses: docker/build-push-action@v3
        with:
          context: .
          file: ./Dockerfile
          platforms: linux/amd64,linux/arm64
          push: true
          tags: asciidoctor/docker-asciidoctor:latest
      - name: Image digest
        run: echo ${{ steps.docker_build.outputs.digest }}

Hi, would you be willing to help by contributing?

[spkane]

Hi, would you be willing to help by contributing?

Yes, I'd be happy to contribute a bit to get this working, but from a fork, I don't think I can easily test GitHub workflows that need access to secrets and such.

That being said, I am willing to pull something together to start with.

[spkane]

As an additional note for now. I had to do this inside the container to use asciidoctor-epub3:

gem uninstall nokogiri -v '1.15.5'

It looks like both nokogiri-1.13.10 and nokogiri-1.15.5-aarch64-linux were installed.

[gounthar]

#I built the aarch64 image on an x86_64 Mac tonight with docker buildx build --platform linux/arm64 -t your_image_name:tag . --load. Unfortunately, I can't build it thanks to make build because of

make build
docker buildx bake asciidoctor-minimal --load --print
[+] Building 0.0s (1/1) FINISHED                                                                                                                                                       
 => [internal] load local bake definitions                                                                                                                                        0.0s
 => => reading docker-bake.hcl 1.29kB / 1.29kB                                                                                                                                    0.0s
{
  "group": {
    "default": {
      "targets": [
        "asciidoctor-minimal"
      ]
    }
  },
  "target": {
    "asciidoctor-minimal": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "target": "main-minimal",
      "platforms": [
        "linux/amd64",
        "linux/arm64"
      ],
      "output": [
        "type=docker"
      ]
    }
  }
}
docker buildx bake asciidoctor-minimal --load
[+] Building 0.5s (1/1) FINISHED                                                                                                                       docker-container:sharp_thompson
 => [internal] load local bake definitions                                                                                                                                        0.0s
 => => reading docker-bake.hcl 1.29kB / 1.29kB                                                                                                                                    0.0s
ERROR: docker exporter does not currently support exporting manifest lists
make: *** [asciidoctor-minimal.build] Error 1

On an aarch64 machine, the docker buildx command works well, as does the make build but the make all fails.

 ✓ We can generate an HTML document with asciidoctor-mathematical as backend 
 ✗ We can generate an EPub document with asciidoctor-epub3
   (in test file tests/asciidoctor.bats, line 216)
     `[ "${status}" -eq 0 ]' failed
 ✓ We can generate an HTML document with asciimath as backend 
 ✓ We can generate a PDF document with asciidoctor-mathematical as backend 
 ✓ We can generate a Reveal.js Slide deck 
 ✓ We can generate HTML documents with different syntax-colored codes 
 ✓ We can generate PDF documents with different syntax-colored codes 
 ✓ We can convert a Markdown file to an AsciiDoc file 
 ✓ We can produce a website with citations from bibtex 

35 tests, 1 failure

make: *** [Makefile:23: asciidoctor.test] Error 1

After cherry-picking from @spkane #407 the line regarding the removal of nokogiri, I confirm the 35 test pass on a native aarch64 machine:

✓ We can generate an HTML document with asciidoctor-mathematical as backend 
 ✓ We can generate an EPub document with asciidoctor-epub3 
 ✓ We can generate an HTML document with asciimath as backend 
 ✓ We can generate a PDF document with asciidoctor-mathematical as backend 
 ✓ We can generate a Reveal.js Slide deck 
 ✓ We can generate HTML documents with different syntax-colored codes 
 ✓ We can generate PDF documents with different syntax-colored codes 
 ✓ We can convert a Markdown file to an AsciiDoc file 
 ✓ We can produce a website with citations from bibtex 

35 tests, 0 failures

Of course, not everything is multi-arch, so it fails in the end with:

mkdir -p "/home/ubuntu/docker-asciidoctor/cache"
curl -sSL -o "/home/ubuntu/docker-asciidoctor/cache/pandoc-2.10.1-linux.tar.gz" \
	https://github.com/jgm/pandoc/releases/download/2.10.1/pandoc-2.10.1-linux-amd64.tar.gz
tar xzf "/home/ubuntu/docker-asciidoctor/cache/pandoc-2.10.1-linux.tar.gz" -C "/home/ubuntu/docker-asciidoctor/cache"
docker run --rm -t -v /home/ubuntu/docker-asciidoctor:/documents --entrypoint bash asciidoctor \
	-c "asciidoctor -b docbook -a leveloffset=+1 -o - README.adoc | /documents/cache/pandoc-2.10.1/bin/pandoc  --atx-headers --wrap=preserve -t gfm -f docbook - > README.md"
bash: line 1: /documents/cache/pandoc-2.10.1/bin/pandoc: cannot execute binary file: Exec format error
Broken pipe @ io_write - <STDOUT>
  Use --trace to show backtrace
make: *** [Makefile:47: README] Error 126

I then modified the Makefile to make it multi-arch, but unfortunately, pandoc 2.10.1 does not exist for aarch64. The first pandoc version for aarch64 seems to be the 2.12. In this version, the pandoc binary is in /usr/bin and not in /bin, so I had to make another modification.

[spkane]

@dduportal See: https://github.com/asciidoctor/docker-asciidoctor/pull/407

[spkane]

I built the aarch64 image on an x86_64 Mac tonight with docker buildx build --platform linux/arm64 -t your_image_name:tag . --load. Unfortunately, I can't build it thanks to make build ...

@gounthar This PR tries to handle that issue: https://github.com/asciidoctor/docker-asciidoctor/pull/407

Using this PR, as a dev, you would run make build-local, but in CI/CD, you would use make build && make deploy in general.

It might still need some tweaks, but in general that is what is required.