docker_images icon indicating copy to clipboard operation
docker_images copied to clipboard

Trouble mounting `binfmt_misc` for Docker builds

Open lstellway opened this issue 1 year ago • 1 comments

Hello,

I'm currently using Gitea with the Gitea act runner. I have the following labels configured to run jobs:

  labels:
    - "ubuntu-latest:docker://ghcr.io/catthehacker/ubuntu:act-latest"
    - "ubuntu-22.04:docker://ghcr.io/catthehacker/ubuntu:act-22.04"
    - "ubuntu-20.04:docker://ghcr.io/catthehacker/ubuntu:act-20.04"

When trying to use the docker/setup-qemu-action@v3 action to prepare the environment for cross-platform builds, I get the following error:

[command]/usr/bin/docker run --rm --privileged tonistiigi/binfmt:latest --install all
error: operation not permitted
cannot mount binfmt_misc filesystem at /proc/sys/fs/binfmt_misc

main.run
	/src/cmd/binfmt/main.go:183
main.main
	/src/cmd/binfmt/main.go:170
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1571

Anybody else have this issue? Any ideas how to get the mount working?

I'm thinking it maybe has something to do with the Docker daemon running outside of this container's filesystem and not having permissions to write to the mount path..?

Here is a sample of my workflow:

name: release-tag

on:
  push:
    tags: ["*"]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3

lstellway avatar Sep 28 '23 02:09 lstellway

Additional notes ...

QEMU script

I came across this script from the qemu/qemu repo (via this article) for setting up QEMU. The script has an option specific to setting up QEMU on Debian (Ubuntu is based on Debian).

When executing this script within the container environment, I need to additionally install an extra package: binfmt-support. Afterwards, I can successfully mount QEMU with the --debian option.

Sample workflow step:

      - name: Set up QEMU
        run: |
          sudo apt-get update
          sudo apt-get install -y binfmt-support
          curl -L -o /tmp/qemu.sh 'https://raw.githubusercontent.com/qemu/qemu/master/scripts/qemu-binfmt-conf.sh'
          chmod +x /tmp/qemu.sh
          /tmp/qemu.sh --debian

With this, my /usr/share/binfmts/ directory is populated with the following:

ls -l /usr/share/binfmts/
python3.10
qemu-sh4
qemu-microblaze
qemu-alpha
qemu-loongarch64
qemu-armeb
qemu-sparc32plus
qemu-microblazeel
qemu-aarch64_be
qemu-ppc
qemu-mipsn32
qemu-riscv64
qemu-mipsel
qemu-riscv32
qemu-sparc64
qemu-mipsn32el
qemu-ppc64
qemu-xtensa
qemu-aarch64
qemu-hppa
qemu-arm
qemu-mips64el
qemu-mips
qemu-ppc64le
qemu-or1k
qemu-hexagon
qemu-s390x
qemu-m68k
qemu-xtensaeb
qemu-mips64
qemu-sparc
qemu-sh4eb

But the only platforms available to Docker buildx are:

linux/amd64,linux/amd64/v2,linux/amd64/v3,linux/386

Run via Docker

As documented in multiarch/qemu-user-static, this script can also be run via Docker:

docker run --rm --privileged multiarch/qemu-user-static:register [--reset][--help][options]

My Gitea-runner host is in an Alpine linux container (gitea/act_runner:0.2.6-dind-rootless).

When I run this with no options:

docker run --rm --privileged multiarch/qemu-user-static:register

I get the error:

mount: permission denied (are you root?)

When I run it with the --debian flag:

docker run --rm --privileged multiarch/qemu-user-static:register --debian

I get the same error with a warning (source reference):

mount: permission denied (are you root?) WARNING: your system is not a Debian based distro

Related issue

Also found an issue in the action's repository that looks related: docker/setup-qemu-action#67

lstellway avatar Sep 28 '23 02:09 lstellway

You are trying to run a root action in rootless daemon. No idea how to help.

catthehacker avatar Jul 26 '24 06:07 catthehacker