setup-qemu-action icon indicating copy to clipboard operation
setup-qemu-action copied to clipboard

Available platforms has more than required `platforms`

Open anuj-scanova opened this issue 2 years ago • 7 comments

Using in the Github actions using the following template

  build:
    name: Build Image
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v1
        with:
          image: tonistiigi/binfmt:latest
          platforms: arm64,arm

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1

      - name: Inspect builder
        run: |
          echo "Name:      ${{ steps.buildx.outputs.name }}"
          echo "Endpoint:  ${{ steps.buildx.outputs.endpoint }}"
          echo "Status:    ${{ steps.buildx.outputs.status }}"
          echo "Flags:     ${{ steps.buildx.outputs.flags }}"
          echo "Platforms: ${{ steps.buildx.outputs.platforms }}"

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.PROD_AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.PROD_AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1

      - name: Get Tags for Image
        id: metadata
        uses: docker/metadata-action@v3
        with:
          images: ${{ env.ECR_REGISTRY }}/${{ env.ECR_REPOSITORY }}
          tags: |
            type=raw,value=latest
            type=raw,value=${{ github.event.release.tag_name }}
            type=sha

      - name: Docker build/tag/push
        uses: docker/build-push-action@v2
        with:
          context: .
          platforms: linux/arm64
          push: true
          tags: ${{ steps.metadata.outputs.tags }}

Where I want want to build the docker image against arm64 architecture and not others. But the execution of the above workflow runs build for all platforms, taking a long time to build (approx 30 minutes) which for a single platform completes in 4-5 minutes.

Screen Shot 2022-06-29 at 10 23 53 AM

The provided value is arm64 which the available platforms are linux/amd64,linux/arm64,linux/386,linux/arm/v7,linux/arm/v6

anuj-scanova avatar Jun 29 '22 04:06 anuj-scanova

That sounds annoying :cry: Unfortunately, slowdowns like this from using QEMU aren't unexpected, since it's essentially emulating another architecture in software. On the buildkit repo itself, QEMU jobs are slow as well - see some of the jobs here if you're interested.

What's the content of your Dockerfile? Some things in there might be slowing things down more than expected, especially CPU-heavy tasks.

jedevc avatar Jun 29 '22 07:06 jedevc

@jedevc There is no slowdown I think, there are multiple builds for each platform because I see the same scripts from Dockerfile are running multiple times in the Docker build/tag/push step. While I want to build only for arm64 architecture and not all.

anuj-scanova avatar Jun 29 '22 08:06 anuj-scanova

Ah, ok :+1:

In the screenshot you shared I think it shows QEMU being setup for all platforms, but I don't think the image itself is actually being built for all platforms. You can restrict the list of QEMU platforms to setup if you like using the options in https://github.com/docker/setup-qemu-action#inputs

jedevc avatar Jun 29 '22 08:06 jedevc

Where I want want to build the docker image against arm64 architecture and not others. But the execution of the above workflow runs build for all platforms

      - name: Docker build/tag/push
        uses: docker/build-push-action@v2
        with:
          context: .
          platforms: linux/arm64
          push: true
          tags: ${{ steps.metadata.outputs.tags }}

Would need some logs but looking at your workflow it will only build for the linux/arm64 platform.

The provided value is arm64 which the available platforms are linux/amd64,linux/arm64,linux/386,linux/arm/v7,linux/arm/v6

This is the supported platforms. See the "Installing QEMU static binaries" section in your logs. You should see qemu-aarch64 and qemu-arm in emulators and not others qemu-*.

crazy-max avatar Jun 29 '22 08:06 crazy-max

Where I want want to build the docker image against arm64 architecture and not others.

You have:

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v1
        with:
          image: tonistiigi/binfmt:latest
          platforms: arm64,arm

If you only want to build for arm64, why are you adding arm?

The screenshot then shows what QEMU can support (with the platforms you chose). ARM platforms (arm) and x86 (32-bit (386) + 64-bit (amd64)), the latter two being due to running on an x86_64 system AFAIK.

polarathene avatar Aug 20 '22 08:08 polarathene

Same issue,

astraldev avatar Apr 25 '23 00:04 astraldev

My workflow is to build linux/amd64 and linux/arm64, but docker/setup-qemu-action's platform output produces linux/386 and the following steps in the workflow fail because linux/386 is not supported but is produced.

I understand that this is intended and not a bug, but adding 1 more output (like steps.qemu.requested-platforms) whose value is the intersection of the requested and supported platforms would be a big help.

FROM ossrs/srs
    - id: qemu
      uses: docker/setup-qemu-action@v3
      with:
        platforms: linux/amd64,linux/arm64

    # Fails here because ossrs/srs does not have a linux/386 variant
    - uses: docker/build-push-action@v5
      with:
        context: .
        platforms: ${{ steps.qemu.outputs.platforms }}
        ...

allanlei avatar Nov 19 '23 09:11 allanlei