buildx Occasionally failed commit on ref

Occasionally failed commit on ref

Open kycheng opened this issue 1 year ago • 5 comments

Contributing guidelines

[X] I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

[X] ... the documentation does not mention anything about my problem
[X] ... there are no open or closed issues that are related to my problem

Description

When using buildx to build a multi-architecture image, an error occurs: failed to solve: failed to compute cache key: failed commit on ref "layer-sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6": unexpected commit digest sha256:e85be1e84e11ec25c2492d934aff8a189811fd70377dbe19baf7b16a3aaba659, expected sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6

Expected behaviour

Stable build succeeds

Actual behaviour

docker buildx build --platform linux/amd64 linux/arm64 --labels commit_id=xxxx --tags jdb11 .

first failure:

#16 [linux/amd64 jdk11 1/1] FROM docker.io/jenkins/agent:3107.v665000b_51092-5-jdk11@sha256:c16642dc269cd208adb76c0d9efe16098d0b47677f767e2c66eeb3bd9008a918
#16 sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2 33.55MB / 37.15MB 210.2s
#16 sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2 35.65MB / 37.15MB 212.0s
#16 DONE 248.9s
#32 [linux/amd64 stage-2 5/8] COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar
#32 ERROR: failed commit on ref "layer-sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2": unexpected commit digest sha256:9bc7b10ff3793ea67fd1628262aafe7e24319ce12b6fd153809a6e66047e4d3b, expected sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2: failed precondition
#31 exporting to image
#31 exporting layers 9.7s done
#31 CANCELED
------
 > [linux/amd64 stage-2 5/8] COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar:
------
docker-agent.Dockerfile:60
--------------------
  58 |     
  59 |     
  60 | >>> COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar
  61 |     RUN chmod 0644 /usr/share/jenkins/agent.jar \
  62 |       && ln -sf /usr/share/jenkins/agent.jar /usr/share/jenkins/slave.jar
--------------------
error: failed to solve: failed to compute cache key: failed commit on ref "layer-sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2": unexpected commit digest sha256:9bc7b10ff3793ea67fd1628262aafe7e24319ce12b6fd153809a6e66047e4d3b, expected sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2: failed precondition

second failure：

#30 [linux/arm64 stage-2 4/8] RUN DEBIAN_FRONTEND=noninteractive apt-get update   && DEBIAN_FRONTEND=noninteractive apt-get --yes --no-install-recommends install     ca-certificates     curl     fontconfig     git     git-lfs     less     netbase     openssh-client     patch     tzdata   && apt-get clean   && rm -rf /tmp/* /var/cache/* /usr/share/doc/* /usr/share/man/* /var/lib/apt/lists/*
#30 CACHED
#31 [linux/arm64 stage-2 3/8] COPY --from=jdk-focal /opt/java/openjdk /opt/java/openjdk
#31 CACHED
#32 [linux/arm64 stage-2 2/8] RUN groupadd -g 1000 jenkins   && useradd -d "/home/jenkins" -u 1000 -g 1000 -l -m -s /bin/bash jenkins
#32 CACHED
#33 [linux/arm64 stage-2 5/8] COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar
#33 ERROR: failed commit on ref "layer-sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6": unexpected commit digest sha256:e85be1e84e11ec25c2492d934aff8a189811fd70377dbe19baf7b16a3aaba659, expected sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6: failed precondition
------
 > [linux/arm64 stage-2 5/8] COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar:
------
docker-agent.Dockerfile:60
--------------------
  58 |     
  59 |     
  60 | >>> COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar
  61 |     RUN chmod 0644 /usr/share/jenkins/agent.jar \
  62 |       && ln -sf /usr/share/jenkins/agent.jar /usr/share/jenkins/slave.jar
--------------------
error: failed to solve: failed to compute cache key: failed commit on ref "layer-sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6": unexpected commit digest sha256:e85be1e84e11ec25c2492d934aff8a189811fd70377dbe19baf7b16a3aaba659, expected sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6: failed precondition

The third run build succeeded, I'm not sure if this has anything to do with the architecture I specified.

Buildx version

github.com/docker/buildx v0.8.2 6224def4dd2c3d347eee19db595348c50d7cb491

Docker info

No response

Builders list

NAME/NODE       DRIVER/ENDPOINT                                                               STATUS   BUILDKIT PLATFORMS
builder *       kubernetes
  amd64         kubernetes:///builder?deployment=amd64&kubeconfig=/Users/mac/.kube/admin.conf inactive          linux/amd64*
  arm64         kubernetes:///builder?deployment=arm64&kubeconfig=/Users/mac/.kube/admin.conf inactive          linux/arm64*

Configuration

ARG VERSION
FROM jenkins/agent:3107.v665000b_51092-5-jdk11 as jdk11
FROM ubuntu:22

ENV JAVA_HOME=/opt/java/openjdk
ENV PATH "${JAVA_HOME}/bin:${PATH}"

...

RUN DEBIAN_FRONTEND=noninteractive apt-get update \
  && DEBIAN_FRONTEND=noninteractive apt-get --yes --no-install-recommends install \
    ca-certificates \
    curl \
    fontconfig \
    git \
    git-lfs \
    less \
    netbase \
    openssh-client \
    patch \
    tzdata \
  && apt-get clean \
  && rm -rf /tmp/* /var/cache/* /usr/share/doc/* /usr/share/man/* /var/lib/apt/lists/*

COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar
RUN chmod 0644 /usr/share/jenkins/agent.jar \
  && ln -sf /usr/share/jenkins/agent.jar /usr/share/jenkins/slave.jar
...

Build logs

No response

Additional info

According to the description in this #235, I understand that multi-arch builds are executed in parallel.

I repeated the operation for the third time and succeeded. Checking the logs of the first two times shows that the expected digest is a certain layer in the image, and the results of the two error reports are not of the same architecture.

first expected: sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6 second expected: sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6

➜ docker buildx imagetools inspect docker.io/jenkins/agent:3107.v665000b_51092-5-jdk11 | grep -B2 "linux/a"
  Name:        docker.io/jenkins/agent:3107.v665000b_51092-5-jdk11@sha256:a5948dcbf6af739b89c551b5eb96709e6f6d13aabd992a4cecf0c8b20ae13a1b
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    linux/amd64
--
  Name:        docker.io/jenkins/agent:3107.v665000b_51092-5-jdk11@sha256:9ba35e957e4a2a39b1e9c7d2370eeb5c127269e71d78768ee3cb858f0bad2325
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    linux/arm64
➜ oras manifest fetch docker.io/jenkins/agent:3107.v665000b_51092-5-jdk11@sha256:9ba35e957e4a2a39b1e9c7d2370eeb5c127269e71d78768ee3cb858f0bad2325
{
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "digest": "sha256:797ccab10de1965272376dcfd38fcb4fd45718a0964bb30095e5a636ecb133db",
    "size": 7221
  },
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:0f5fe16b1836feccd4765ac5685fc7a7b9c73445cac94fc595d2ffbc3cb59a7a",
      "size": 53703215
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:40e542b51b4fa61d5fe9cf01aeca42c1cba52fe75e6c34ff9a318cd9b7004166",
      "size": 3345
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6",
      "size": 36819598
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:aad4631e2640a7357148a952d7aa21fe29bc0f9ab0a3bec3ee774ac6f7234bbe",
      "size": 1211879
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:94cd29ebaad3c8c6c159f1014d697704cfb18a1eebdcbfbf5413de99c5f459c6",
      "size": 1211972
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:fc7932cf2fd9b57929c0452a37633740986a954029ce13e535bf88613918c964",
      "size": 63444276
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:3899b98824724066ba5b1c916e0571acf76f3f427bdd0a484baed39569bac53b",
      "size": 169
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1",
      "size": 32
    }
  ]
}%
➜ oras manifest fetch docker.io/jenkins/agent:3107.v665000b_51092-5-jdk11@sha256:a5948dcbf6af739b89c551b5eb96709e6f6d13aabd992a4cecf0c8b20ae13a1b
{
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "digest": "sha256:303477e0779f3b5451f41fc29bfc370be6d2f854439c33c5b7279016c90e893f",
    "size": 7221
  },
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:32fb02163b6bb519a30f909008e852354dae10bdfd6b34190dbdfe8f15403ea0",
      "size": 55045922
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:18f33ebd1600b157b7fad942042fd1de0291734156602e239f71dcdb394a6059",
      "size": 3339
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2",
      "size": 37150529
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:c8e02e85f6f6109e22689e05f36dc9955b2e16fbb7e17b35c78e3d23e7e41be2",
      "size": 1211874
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:6cb88d50449b2f3a875c53c37fd9d28d030cc949a77277cf8bb29b76e51b9a92",
      "size": 1211970
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:52a4b0758312e424d845a32169faa3f54412a011e281c66e8f5ada72e357bd90",
      "size": 65031229
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:fdafead6b594fcb15d4d5cb0e0b400b8573c4f60d2ca21b8353e716317835d18",
      "size": 174
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1",
      "size": 32
    }
  ]
}%

May 06 '23 06:05 kycheng

buildx buildx copied to clipboard

Occasionally failed commit on ref

Contributing guidelines

I've found a bug and checked that ...

Description

Expected behaviour

Actual behaviour

Buildx version

Docker info

Builders list

Configuration

Build logs

Additional info

buildx
buildx copied to clipboard