build-push-action
build-push-action copied to clipboard
docker
Build should not fail on corrupted cache
Expected behaviour
Building with build-push-action and buildx works as expected.
Actual behaviour
Yesterday we started seeing our docker build push action fail on every commit. This is not correlated to any specific code change that we did.
We temporary fixed this by removing the buildx from our workflow.
Configuration
- Repository URL (if public): N/A
- Build URL (if public): N/A
name: Build & Push Docker Next Image
on: push
jobs:
docker:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Cache Docker layers
uses: actions/cache@v3
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-next-prod-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-next-prod
- name: Login to GitHub Container Registry
uses: docker/login-action@v1
with:
registry: ghcr.io
username: ${{ secrets.GH_BOT_USER }}
password: ${{ secrets.GH_PAT_TOKEN }}
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
- name: Build and push
id: docker_build
uses: docker/build-push-action@v2
with:
context: .
push: true
tags: |
ghcr.io/companyx/next:${{ env.GITHUB_REF_SLUG }}
ghcr.io/companyx/next:${{ github.sha }}
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new
- name: Image digest
run: echo ${{ steps.docker_build.outputs.digest }}
# Temporary fix
# https://github.com/docker/build-push-action/issues/252
# https://github.com/moby/buildkit/issues/1896
- name: Move cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
- name: Trigger release workflow
if: github.ref == 'refs/heads/master'
uses: benc-uk/workflow-dispatch@v1
with:
workflow: release
inputs: '{ "commit_sha": "${{ github.sha }}" }'
token: ${{ secrets.GH_PAT_TOKEN }}
Logs
#9 [builder 1/9] COPY package.json yarn.lock ./
#9 ERROR: failed commit on ref "layer-sha256:0b0af05a4d868593f859eaa5815fc1c3596d77318a4ed756f3865a5fa3f290c6": unexpected commit digest sha256:7097d9f739d4ed17f8875508553264eb5e496d5e07edd35330157cae6d219a2c, expected sha256:0b0af05a4d868593f859eaa5815fc1c3596d77318a4ed756f3865a5fa3f290c6: failed precondition
------
> [builder 1/9] COPY package.json yarn.lock ./:
------
Dockerfile:9
--------------------
7 | FROM base AS builder
8 |
9 | >>> COPY package.json yarn.lock ./
10 |
11 | # Install production dependencies
--------------------
error: failed to solve: failed to compute cache key: failed commit on ref "layer-sha256:0b0af05a4d868593f859eaa5815fc1c3596d77318a4ed756f3865a5fa3f290c6": unexpected commit digest sha256:7097d9f739d4ed17f8875508553264eb5e496d5e07edd35330157cae6d219a2c, expected sha256:0b0af05a4d868593f859eaa5815fc1c3596d77318a4ed756f3865a5fa3f290c6: failed precondition
Error: buildx failed with: error: failed to solve: failed to compute cache key: failed commit on ref "layer-sha256:0b0af05a4d868593f859eaa5815fc1c3596d77318a4ed756f3865a5fa3f290c6": unexpected commit digest sha256:7097d9f739d4ed17f8875508553264eb5e496d5e07edd35330157cae6d219a2c, expected sha256:0b0af05a4d868593f859eaa5815fc1c3596d77318a4ed756f3865a5fa3f290c6: failed precondition
@Meemaw Can you enable debug in the Set up Docker Buildx step:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
with:
buildkitd-flags: --debug
And post BuildKit container logs available in the Post Set Up Docker Buildx please?
If you have a link to your repo or a simple repro so we can check on our side, that would be great, thanks!
Done. Here are the logs:
/usr/bin/docker logs buildx_buildkit_builder-051c83e6-10bc-4d70-97f2-04c81188f3ae0
time="2022-03-24T08:20:55Z" level=info msg="auto snapshotter: using overlayfs"
time="2022-03-24T08:20:55Z" level=warning msg="using host network as the default"
time="2022-03-24T08:20:55Z" level=info msg="found worker \"topm57vu4yerinbavmjfyb41b\", labels=map[org.mobyproject.buildkit.worker.executor:oci org.mobyproject.buildkit.worker.hostname:221586c425c5 org.mobyproject.buildkit.worker.network:host org.mobyproject.buildkit.worker.oci.process-mode:sandbox org.mobyproject.buildkit.worker.snapshotter:overlayfs], platforms=[linux/amd64 linux/amd64/v2 linux/amd64/v3 linux/amd64/v4 linux/arm64 linux/riscv64 linux/ppc64le linux/s390x linux/386 linux/mips64le linux/mips64 linux/arm/v7 linux/arm/v6]"
time="2022-03-24T08:20:55Z" level=warning msg="skipping containerd worker, as \"/run/containerd/containerd.sock\" does not exist"
time="2022-03-24T08:20:55Z" level=info msg="found 1 workers, default=\"topm57vu4yerinbavmjfyb41b\""
time="2022-03-24T08:20:55Z" level=warning msg="currently, only the default worker can be used."
time="2022-03-24T08:20:55Z" level=info msg="running server on /run/buildkit/buildkitd.sock"
time="2022-03-24T08:21:01Z" level=debug msg="session started"
time="2022-03-24T08:21:01Z" level=debug msg="session finished: <nil>"
time="2022-03-24T08:21:01Z" level=debug msg="session started"
time="2022-03-24T08:21:01Z" level=debug msg="skipping invalid cache export mode: "
time="2022-03-24T08:21:01Z" level=debug msg="new ref for local: n4zjd8rul06ng8v7zydkpx87x" span="[internal] load build definition from Dockerfile"
time="2022-03-24T08:21:01Z" level=debug msg="diffcopy took: 30.521205ms" span="[internal] load build definition from Dockerfile"
time="2022-03-24T08:21:01Z" level=debug msg="saved n4zjd8rul06ng8v7zydkpx87x as dockerfile:dockerfile:companyx-next:35b65b22e5785078" span="[internal] load build definition from Dockerfile"
time="2022-03-24T08:21:01Z" level=debug msg="new ref for local: xyjw3ta0nznzfxcfnmcjhwrbh" span="[internal] load .dockerignore"
time="2022-03-24T08:21:01Z" level=debug msg="diffcopy took: 17.165891ms" span="[internal] load .dockerignore"
time="2022-03-24T08:21:01Z" level=debug msg="saved xyjw3ta0nznzfxcfnmcjhwrbh as context:context-.dockerignore:companyx-next:35b65b22e5785078" span="[internal] load .dockerignore"
time="2022-03-24T08:21:02Z" level=debug msg=resolving host=registry-1.docker.io
time="2022-03-24T08:21:02Z" level=debug msg="do request" host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/0.0.0+unknown request.method=HEAD url="https://registry-1.docker.io/v2/library/node/manifests/16.11.0-buster"
time="2022-03-24T08:21:02Z" level=debug msg="fetch response received" host=registry-1.docker.io response.header.content-length=155 response.header.content-type=application/json response.header.date="Thu, 24 Mar 2022 08:21:02 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=52.191.252.221 response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="***"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/node:pull\"" response.status="401 Unauthorized" url="https://registry-1.docker.io/v2/library/node/manifests/16.11.0-buster"
time="2022-03-24T08:21:02Z" level=debug msg=Unauthorized header="***"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/node:pull\"" host=registry-1.docker.io
time="2022-03-24T08:21:02Z" level=debug msg="do request" host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/0.0.0+unknown request.method=HEAD url="https://registry-1.docker.io/v2/library/node/manifests/16.11.0-buster"
time="2022-03-24T08:21:02Z" level=debug msg="fetch response received" host=registry-1.docker.io response.header.content-length=1206 response.header.content-type=application/vnd.docker.distribution.manifest.list.v2+json response.header.date="Thu, 24 Mar 2022 08:21:02 GMT" response.header.docker-content-digest="sha256:aa1e997e30f6616f0517f6968982e3cd73bff84529d852cbdf0521592bc582aa" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=52.191.252.221 response.header.etag="\"sha256:aa1e997e30f6616f0517f6968982e3cd73bff84529d852cbdf0521592bc582aa\"" response.header.strict-transport-security="max-age=31536000" response.status="200 OK" url="https://registry-1.docker.io/v2/library/node/manifests/16.11.0-buster"
time="2022-03-24T08:21:02Z" level=debug msg=resolved desc.digest="sha256:aa1e997e30f6616f0517f6968982e3cd73bff84529d852cbdf0521592bc582aa" host=registry-1.docker.io
time="2022-03-24T08:21:02Z" level=debug msg=fetch digest="sha256:aa1e997e30f6616f0517f6968982e3cd73bff84529d852cbdf0521592bc582aa" mediatype=application/vnd.docker.distribution.manifest.list.v2+json size=1206
time="2022-03-24T08:21:02Z" level=debug msg="do request" digest="sha256:aa1e997e30f6616f0517f6968982e3cd73bff84529d852cbdf0521592bc582aa" mediatype=application/vnd.docker.distribution.manifest.list.v2+json request.header.accept="application/vnd.docker.distribution.manifest.list.v2+json, */*" request.header.user-agent=buildkit/0.0.0+unknown request.method=GET size=1206 url="https://registry-1.docker.io/v2/library/node/manifests/sha256:aa1e997e30f6616f0517f6968982e3cd73bff84529d852cbdf0521592bc582aa"
time="2022-03-24T08:21:02Z" level=debug msg="fetch response received" digest="sha256:aa1e997e30f6616f0517f6968982e3cd73bff84529d852cbdf0521592bc582aa" mediatype=application/vnd.docker.distribution.manifest.list.v2+json response.header.content-length=1206 response.header.content-type=application/vnd.docker.distribution.manifest.list.v2+json response.header.date="Thu, 24 Mar 2022 08:21:02 GMT" response.header.docker-content-digest="sha256:aa1e997e30f6616f0517f6968982e3cd73bff84529d852cbdf0521592bc582aa" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=52.191.252.221 response.header.etag="\"sha256:aa1e997e30f6616f0517f6968982e3cd73bff84529d852cbdf0521592bc582aa\"" response.header.strict-transport-security="max-age=31536000" response.status="200 OK" size=1206 url="https://registry-1.docker.io/v2/library/node/manifests/sha256:aa1e997e30f6616f0517f6968982e3cd73bff84529d852cbdf0521592bc582aa"
time="2022-03-24T08:21:02Z" level=debug msg=fetch digest="sha256:ae3a1ce333f322728ef3ea12186598067c31248a14a33aaa148bac7c6efc3a14" mediatype=application/vnd.docker.distribution.manifest.v2+json size=2214
time="2022-03-24T08:21:02Z" level=debug msg="do request" digest="sha256:ae3a1ce333f322728ef3ea12186598067c31248a14a33aaa148bac7c6efc3a14" mediatype=application/vnd.docker.distribution.manifest.v2+json request.header.accept="application/vnd.docker.distribution.manifest.v2+json, */*" request.header.user-agent=buildkit/0.0.0+unknown request.method=GET size=2214 url="https://registry-1.docker.io/v2/library/node/manifests/sha256:ae3a1ce333f322728ef3ea12186598067c31248a14a33aaa148bac7c6efc3a14"
time="2022-03-24T08:21:02Z" level=debug msg="fetch response received" digest="sha256:ae3a1ce333f322728ef3ea12186598067c31248a14a33aaa148bac7c6efc3a14" mediatype=application/vnd.docker.distribution.manifest.v2+json response.header.content-length=2214 response.header.content-type=application/vnd.docker.distribution.manifest.v2+json response.header.date="Thu, 24 Mar 2022 08:21:02 GMT" response.header.docker-content-digest="sha256:ae3a1ce333f322728ef3ea12186598067c31248a14a33aaa148bac7c6efc3a14" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=52.191.252.221 response.header.etag="\"sha256:ae3a1ce333f322728ef3ea12186598067c31248a14a33aaa148bac7c6efc3a14\"" response.header.strict-transport-security="max-age=31536000" response.status="200 OK" size=2214 url="https://registry-1.docker.io/v2/library/node/manifests/sha256:ae3a1ce333f322728ef3ea12186598067c31248a14a33aaa148bac7c6efc3a14"
time="2022-03-24T08:21:02Z" level=debug msg=fetch digest="sha256:a22d52bf85f0130cb32ba91725b03249e94c8ad99c716a5d04858f7b001b2fd1" mediatype=application/vnd.docker.container.image.v1+json size=7605
time="2022-03-24T08:21:02Z" level=debug msg="do request" digest="sha256:a22d52bf85f0130cb32ba91725b03249e94c8ad99c716a5d04858f7b001b2fd1" mediatype=application/vnd.docker.container.image.v1+json request.header.accept="application/vnd.docker.container.image.v1+json, */*" request.header.user-agent=buildkit/0.0.0+unknown request.method=GET size=7605 url="https://registry-1.docker.io/v2/library/node/blobs/sha256:a22d52bf85f0130cb32ba91725b03249e94c8ad99c716a5d04858f7b001b2fd1"
time="2022-03-24T08:21:02Z" level=debug msg="fetch response received" digest="sha256:a22d52bf85f0130cb32ba91725b03249e94c8ad99c716a5d04858f7b001b2fd1" mediatype=application/vnd.docker.container.image.v1+json response.header.accept-ranges=bytes response.header.age=1457457 response.header.cache-control="public, max-age=14400" response.header.cf-cache-status=HIT response.header.cf-ray=6f0dfdb1faab81af-IAD response.header.connection=keep-alive response.header.content-length=7605 response.header.content-type=application/octet-stream response.header.date="Thu, 24 Mar 2022 08:21:02 GMT" response.header.etag="\"c74f91a101755bea448a3bf674988dbf\"" response.header.expect-ct="max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"" response.header.expires="Thu, 24 Mar 2022 12:21:02 GMT" response.header.last-modified="Mon, 11 Oct 2021 18:34:54 GMT" response.header.server=cloudflare response.header.vary=Accept-Encoding response.header.x-amz-id-2="ftbjuVU852ISTQL1hGPC+zKUD4UFg5f1Nf81we5P6M8BxS0CVzOZls+AhhPFSxLmTCz6l1LCDRQ=" response.header.x-amz-request-id=MBYQ8ZG4F076N99N response.header.x-amz-version-id=e7LaWluPegkjLrUVsB3gaMRicH.sD28m response.status="200 OK" size=7605 url="https://registry-1.docker.io/v2/library/node/blobs/sha256:a22d52bf85f0130cb32ba91725b03249e94c8ad99c716a5d04858f7b001b2fd1"
time="2022-03-24T08:21:02Z" level=debug msg=fetch digest="sha256:aa1e997e30f6616f0517f6968982e3cd73bff84529d852cbdf0521592bc582aa" mediatype=application/vnd.docker.distribution.manifest.list.v2+json size=1206
time="2022-03-24T08:21:02Z" level=debug msg=fetch digest="sha256:ae3a1ce333f322728ef3ea12186598067c31248a14a33aaa148bac7c6efc3a14" mediatype=application/vnd.docker.distribution.manifest.v2+json size=2214
time="2022-03-24T08:21:02Z" level=debug msg=fetch digest="sha256:a22d52bf85f0130cb32ba91725b03249e94c8ad99c716a5d04858f7b001b2fd1" mediatype=application/vnd.docker.container.image.v1+json size=7605
time="2022-03-24T08:21:02Z" level=debug msg="new ref for local: mnco4lvim8gcx8k3nrzgbigzc" span="[internal] load build context"
time="2022-03-24T08:21:03Z" level=debug msg="diffcopy took: 1.055925172s" span="[internal] load build context"
time="2022-03-24T08:21:03Z" level=debug msg="saved mnco4lvim8gcx8k3nrzgbigzc as context:context:companyx-next:35b65b22e5785078" span="[internal] load build context"
time="2022-03-24T08:21:03Z" level=debug msg="load cache for [base 2/2] WORKDIR /usr/app with sha256:7955be70ba2c6cc246ea7ef9fabe8a99fb4e3e218b2a8b257c5fc343846eba3a"
time="2022-03-24T08:21:03Z" level=debug msg=fetch digest="sha256:81b3aa9536df1989c63621d43c6881bbb4f6bd83eee22a56441683ed795ba82b" mediatype=application/vnd.oci.image.layer.v1.tar+gzip size=116
time="2022-03-24T08:21:03Z" level=debug msg=fetch digest="sha256:2d5feb05857fef73be65a2c0228bbf66fdc86b6ec8733245398960da4b0ad39e" mediatype=application/vnd.oci.image.layer.v1.tar+gzip size=283
time="2022-03-24T08:21:03Z" level=debug msg=fetch digest="sha256:4b19957ec80a370ab74f7397cc90d8952a150257942ecbecd57c29c09b6739f0" mediatype=application/vnd.oci.image.layer.v1.tar+gzip size=2268011
time="2022-03-24T08:21:03Z" level=debug msg=fetch digest="sha256:ca7417c24e969235aaade789b8c85e208e0a7baa5752395d0c987781bd9e1bf5" mediatype=application/vnd.oci.image.layer.v1.tar+gzip size=33350248
time="2022-03-24T08:21:03Z" level=debug msg=fetch digest="sha256:8c33de21d690878efd5d2c7872e0f6afd1316b6173a288bb2e25b6c962c36cfc" mediatype=application/vnd.oci.image.layer.v1.tar+gzip size=4197
time="2022-03-24T08:21:03Z" level=debug msg=fetch digest="sha256:ca4689f0588c1ba01108f5c6d33943d3d542ae339fd526390d07c7a00ded6473" mediatype=application/vnd.oci.image.layer.v1.tar+gzip size=192395236
time="2022-03-24T08:21:03Z" level=debug msg=fetch digest="sha256:0b0af05a4d868593f859eaa5815fc1c3596d77318a4ed756f3865a5fa3f290c6" mediatype=application/vnd.oci.image.layer.v1.tar+gzip size=51841311
time="2022-03-24T08:21:03Z" level=debug msg=fetch digest="sha256:1ae016bc26876abbd5e952133b02b04d4c1dae1bc75a3d9386250e4797ccd87a" mediatype=application/vnd.oci.image.layer.v1.tar+gzip size=9997190
time="2022-03-24T08:21:03Z" level=debug msg=fetch digest="sha256:5e7b6b7bd506c12399d65977c0ba8dd02824dc5d0e65fc55d7382da889bdac7d" mediatype=application/vnd.oci.image.layer.v1.tar+gzip size=50436209
time="2022-03-24T08:21:03Z" level=debug msg=fetch digest="sha256:fd67d668d6911bf21ad4701522e1ed3af416837433fdba3f88cff06a23e23861" mediatype=application/vnd.oci.image.layer.v1.tar+gzip size=7833602
time="2022-03-24T08:21:08Z" level=error msg="/moby.buildkit.v1.Control/Solve returned error: rpc error: code = Unknown desc = failed to compute cache key: failed commit on ref \"layer-sha256:0b0af05a4d868593f859eaa5815fc1c3596d77318a4ed756f3865a5fa3f290c6\": unexpected commit digest sha256:7097d9f739d4ed17f8875508553264eb5e496d5e07edd35330157cae6d219a2c, expected sha256:0b0af05a4d868593f859eaa5815fc1c3596d77318a4ed756f3865a5fa3f290c6: failed precondition\n"
time="2022-03-24T08:21:08Z" level=debug msg="session finished: <nil>"
Unfortunately I can't link the repo, as its private.
Thanks. Can you give the output of Docker info and BuildKit version in the Set up Docker Buildx step please?
Docker info
/usr/bin/docker version
Client:
Version: 20.10.11+azure-3
API version: 1.41
Go version: go1.16.12
Git commit: dea9396e1[8](https://github.com/companyx/companyx-next/runs/5672982129?check_suite_focus=true#step:4:8)42[9](https://github.com/companyx/companyx-next/runs/5672982129?check_suite_focus=true#step:4:9)0f638ea873c76db7c80efd5a1d2
Built: Wed Nov 17 23:49:46 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server:
Engine:
Version: 20.[10](https://github.com/companyx/companyx-next/runs/5672982129?check_suite_focus=true#step:4:10).[11](https://github.com/companyx/companyx-next/runs/5672982129?check_suite_focus=true#step:4:11)+azure-3
API version: 1.41 (minimum version 1.[12](https://github.com/companyx/companyx-next/runs/5672982129?check_suite_focus=true#step:4:12))
Go version: go1.16.12
Git commit: 847da184ad5048b27f5bdf9d53d070f731b43180
Built: Thu Nov 18 00:21:59 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.12+azure-2
GitCommit: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
runc:
Version: 1.0.3
GitCommit: f46b6ba2c93[14](https://github.com/companyx/companyx-next/runs/5672982129?check_suite_focus=true#step:4:14)cfc8caae24a32ec5fe9ef1059fe
docker-init:
Version: 0.19.0
GitCommit:
/usr/bin/docker info
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc., 0.7.1+azure-2)
compose: Docker Compose (Docker Inc., 2.2.3+azure-1)
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 20
Server Version: 20.10.11+azure-3
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: false
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2
Default Runtime: runc
Init Binary: docker-init
containerd version: 7b11cfaabd73bb80907dd23[18](https://github.com/companyx/companyx-next/runs/5672982129?check_suite_focus=true#step:4:18)2b9347b4245eb5d
runc version: f46b6ba2c9314cfc8caae24a32ec5fe9ef1059fe
init version:
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 5.11.0-1028-azure
Operating System: Ubuntu [20](https://github.com/companyx/companyx-next/runs/5672982129?check_suite_focus=true#step:4:20).04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 6.785GiB
Name: fv-az1[22](https://github.com/companyx/companyx-next/runs/5672982129?check_suite_focus=true#step:4:22)-94
ID: DFUV:3MIO:QJB4:SFAI:CWPP:LEBI:ETH4:AT3U:T5PZ:KRMI:AS3I:HDYW
Docker Root Dir: /var/lib/docker
Debug Mode: false
Username: githubactions
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
1[27](https://github.com/companyx/companyx-next/runs/5672982129?check_suite_focus=true#step:4:27).0.0.0/8
Live Restore Enabled: false
and
BuildKit version
moby/buildkit:buildx-stable-1 => buildkitd github.com/moby/buildkit v0.10.0 068cf686a7e5c3254244d0acc54636f4f393611e
@ktock @sipsma Any way you see the blob management or lazy loading changes could cause something like this? Looks like error on blob commit. Possible cases could be that the verification digest is for wrong blob or there are parallel writes happening to the same location, messing up the digest. Assuming whatever base is in that Dockerfile is not an invalid image or registry did not provide invalid layers.
@Meemaw If you think this could be v0.10.0 regression you can set a different version on setup-buildx-action. If you can reproduce this then please provide us a way we could do it as well.
@ktock @sipsma Any way you see the blob management or lazy loading changes could cause something like this? Looks like error on blob commit. Possible cases could be that the verification digest is for wrong blob or there are parallel writes happening to the same location, messing up the digest. Assuming whatever
baseis in that Dockerfile is not an invalid image or registry did not provide invalid layers.@Meemaw If you think this could be v0.10.0 regression you can set a different version on
setup-buildx-action. If you can reproduce this then please provide us a way we could do it as well.
Can you give me an example on how to change the buildkit version in setup-buildx-action? I can check if its related to v0.10.0.
Also seeing this error in the Cache Docker layers step in case relevant:
Run actions/cache@v3
Received 1845493[7](https://github.com/ProjectOpenSea/companyx-next/runs/5691537979?check_suite_focus=true#step:5:7)6 of 570426441 (32.4%), 175.5 MBs/sec
Received 461373440 of 570426441 ([8](https://github.com/ProjectOpenSea/companyx-next/runs/5691537979?check_suite_focus=true#step:5:8)0.[9](https://github.com/ProjectOpenSea/companyx-next/runs/5691537979?check_suite_focus=true#step:5:9)%), 219.6 MBs/sec
Received 570426441 of 570426441 ([10](https://github.com/ProjectOpenSea/companyx-next/runs/5691537979?check_suite_focus=true#step:5:10)0.0%), 193.4 MBs/sec
Cache Size: ~544 MB (570426441 B)
/usr/bin/tar --use-compress-program zstd -d -xf /home/runner/work/_temp/b0ef45b7-d47c-4858-8359-cd9ba6c787aa/cache.tzst -P -C /home/runner/work/companyx-next/companyx-next
/*stdin*\ : Decoding error (36) : Restored data doesn't match checksum
/usr/bin/tar: Child returned status 1
/usr/bin/tar: Error is not recoverable: exiting now
Warning: Tar failed with error: The process '/usr/bin/tar' failed with exit code 2
Also seeing this error in the Cache Docker layers step in case relevant:
Interesting so cache seems corrupted in GitHub.
Can you give me an example on how to change the buildkit version in
setup-buildx-action? I can check if its related to v0.10.0.
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
with:
version: v0.9.3
Also seeing this error in the Cache Docker layers step in case relevant:
Interesting so cache seems corrupted in GitHub.
Can you give me an example on how to change the buildkit version in
setup-buildx-action? I can check if its related to v0.10.0.- name: Set up Docker Buildx uses: docker/setup-buildx-action@v1 with: version: v0.9.3
Download and install buildx Error: Cannot find buildx v0.9.3 release
@Meemaw
Error: Cannot find buildx v0.9.3 release
My bad:
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
with:
driver-opts: |
image=moby/buildkit:v0.9.3
@Meemaw With the exact same step in https://github.com/docker/build-push-action/issues/577#issuecomment-1079030579?
@Meemaw With the exact same step in #577 (comment)?
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
with:
driver-opts: |
image=moby/buildkit:v0.9.3
buildkitd-flags: --debug
@Meemaw Works fine for me with this step: https://github.com/crazy-max/buildx-buildkit-tests/blob/43e41154a597cdf1548852266db3daa179901510/.github/workflows/build-push-action-577.yml#L16-L22
@crazy-max the setup-buildx step works, but the build/push step fails with the original error.
@Meemaw
Fails on that version as well -- same error.
Ok, I thought the setup-buildx step had failed.
So looking at https://github.com/docker/build-push-action/issues/577#issuecomment-1078973976, your GitHub cache looks busted. Looks like something wrong occurs with actions/cache. To be sure can you change your cache key with smth like:
- name: Cache Docker layers
uses: actions/cache@v3
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-next-prod-v2-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-next-prod-v2
@crazy-max changing the cache key fixed the issue. Wondering why that would break the next steps though? I would expect cache action to ignore the cache if corrupted.
Yes I think we should just skip and not fail in this case. Will try to repro on my side.
https://github.com/moby/buildkit/pull/3430 introduced an ignore-error parameter, which seems to work (https://github.com/moby/buildkit/issues/2836#issuecomment-1543694979):
cache-from: type=gha,ignore-error=true
cache-to: type=gha,mode=max,ignore-error=true
@thewilkybarkid Yes indeed, we have added docs for this too: https://docs.docker.com/build/cache/backends/gha/#synopsis
Also ignore-error opt is just handled for cache-to.
Closing this one as this is tracked upstream: https://github.com/moby/buildkit/issues/2836