[v0.13] fail to export image when using rewrite-timestamp=true

[emalihin]

Hello,

I've been running BuildKit 0.13.0-beta1 for a while as a docker-container driver to get caching ECR registry integration and timestamp rewriting for reproducibility. This worked well for a couple of months.

Today I tried upgrading BuildKit to the stable 0.13.0 and 0.13.1 releases with the same docker-container setup, and started encountering these errors:

...
#18 [10/10] COPY test.conf /etc/test.conf
#18 CACHED
#19 exporting to image
#19 exporting layers done
#19 rewriting layers with source-date-epoch 1 (1970-01-01 00:00:01 +0000 UTC)
------
 > exporting to image:
------
ERROR: failed to solve: content digest sha256:7b722c1070cdf5188f1f9e43b8413157f8dfb2b4fe84db3c03cb492379a42fcc: not found

For some reason jobs that use this cached layer fail to export it. Once I delete the cache from ECR - 1st build passes, and subsequent builds fail.

Everything works again once I revert to using BuildKit 0.13.0-beta1, with new and existing caches.

This is the command I use:

docker buildx build \
--build-arg SOURCE_DATE_EPOCH=1 \
--builder buildkit-6f9724f17cbb483a5f1e \
--provenance=false \
--sbom=false \
--cache-from type=registry,ref=0000000000.dkr.ecr.us-east-2.amazonaws.com/image-name:branch-name-cache \
--cache-from type=registry,ref=0000000000.dkr.ecr.us-east-2.amazonaws.com/image-name:main-cache \
--cache-to mode=max,image-manifest=true,oci-mediatypes=true,type=registry,ref=0000000000.dkr.ecr.us-east-2.amazonaws.com/image-name:branch-name \
--output type=registry,rewrite-timestamp=true \
--tag 0000000000.dkr.ecr.us-east-2.amazonaws.com/image-name:tmp.4521fd86 .

Edit: similar to this issue, but I'm not doing COPY --link