No error when adding index annotations using docker exporter and docker driver

[dvdksn]

Contributing guidelines

• [X] I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• [X] ... the documentation does not mention anything about my problem
• [X] ... there are no open or closed issues that are related to my problem

Description

Building an image with the docker builder, while also trying to add annotations to index, doesn't result in an error.

Expected behaviour

When trying to build an image with annotations on index, using a docker-container builder, my build fails as expected if I try to use the docker exporter.

$ docker buildx b --builder container --annotation "index:foo=bar" --output type=docker .
...
 => ERROR exporting to docker image format                                                     0.0s
 => => exporting layers                                                                        0.0s
------
 > exporting to docker image format:
------
ERROR: failed to build: failed to solve: index annotations not supported for single platform export

Actual behaviour

$ docker buildx b --builder desktop-linux --annotation "index:foo=bar" .
[+] Building 1.2s (9/9) FINISHED                                               docker:desktop-linux
...
 => exporting to image                                                                         0.0s
 => => exporting layers                                                                        0.0s
 => => writing image sha256:a10a6248e2608c811b279aab218200dfe92795a7724c40616574edb30df6c1fc   0.0s

Buildx version

github.com/docker/buildx 3005743 3005743f7c52e65944ef201110571e8c8bca3541

Docker info

No response

Builders list

NAME/NODE              DRIVER/ENDPOINT     STATUS    BUILDKIT   PLATFORMS
container*             docker-container                         
 \_ container0          \_ desktop-linux   running   v0.14.1    linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
default                docker                                   
 \_ default             \_ default         running   v0.14.1    linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
desktop-linux          docker                                   
 \_ desktop-linux       \_ desktop-linux   running   v0.14.1    linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
cloud-docker-default   cloud               error

Configuration

$ docker buildx b --builder desktop-linux --output type=docker --annotation "index:foo=bar" -<<< "FROM alpine"

Build logs

No response

Additional info

No response

[tonistiigi]

Was this with containerd storage backend?

[dvdksn]

Both had the same issue in my tests

[crazy-max]

I was looking at this issue on moby and seems annotations are not passed to moby exporter on Export in inp.Metadata: https://github.com/moby/moby/blob/d2ad7c36dc268e0d9751b803cec3003234740636/builder/builder-next/exporter/mobyexporter/export.go#L107 after debugging.

Annotations on build are set as exporter attributes in: https://github.com/docker/buildx/blob/0d708c0bc2422efb7bad7026c7dbfcb84ded51fe/controller/build/build.go#L142-L146

But exporter is not set in this repro so annotations will not be set in this case.

However if exporter is specified, annotations are set in exporter attributes:

docker buildx b --builder desktop-linux --annotation "index:foo=bar" --output type=docker .

So I think it needs both a fix on buildx and moby.

[tonistiigi]

Moby exporter that creates images in non-containerd Docker imagestore can not support annotations because BuildKit has no control of the manifests there. Manifests are only created by docker push. Same for all the oci-mediatypes, compression etc. options and attestations.

[crazy-max]

Opened https://github.com/moby/moby/pull/50295 to fix this on moby side first.

[crazy-max]

Opened moby/moby#50295 to fix this on moby side first.

This would happen during export at the end of the build so it might be better to fail early: https://github.com/docker/buildx/pull/3384

[dvdksn]

Makes sense! Thanks!