BuildKit builds are not cached enough to allow offline no-op builds

[ranma42]

• [x] I have tried with the latest version of Docker Desktop
• [x] I have tried disabling enabled experimental features
• [x] I have uploaded Diagnostics
• Diagnostics ID: EF92519E-0271-4D3F-B8A4-195DBC1AF8B7/20210818084741

Actual behavior

After a BuildKit build, rebuilding (with no change) does not work if offline.

Expected behavior

After a BuildKit build, rebuilding (with no change) should work regardless of connectivity.

Information

• Windows Version: 10.0.19043 Build 19043
• Docker Desktop Version: 3.6.0 (67351)
• WSL2 or Hyper-V backend? WSL2
• Are you running inside a virtualized Windows e.g. on a cloud server or a VM: no

Steps to reproduce the behavior

From a clean env, in an empty folder:

• run echo "FROM alpine" > Dockerfile
• go online
• run DOCKER_BUILDKIT=1 docker build .
• go offline
• run DOCKER_BUILDKIT=1 docker build . (fails, should succeed)

The reproduction is described in more detail in this repo https://github.com/ranma42/docker-build-issue

Might be related to https://github.com/docker/for-win/issues/10247

[tonistiigi]

You need to define your build dependencies locally, eg. for alpine:latest your need to docker pull alpine:latest or your need to define them with immutable digests. Otherwise builder needs to check if the latest tag has been updated or not.

[ranma42]

You need to define your build dependencies locally, eg. for alpine:latest your need to docker pull alpine:latest or your need to define them with immutable digests. Otherwise builder needs to check if the latest tag has been updated or not.

Look at the detailed repro; apparently it does not need to check that if a non-BuildKit build has completed locally.

[ranma42]

It might be that the non-BuildKit build implicitly pulls the image (as in docker pull alpine:latest), while the BuildKit build intentionally does not, so that at every build it is fetched again. Usually I would expect the image to be pulled at the first build (and at every build if the --pull flag is specified).

[frichard35]

Even without talking of cache. It must be possible to build image offline with a private repo. For now, it's impossible with buildkit.

echo -e "# syntax = docker/dockerfile:1.3\nFROM my.private.repo/alpine:latest" > Dockerfile
DOCKER_BUILDKIT=1 docker build .

Results in:

[+] Building 30.6s (3/3) FINISHED                                                                                                                                              
 => [internal] load build definition from Dockerfile                                                                                                                      0.0s
 => => transferring dockerfile: 192B                                                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                                                         0.0s
 => => transferring context: 2B                                                                                                                                           0.0s
 => ERROR resolve image config for docker.io/docker/dockerfile:1.3                                                                                                       30.5s
------
 > resolve image config for docker.io/docker/dockerfile:1.3:
------
failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: failed to authorize: rpc error: code = Unknown desc = failed to fetch anonymous token: G
et https://auth.docker.io/token?scope=repository%3Adocker%2Fdockerfile%3Apull&service=registry.docker.io: dial tcp 111.111.111.111:443: i/o timeout

Here the internet resources which is mandatory for buildkit : https://registry-1.docker.io/v2/docker/dockerfile/manifests/1.3

[frichard35]

Sorry, i found a workaround for my "offline" issue. I just entered the following line in my daemon.json.

"registry-mirrors": ["https://my.private.repo"]

Tested with docker desktop.

[n-bes]

Usage an image with checksum helped me, example: centos:7@sha256:c73f515d06b0fa07bb18d8202035e739a494ce760aa73129f60f4bf2bd22b407

[Aisuko]

Using a checksum of images may help, but we want a cross-platform build engine. The checksum cannot work in the cross-platform build situation.

[Aisuko]

We can have a configuration for buildx here, named buildkitd.toml. Adding configuration for image registry,

[registry."domain of image registry"]
  # http = true
  # insecure = true

Reference document: https://github.com/moby/buildkit/blob/master/docs/buildkitd.toml.md

[papb]

Using a checksum of images may help, but we want a cross-platform build engine. The checksum cannot work in the cross-platform build situation.

I didn't understand this, can you clarify?

[Derkades]

A checksum is only valid for one platform, so using a checksum tag won't work. However, I'm not sure how many people really need multi-platform builds offline? As far as I know, loading a multi-platform image in docker is not possible (only buildx can handle it)

[ussserrr]

https://github.com/docker/buildx/issues/738#issuecomment-1061555679 Are there any solution when I don't have a private registry and just want to build an image behind a proxy without messing up with environment variables? Maybe use some cached syntax config or to "spoof" the BuildKit when it tries to download it...

[JonZeolla]

I have the same multi-platform offline build problem. Other than running a local mirror I don't see a solution :/