buildx icon indicating copy to clipboard operation
buildx copied to clipboard
verified publisher icon docker

FROM line output inconsistent and confusing when using or disabling the build cache

Open spkane opened this issue 1 year ago • 4 comments

Description

FROM line output inconsistent and confusing when using or disabling the build cache.

$ docker build .
...
=> [ 1/13] FROM docker.io/library/alpine:3.20@sha256:1e42bbe2508154c9126d48c2b8a75420c3544343bf86fd041fb7527e017a4  0.0s
...

$ docker build --no-cache .
...
=> CACHED [ 1/13] FROM docker.io/library/alpine:3.20@sha256:1e42bbe2508154c9126d48c2b8a75420c3544343bf86fd041fb752  0.0s
...

Reproduce

  1. git clone https://github.com/spkane/balance_game.git (or very likely anything else)
  2. cd balance_game
  3. docker build .
  4. docker build --no-cache .

Expected behavior

The output for the from line should not change between evocations, or if there is a good reason for it, the output should make that reason much clearer. At the moment, it is very odd, that CACHED shows up when running with --no-cache, but does not show up normally, even when the rest of the build is indeed CACHED.

docker version

Client:
 Version:           26.1.3
 API version:       1.45
 Go version:        go1.21.10
 Git commit:        b72abbb
 Built:             Thu May 16 08:30:38 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.35.1 (173168)
 Engine:
  Version:          27.3.1
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.7
  Git commit:       41ca978
  Built:            Fri Sep 20 11:41:19 2024
  OS/Arch:          linux/arm64
  Experimental:     true
 containerd:
  Version:          1.7.21
  GitCommit:        472731909fa34bd7bc9c087e4c27943f9835f111
 runc:
  Version:          1.1.13
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    26.1.3
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.17.1-desktop.1
    Path:     /Users/spkane/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.7-desktop.1
    Path:     /Users/spkane/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.37
    Path:     /Users/spkane/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.15
    Path:     /Users/spkane/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/spkane/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.27
    Path:     /Users/spkane/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/spkane/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     /Users/spkane/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/spkane/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.14.0
    Path:     /Users/spkane/.docker/cli-plugins/docker-scout

Server:
 Containers: 117
  Running: 1
  Paused: 0
  Stopped: 116
 Images: 58
 Server Version: 27.3.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 472731909fa34bd7bc9c087e4c27943f9835f111
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.11-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 7.751GiB
 Name: docker-desktop
 ID: 0a9b2318-fa26-4178-baaa-dc8dbcdc0fd0
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/spkane/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: true
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Additional Info

No response

spkane avatar Nov 15 '24 19:11 spkane

Hi @spkane , I think this is not even a issue. I have checked. If you do docker system prune and then run docker build --no-cache . then it doesn't print it shows below output. $ docker build --no-cache . ... => [ 1/13] FROM docker.io/library/alpine:3.20@sha256:1e42bbe2508154c9126d48c2b8a75420c3544343bf86fd041fb752 0.0s ...

By printing CACHED in front of FROM statement is that it is already cached and picking it from there. If you see from command is fetching image from docker hub right there is no way it could skip caching it. Image is building block of docker containers. So you put --no-cache or not the images which will you fetch in dockerfile those will be cached and stored in your local.

On the second time running this command again then it rebuild the image from scratch because except from all other layers are not cached due to no-cache flag.

I hope this justifies the issue stated above. Please close this issue.

octocamocoder47 avatar Nov 30 '24 08:11 octocamocoder47

@octocamocoder47 Maybe my description wasn't clear enough. I am not confused about WHY it is saying CACHED. I am confused about why it isn't saying CACHED in BOTH use cases. If the base image is CACHED, it only says CACHED when you use --no-cache which is extremely odd. It should still say CACHED when using the build cache if the base image is indeed CACHED, yet it does not.

spkane avatar Nov 30 '24 16:11 spkane

This looks to be a duplicate of a ticket I once created, because it confused me as well;

  • https://github.com/moby/moby/issues/38255

The explanation here is that --no-cache will instruct BuildKit to re-run steps that may be non-deterministic (e.g. a RUN <download and install>).

For FROM (and other steps that may involve pulling images from a registry), it will contact the registry to resolve the image's content-addressable digest. That digest may still be in the build-cache, and as it's content-addressable, it does not have to pull / download the image again, even if --no-cache is set. In that case, it prints CACHED as it verified the cache, and was still able to use it.

So, it's mostly a presentation issue; comments on my original ticket suggested changing the output to (e.g.) CHECKED or EXISTS to indicate "something happened"

  • https://github.com/moby/moby/issues/38255#issuecomment-441112692
  • https://github.com/moby/moby/issues/38255#issuecomment-442129397

But the ticket was closed, and I don't know if someone ever got around to opening a proposal in the BuildKit issue tracker for this (https://github.com/moby/buildkit/issues)

thaJeztah avatar Feb 13 '25 13:02 thaJeztah

Let me transfer this ticket to the buildx issue tracker (I can't move it to the BuildKit issue tracker, as GitHub doesn't allow transferring between orgs 😅(

thaJeztah avatar Feb 13 '25 13:02 thaJeztah