[BUG] `docker-compose build` doesn't respect `cache_from`

[terrynguyen255]

Description

Built images are not being used when rebuilding images with docker-compose build This worked flawlessly in v2.17 but has stopped working in v2.20.

Steps To Reproduce

1. Run bash build-image.sh
2. Observe that cache for steps in Dockerfile are not used

# docker-compose.yml
version: '3'
services:
  my_server:
    image: my_server
    build:
      context: .
      dockerfile: path/to/Dockerfile
      cache_from:
        - my_server

# Dockerfile
FROM cypress/browsers:node14.19.0-chrome100-ff99-edge
COPY install-python3.sh scripts/ # should be cached
RUN bash scripts/install-python3.sh # should be cached
CMD start.sh

# build-image.sh
docker pull gcr.io/blablabla/my_server
docker tag gcr.io/blablabla/my_server my_server
docker-compose build

Compose Version

- Docker Compose version v2.17.3 -> WORKS
- Docker Compose version v2.20.3 -> NOT WORKING ANYMORE
- Docker Compose version v2.20.2-desktop.1 -> NOT WORKING ANYMORE

Docker Environment

Client:
 Version:    24.0.5
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.1
    Path:     /Users/tunguyen/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.20.2-desktop.1
    Path:     /Users/tunguyen/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/tunguyen/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/tunguyen/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.6
    Path:     /Users/tunguyen/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/tunguyen/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/tunguyen/.docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  0.20.0
    Path:     /Users/tunguyen/.docker/cli-plugins/docker-scout

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 20
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 5.15.49-linuxkit-pr
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 7
 Total Memory: 9.718GiB
 Name: docker-desktop
 ID: 021d80e1-3a9d-4ad3-8a3a-7df54e6d31fc
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Anything else?

No response

[ndeloof]

don't you get the same running docker buildx build -f path/to/Dockerfile --cache-from my_server . ?

[terrynguyen255]

Thank you for the response @ndeloof

don't you get the same running docker buildx build -f path/to/Dockerfile --cache-from my_server . ?

Yes! Exact symptom.

I also figured out that docker-compose build works fine on Google Cloud Build but not on Github Action ubuntu-latest. The followings are their versions. I hope that helps.

Google Cloud Build:

> cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

> docker-compose version
docker-compose version 1.25.0, build unknown
docker-py version: 4.1.0
CPython version: 3.8.10
OpenSSL version: OpenSSL 1.1.1f  31 Mar 2020

> docker compose version
Docker Compose version v2.21.0

> docker version
Client: Docker Engine - Community
 Version:           20.10.14
 API version:       1.41
 Go version:        go1.16.15
 Git commit:        a224086
 Built:             Thu Mar 24 01:48:02 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.24
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.19.7
  Git commit:       5d6db84
  Built:            Tue Apr  4 18:18:42 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Github Action ubuntu-latest:

> cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

> docker-compose version
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.7.10
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

> docker compose version
Docker Compose version v2.21.0

> docker version
Client: Docker Engine - Community
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:31:44 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:44 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[terrynguyen255]

Hi @ndeloof ,

I have another finding: docker version 20.10 works. I uninstall Github Action's docker and reinstall with curl https://releases.rancher.com/install-docker/20.10.sh

[maroux]

Is there any update on this? It kinda breaks all caching with compose and makes moving to v2 impossible.

[ndeloof]

If you can reproduce this behavior with docker buildx build ... command, please report to github.com/docker/buildx (which we rely on for the build command

[luciangabor]

... and check the docs too: https://docs.docker.com/reference/cli/docker/image/build/#cache-from For that to work with buildx (which became the default as of docker engine 23) the source of the cache-from needs to be built in a particular way

[maroux]

... and check the docs too: https://docs.docker.com/reference/cli/docker/image/build/#cache-from For that to work with buildx (which became the default as of docker engine 23) the source of the cache-from needs to be built in a particular way

Ah, thanks, this helps.

Is there a document describing how to use cache_to and cache_from with docker compose? That is, I'm guessing you can use an image that's cached using cache_to in cache_from directly, but the linked doc doesn't explain that.

[luciangabor]

Here's a link: https://docs.docker.com/build/cache/backends/