The `pull_policy: always` property is forcing the use of the image defined in the `image` property instead of using the image built in the `build` property

[xserrat]

Description

I have a service that is defined in the docker-compose.yaml as well as in the docker-compose.override.yaml. The difference is basically that in the docker-compose.yaml I'm referencing the image property while in the docker-compose.override.yaml I have the build property to build the image based on a local Dockerfile. Also, the docker-compose.yaml contains a pull_policy: always property that is inherited in the docker-compose.override.yaml.

The problem is that the container does not start with the generated image from the Dockerfile if I have in the parent yaml (docker-compose.yaml) the pull_policy: always. It starts with the parent image defined. This is the configuration:

# docker-compose.yaml
services:
    myservice:
        image: an-image
        pull_policy: always

# docker-compose.override.yaml
services:
    myservice:
        build:
             dockerfile: docker/Dockerfile
             context: .

You can see a complete example with an nginx image here -> https://github.com/xserrat/docker-compose-pull-policy

Is that the expected behaviour?

No. I expected the use of the overridden image defined in the build.dockerfile property in the docker-compose.override.yaml file.

Steps to reproduce the issue:

1. Create a docker-compose.yaml like the following:

# docker-compose.yaml
services:
    myservice:
        image: an-image
        pull_policy: always

2. Create a docker-compose.override.yaml overriding the previous service by adding the build property to build a custom Dockerfile:

# docker-compose.override.yaml
services:
    myservice:
        build:
             dockerfile: docker/Dockerfile
             context: .

3. Execute docker compose up -d myservice.
4. You'll see that the image used in the container is the one defined in the docker-compose.yaml instead of using the image from the build.dockerfile property.
5. You can see a complete example here: https://github.com/xserrat/docker-compose-pull-policy

Describe the results you received:

I see that the image used to run the container is not the local one built using the Dockerfile.

Describe the results you expected:

I should see that the image used to run the container is the local one because the image property defined in the docker-compose.yml is overridden by the build property from the docker-compose.override.yml file.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker compose version:

Docker Compose version v2.6.1

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.8.2)
  compose: Docker Compose (Docker Inc., v2.6.1)
  extension: Manages Docker extensions (Docker Inc., v0.2.7)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 42
  Running: 27
  Paused: 0
  Stopped: 15
 Images: 44
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc version: v1.1.2-0-ga916309
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.10.104-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 8.721GiB
 Name: docker-desktop
 ID: AKFP:GZQE:Y6C5:R4RQ:RGWI:AQBD:U6BG:JGQV:AFUP:GEZP:5K6P:PI4C
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:

I am using a Mac M1.

[thaJeztah]

Hm.., I see where you're coming from. That said; my immediate thought here would be that, while image and build are related, only setting a build option wouldn't fully "override" the image section. Effectively, after merging, the service would look like;

services:
  myservice:
    build:
      context: .
      dockerfile: docker/Dockerfile
    image: an-image
    pull_policy: always

In the above;

• running docker compose up would follow the pull policy (always pull)
• docker compose would check if an-image exists (which it now would if the pull was successful), and only trigger a build if not

When explicitly asking for compose to build (--build) the image, the pull is not executed (but the image is built instead);

docker compose up --build
[+] Building 6.2s (6/6) FINISHED
 => [internal] load build definition from Dockerfile                                             2.2s
 => => transferring dockerfile: 75B                                                              0.1s
 => [internal] load .dockerignore                                                                2.7s
 => => transferring context: 2B                                                                  0.0s
 => [internal] load metadata for docker.io/library/alpine:latest                                 0.0s
 => [1/2] FROM docker.io/library/alpine:latest                                                   0.2s
 => [2/2] RUN echo foo > bar                                                                     2.0s
 => exporting to image                                                                           0.3s
 => => exporting layers                                                                          0.2s
 => => writing image sha256:72f4b292d9ee8ec274f12397a22080e8f993be28a0af77980c014a0f3d82fe5a     0.0s
 => => naming to docker.io/library/an-image                                                      0.0s
[+] Running 2/2
 ⠿ Network issue-9730_default        Created                                                     0.4s
 ⠿ Container issue-9730-myservice-1  Created                                                     0.8s
Attaching to issue-9730-myservice-1

[thaJeztah]

Forgot to post; so the compose-spec describes this; https://github.com/compose-spec/compose-spec/blob/master/spec.md#pull_policy

pull_policy

pull_policy defines the decisions Compose implementations will make when it starts to pull images. Possible values are:

• always: Compose implementations SHOULD always pull the image from the registry.
• never: Compose implementations SHOULD NOT pull the image from a registry and SHOULD rely on the platform cached image. If there is no cached image, a failure MUST be reported.
• missing: Compose implementations SHOULD pull the image only if it's not available in the platform cache. This SHOULD be the default option for Compose implementations without build support. if_not_present SHOULD be considered an alias for this value for backward compatibility
• build: Compose implementations SHOULD build the image. Compose implementations SHOULD rebuild the image if already present.

If pull_policy and build both presents, Compose implementations SHOULD build the image by default. Compose implementations MAY override this behavior in the toolchain.

From that, indeed, if both are set, compose should build the image;

If pull_policy and build both presents, Compose implementations SHOULD build the image by default. Compose implementations MAY override this behavior in the toolchain.

The combination of some of those is a bit ambiguous, as it doesn't describe in which cases it should build (if the image is missing, or always?); effectively that would mean that adding a build section implicitly sets pull_policy to build. Perhaps it would be better to make it either produce an error if pull_policy is explicitly set to always or missing and a build section is also present, or the current behavior (pull_policy=always or pull_policy=missing taking precedence unless --build is used. The --build flag is not part of the compose file specification though, so that may be difficult to describe as part of the spec).

[xserrat]

First of all @thaJeztah, thanks for your fast reply!

I see your point... I guess that the option to produce an error is a good solution so we have fast feedback and we can understand which is the issue in our configuration. Something like:

Error: Unable to start service XXX. `build` option can only be used with `pull_policy: build`, `pull_policy: always` used. 

Also, we could only display a warning message and do the current behaviour, so at least the user can see which is the behaviour:

Warning: Service XXX uses the `image XXXX` instead of building the image from the defined Dockerfile

Anyway, thanks for your explanation, now it's clear for me so I think I'm going to change my override YAML to the following to make it explicit:

services:
  myservice:
    build:
      context: .
      dockerfile: docker/Dockerfile
    pull_policy: build

[thaJeztah]

It's definitely a tricky one! And I recall I had some debates with others wether or not pull_policy should be part of the compose file, as it's not directly a definition of the compose service itself, and more an "operational" concern (better to be passed as command-line argument?), but this really depends on how the compose file is looked at, and there's many different use-cases on how compose files are used. I also had a proposal to make build more independent from the service's definition (https://github.com/compose-spec/compose-spec/issues/188), but that needs further discussion, and I'd have to look if that would make this less ambiguous :thinking:.

So yes, the ambiguity is mostly that there can be different intents, both of which are "valid", but from a different angle ("1" to be more for deploying, and "2" to be more for local development);

1. pull_policy: always - I always want the image to be pulled from the registry to prevent locally built images from overwriting the last image that was successfully built and pushed.
2. build: - I'm using this compose-file for local development and want local builds to be preferred over pulling the image (prevent overwriting my last local build), so that I can test locally before pushing.

[ndeloof]

with your setup, docker compose config makes it clear pull_policy: always is preserved after override has been applied, and so it is expected the image reference is pulled and used for service. Your override file should set pull_policy: build if your goal is to force a build to take place. You also soon will be able in such an override to reset the attribute to it's default value:

services:
  myservice:
    build:
      context: .
      dockerfile: docker/Dockerfile
    pull_policy: !reset

see https://github.com/compose-spec/compose-spec/blob/master/13-merge.md#reset-value