[BUG] `extends` with YAML merge anchors will ignore !override and !reset

[ShadowLNC]

Description

I think this might be two separate issues - please let me know and feel free to split it up.

End goal: 10+ containers use the configs entry with dummy data, so I need that to be inherited. However a small number (about 3) need a volume mount of real information, so I want those to override (but inherit from each other using YAML anchors/aliases) - these are represented by x and y in my example below..

When a volume is in the same location as a config, there is no error and it is undefined behaviour or a race condition as to which one is applied first. In my case, the config is actually a sub-path of the volume, and the volume is read only, so if the volume wins, Compose reports an error when trying to deal with the config, otherwise it just works.

Secondly, I have a complex inheritance system where extends is used in a YAML anchor. When declaring the configs element with a !reset or !override directive, the new configs declaration is lost/ignored when the alias is later referenced.

Because I was unaware of the second issue, I was unknowingly running into the first issue (though I had explicitly tried to avoid it). Had Compose had a clearer error message, the second issue would have been easier to identify.

Steps To Reproduce

Files:

stuff/file1 and stuff/file2 -> some random text

docker-compose.template.yml

services:
    base:
        configs:
            - source: credentials
              target: /credentials/file1

docker-compose.yml

services:
    x: &x
        extends:
            file: ./docker-compose.template.yml
            service: base
        image: alpine
        command: echo "hi"
        # configs: !reset []
        configs: !override
            - source: credentials
              target: /literally-anywhere-else
        volumes:
            - type: bind
              source: ./stuff/
              target: /credentials/
              read_only: true
              bind:
                  create_host_path: false

    y:
        <<: *x
        command: echo "y"

configs:
    credentials:
        content: |
            dummy value

Try docker compose up -d.

When the volume is mounted first (for container y), the race condition causes this error:

Error response from daemon: unlinkat /credentials/file1: read-only file system

Sometimes it will just work if the configs are applied first.

Compose Version

docker compose version
Docker Compose version v2.26.1-desktop.1

Docker Environment

Output of docker info

Client:
 Version:    26.0.0
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.13.1-desktop.1
    Path:     /Users/scott/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.26.1-desktop.1
    Path:     /Users/scott/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container. (Docker Inc.)
    Version:  0.0.27
    Path:     /Users/scott/.docker/cli-plugins/docker-debug
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/scott/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.23
    Path:     /Users/scott/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.4
    Path:     /Users/scott/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.1.0
    Path:     /Users/scott/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/scott/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.6.3
    Path:     /Users/scott/.docker/cli-plugins/docker-scout
WARNING: Plugin "/Users/scott/.docker/cli-plugins/docker-scan" is not valid: failed to fetch metadata: fork/exec /Users/scott/.docker/cli-plugins/docker-scan: no such file or directory

Server:
 Containers: 22
  Running: 1
  Paused: 0
  Stopped: 21
 Images: 37
 Server Version: 26.0.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.6.22-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 7.755GiB
 Name: docker-desktop
 ID: 85b60fc6-eedd-4b8c-996a-89f62a45ab85
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/scott/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Anything else?

docker compose config output

docker compose config
name: testing
services:
  x:
    command:
      - echo
      - hi
    configs:
      - source: credentials
        target: /literally-anywhere-else
    image: alpine
    networks:
      default: null
    volumes:
      - type: bind
        source: /Users/scott/Documents/infra/docker-dev-environment/testing/stuff
        target: /credentials/
        read_only: true
        bind: {}
  "y":
    command:
      - echo
      - "y"
    configs:
      - source: credentials
        target: /credentials/file1
      - source: credentials
        target: /literally-anywhere-else
    image: alpine
    networks:
      default: null
    volumes:
      - type: bind
        source: /Users/scott/Documents/infra/docker-dev-environment/testing/stuff
        target: /credentials/
        read_only: true
        bind: {}
networks:
  default:
    name: testing_default
configs:
  credentials:
    name: testing_credentials
    content: |
      dummy value

You can see that for service y, the configs directive is a merge of both base and x while ignoring the !override directive. A reset directive behaves the same way (it just inherits from base as x is empty).

[arankeparth]

@ShadowLNC Created pull request for this: https://github.com/compose-spec/compose-go/pull/618.

[ShadowLNC]

Hi @arankeparth Thanks for submitting the PR!

I've updated Docker Desktop to 4.30.0 and checked my Compose version:

$ docker compose version
Docker Compose version v2.27.0-desktop.2

✅ I confirmed that the !override directive seems to work. ❌ However, the !reset [] directive (commented out from my original post in favour of an override example) still does not clear the configs entry for service y when I run docker compose config.

As a workaround for now, I can use configs: !override [] in service x (instead of !reset []) to clear it for both x and y.