Circular Reference Error in Docker Compose v2.24.5 with extends Feature

[asmundskalevik]

trafficstars

Description

After upgrading to Docker Compose version 2.24.5, I encountered a "Circular reference" error when using the extends feature across multiple docker-compose.yml files. This issue did not occur in version 2.24.2. The setup involves three Docker Compose files where one service extends from another, which in turn extends from a base service defined in another file. The intended behavior is to inherit configurations across these files without issues.

Steps To Reproduce

Create three Docker Compose files (base.yaml, intermediate.yaml, final.yaml) with the following content: base.yaml:

version: '3.8'
services:
  app_base:
    image: alpine
    command: ["echo", "Hello from Base Service"]

intermediate.yaml:

version: '3.8'
services:
  app_service:
    extends:
      file: base.yaml
      service: app_base
    environment:
      - APP_MODE=intermediate

final.yaml:

version: '3.8'
services:
  app_service:
    extends:
      file: intermediate.yaml
      service: app_service
    ports:
      - "8080:80"

Run docker compose -f final.yaml up with Docker Compose version 2.24.5-1.

Expected Result: The service app_service starts up successfully, inheriting configurations from both base.yaml and intermediate.yaml.

Actual Result: Received a circular reference error:

Circular reference:
  app_service in /path/to/final.yaml
  extends app_service in /path/to/final.yaml

Compose Version

Docker Compose version v2.24.5

Docker Environment

Client: Docker Engine - Community
 Version:    25.0.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.5
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 15
  Running: 10
  Paused: 0
  Stopped: 5
 Images: 130
 Server Version: 25.0.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.0-18-generic
 Operating System: Ubuntu 22.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 23.08GiB
 Name: hi-07679
 ID: MNE3:34UN:ICEY:TH6C:JDAJ:SL7M:ONQF:AQ5J:ORRX:N46F:Q42L:OLD7
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: aasmunds
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

The problem persists only with Docker Compose version 2.24.5-1. This issue significantly impacts development workflows that rely on the extends feature for modular and reusable service configurations.

[ndeloof]

This has been fixed already in v2.24.6, could you please give it a try to confirm ?

[gisostallenberg]

When I look at the 'Fixes' in https://github.com/docker/compose/releases/tag/v2.24.6, I don't see this mentioned here.

Is this fix the reason I'm seeing this?

service "foo-service" can't be used with `extends` as it declare `links`

[ndeloof]

@gisostallenberg this doesn't appear on docker/compose release note as this is related to the compose file parser library see https://github.com/compose-spec/compose-go/pull/568

[dannyroberts]

Looks like the latest version of Docker Desktop doesn't yet have access to this docker-compose bug fix? https://docs.docker.com/desktop/release-notes/#4272

[AngellusMortis]

2.24.6 is still not in the Linux apt repos.

[asmundskalevik]

I have changed the 3 files a bit: base.yaml

version: '3.8'
services:
  app_base:
    image: alpine
    command: ["echo", "Hello from Base Service"]

intermediate.yaml

version: '3.8'
services:
    db_service:
    image: postgres
    environment:
      POSTGRES_DB: exampledb
      POSTGRES_USER: exampleuser
      POSTGRES_PASSWORD: examplepass
    app_service:
    extends:
      file: base.yaml
      service: app_base
    environment:
      - APP_MODE=intermediate
    depends_on:
      - db_service

final.yaml

version: '3.8'
services:

  db_service:
    extends:
      file: intermediate.yaml
      service: db_service
    environment:
      - DB_USER=user
  app_service:
    extends:
      file: intermediate.yaml
      service: app_service
    ports:
      - "8080:80"

apt install docker-compose-plugin=2.24.2-1~ubuntu.22.04~jammy
docker compose -f final.yaml up -d
docker compose -f final.yaml up -d
[+] Running 2/2
 ✔ Container dockerbug-db_service-1   Started                              0.7s 
 ✔ Container dockerbug-app_service-1  Started                              0.6s 

BUT:

apt install docker-compose-plugin=2.24.6-1~ubuntu.22.04~jammy
docker compose -f final.yaml up -d
service "app_service" can't be used with `extends` as it declare `depends_on`

2.24.6-1 now comes with this bug which still impacts my development significantly.:(

[ndeloof]

2.24.6-1 now comes with this bug which still impacts my development significantly.:(

This constraint was well documented, and I'm surprised so many users just discover this as some check was added to the parser. wonder why I spend so much time writing docs 😅 Anyway, based on user feedback, see https://github.com/compose-spec/compose-go/pull/591

[ndeloof]

Closing this issue as the original reported bug is confirmed to be fixed