docker-compose v2 not working with multiple configuration files

[shadowspawn]

Description

I upgraded from Mac Docker Desktop 4.5.0 to 4.6.0 and some of my docker workflows broke. In a situation with services split across two configuration files and services in the second depending on services in the first, I now getting an error saying no such service: some-service-in-first.

Steps to reproduce the issue:

1. Setup with multiple configurations files. Some services in file2 depend on services defined in file1.
2. "Use Docker Compose V2" enabled in preferences
3. run docker-compose -f file1 -f file2 run service-in-file2

Describe the results you received:

Error No such service: some-service-in-first. The actual service listed in the error varies.

Describe the results you expected:

Error free operation.

Additional information you deem important (e.g. issue happens only occasionally):

I tried unchecking "Use Docker Compose V2" in the preferences and it fixed the error.

The service listed in the No such service does vary.

The configuration files are using version: '2.3'. I am not sure that is significant, but thought it worth mentioning.

Output of docker compose version:

 % docker-compose --version
Docker Compose version v2.3.3

Output of docker info:

% docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.8.0)
  compose: Docker Compose (Docker Inc., v2.3.3)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 6
  Running: 1
  Paused: 0
  Stopped: 5
 Images: 135
 Server Version: 20.10.13
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2a1d4dbdb2a1030dc5b01e96fb110a9d9f150ecc
 runc version: v1.0.3-0-gf46b6ba
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.10.104-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 15.64GiB
 Name: docker-desktop
 ID: I7EO:FMG4:UPPV:TSUZ:46GH:ZWQM:GJ6H:T5DN:TM2D:RR6Y:K6WJ:GL4K
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 45
  Goroutines: 49
  System Time: 2022-03-17T01:13:59.790909746Z
  EventsListeners: 5
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:

Working with: Mac Docker Desktop version 4.5.0. Compose: v2.2.3

Problems with: Mac Docker Desktop version 4.6.0. Compose: v2.3.3

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as not stale anymore due to the recent activity.

[shadowspawn]

I tried again with the current version. Still broken, but the error behaviour has changed.

% docker-compose --version
Docker Compose version v2.10.2
% docker-compose -f docker-compose.yml -f docker-compose.admin.yml run mysql-cli
Invalid template: "$270466015406309"

[milas]

For this error:

Invalid template: "$270466015406309"

A fix will be in the next Compose version (about to be released). It's possible to workaround it on older Compose versions by single-quoting the value so that no interpolation happens:

FOO='$270466015406309'

(NOTE: This is independent from the original issue you reported.)

[shadowspawn]

I'll fix the interpolation problem and see if the original problem remains. Thanks for tip @milas . (Took me a while to realise we actually did have that value in a dusty corner of an environment file!)

[shadowspawn]

After fixing the enviroment variable, looking hopeful. Working with my initial tests.

Mac Docker Desktop version 4.12.0 Compose: 2.20.2

[shadowspawn]

I no longer see the problems I originally reported. I can enable "Use Docker Compose V2" and work across multiple configuration files.