Compose V1 doesn't behave as V2 when referencing environment variables across multiple .env files

[Nico769]

Description

Environment variable substitution doesn't seem to behave consistently in Compose V1 and V2 when:

• multiple .env files are used AND;
• one .env file declares a variable that is referencing a different variable, declared in another .env file.

Steps to reproduce the issue:

See the following minimal reproducible example with some (hopefully!) clarifying comments

.env file

TO_REFERENCE="world!"

# Using TO_REFERENCE variable here works,
# either when `docker-compose up` or `docker compose up` is being used.
# Therefore, variable substitution happens as expected.
WORKING_SUBSTITUTION="Hello ${TO_REFERENCE}"

.another.env

# Using TO_REFERENCE variable here doesn't work when `docker-compose up` is being used,
# while `docker compose up` makes the substitution happen as expected.
# 
# In other words, the final value should be "Hello beautiful world!"
# but the variable is left empty if `docker-compose up` is being used. 
NON_WORKING_SUBSTITUTION="Hello beautiful ${TO_REFERENCE}"

docker-compose.yaml

version: '3.9'

services:
  helloworld:
    container_name: hello-world
    image: hello-world
    env_file:
      - .env 
      - .another.env

Describe the results you received:

Running docker inspect hello-world | grep -C 5 Env shows how Hello beautiful is assigned to NON_WORKING_SUBSTITUTION environment variable if docker-compose up is being used

[...]
"Env": [
    "TO_REFERENCE=world!",
    "WORKING_SUBSTITUTION=Hello world!",
    "NON_WORKING_SUBSTITUTION=Hello beautiful ",
    "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],

Describe the results you expected:

Running docker inspect hello-world | grep -C 5 Env should give the following if docker-compose up is being used

[...]
"Env": [
    "TO_REFERENCE=world!",
    "WORKING_SUBSTITUTION=Hello world!",
    "NON_WORKING_SUBSTITUTION=Hello beautiful world!",
    "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],

Output of docker compose version:

Docker Compose version v2.7.0

Output of docker-compose version:

docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1g  21 Apr 2020

Output of docker version:

Client: Docker Engine - Community
 Cloud integration: v1.0.28
 Version:           20.10.17
 API version:       1.41
 Go version:        go1.17.11
 Git commit:        100c701
 Built:             Mon Jun  6 23:03:17 2022
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true

Server: Docker Desktop
 Engine:
  Version:          20.10.17
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.11
  Git commit:       a89b842
  Built:            Mon Jun  6 23:01:23 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.6
  GitCommit:        10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc:
  Version:          1.1.2
  GitCommit:        v1.1.2-0-ga916309
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.8.2)
  compose: Docker Compose (Docker Inc., v2.7.0)
  extension: Manages Docker extensions (Docker Inc., v0.2.8)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 2
  Running: 1
  Paused: 0
  Stopped: 1
 Images: 19
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc version: v1.1.2-0-ga916309
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.60.1-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 5.8GiB
 Name: docker-desktop
 ID: 2OTH:M7O7:3J54:BFMN:YT4P:6KV5:Y66L:225T:H4UZ:FNL3:L3ES:JWP7
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

Cheers!

[curry684]

This was supposed to be fixed by https://github.com/docker/compose/pull/9636 in v2.7, but I'm also not seeing any changes at all. I'm forced to downgrade to Compose 1 as well (again....) as Compose 2 keeps breaking production environments with undefined, messy and inconsistent env file handling.

[Nico769]

Thanks for sharing your thoughts and the PR @curry684!

I'd argue that V2, in this instance, is behaving correctly while V1 has a bug somewhere in the variables resolution part of the code. Indeed, my example shows that V1 doesn't build the string as expected while V2 does.

[radoslawkoziol]

I am having a similar issue. I have the "env_file" config with just one ".env.test" file connected and the container inside has env variables randomly taken, some from .env.test and some from .env

eg.

#.env file
VAR_1=env1
VAR_2=env2

#.env.test file
VAR_1=env_test1
VAR_2=env_test2

env_file:
     - .env.test

printing ENV variables from the inside of the container

VAR_1=env1
VAR_2=env_test2

It worked fine in docker 4.6.1 (docker used variables only from the .env.test file) but stopped working correctly after updating to 4.10.

[ulyssessouza]

@Nico769 As you stated in https://github.com/docker/compose/issues/9725#issuecomment-1210425431 the behaviour in V2 is the expected. What happens is that V2 evaluates the variables in .env (with precedence over the OS env) making them available for interpolation.

[ulyssessouza]

@radoslawkoziol I just tested your test case with Docker Compose v2.9.0 and got the expected result:

VAR_1=env_test1
VAR_2=env_test2

[radoslawkoziol]

@radoslawkoziol I just tested your test case with Docker Compose v2.9.0 and got the expected result:

I mean, I know it sounds weird but it happens randomly. Sometimes it works fine, sometimes it doesn't, without changing anything, just after docker restart. It's not only my PC, My colleague's one behaves the same.

Is there something I can send you to help debug? Some logs?

[ulyssessouza]

@radoslawkoziol Did you try it with the latest version (v2.9.0)?

Actually the evaluation is not meant to be concurrent, so it should not have any race condition problems. Could you please post a minimal test case with all the relevant files?

[radoslawkoziol]

@ulyssessouza I'm not sure how to update windows docker-compose to 2.9. My docker desktop app shows I have the newest version of docker installed.

docker-compose.yml

version: "3.7"
services:
  test:
    image: nginx:alpine
    env_file:
      - .env.adapter

.env

MYSQL_HOST=middleware_db
TEST1=env_test1
TEST2=env_test2
TEST3=env_test3

.env.adapter

MYSQL_HOST=adapter_db
TEST1=env_adapter_test1
TEST2=env_adapter_test2
TEST3=env_adapter_test3

Docker v2 (Docker Compose version v2.7.0) result:

Docker v1 (docker-compose version 1.29.2, build 5becea4c) result:

Operating system: W10 + WSL2

Docker version:

[Nico769]

Hi @ulyssessouza, thanks for confirming that!

I wanted to assess whether this is indeed a bug in V1, so I tried to run the test suite (i.e. make test on git tag 1.29.2) but I've came across several errors during the build process, some of which I managed to fix. I've also tried to join the community Slack channel to ask for help, but apparently I need an invite of some sort.

Since I'm not sure this is the right place to troubleshoot my build errors, I'm attaching the test case I've come up with in order to investigate further into the matter. Hopefully it's helpful to others while I figure out what's wrong with the build.

tests/integration/service_test.py

    def test_env_variables_interpolation_across_multiple_env_files(self):
        service = self.create_service(
            'web',
            env_file=['tests/fixtures/env/first.env', 'tests/fixtures/env/referencing.env'])
        env = create_and_start_container(service).environment
        for k, v in {
            'ONE': '1',
            'FOO': 'bar',
            'INTERPOLATION': 'barbaz'
        }.items():
            assert env[k] == v

tests/fixtures/env/first.env

# first.env

ONE=1
FOO=bar

tests/fixtures/env/referencing.env

# referencing.env

INTERPOLATION=${FOO}baz

Cheers.

[ndeloof]

tested with latest v2.17.3 release, and get the expected variable set:

TO_REFERENCE=world!
WORKING_SUBSTITUTION=Hello world!
NON_WORKING_SUBSTITUTION=Hello beautiful world!

so, closing this issue as fixed. Feel free to comment or open a new issue if some corner cases haven't been addressed