docker-compose v2.0.0-rc.1 breaks backward compatibility for variable names in env files

[fsiegrist]

Steps to reproduce the issue:

1. Add the following to your docker-compose.yaml:

    env_file:
    - my_env_file.env

2. Add a variable containing a hyphen in its name to the .env file: my-variable=value
3. Call docker-compose up or docker compose up

Result received: invalid variable name "my-variable"

Result expected: docker-compose up runs without errors

Output of docker-compose --version:

Docker Compose version v2.0.0-rc.1

Output of docker version:

Docker version 20.10.8, build 3967b7d

Output of docker context inspect default:

[
    {
        "Name": "default",
        "Metadata": {
            "StackOrchestrator": "swarm"
        },
        "Endpoints": {
            "docker": {
                "Host": "unix:///var/run/docker.sock",
                "SkipTLSVerify": false
            }
        },
        "TLSMaterial": {},
        "Storage": {
            "MetadataPath": "\u003cIN MEMORY\u003e",
            "TLSPath": "\u003cIN MEMORY\u003e"
        }
    }
]

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
  compose: Docker Compose (Docker Inc., v2.0.0-rc.1)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 2
  Running: 1
  Paused: 0
  Stopped: 1
 Images: 98
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b63
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.47-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 1.938GiB
 Name: docker-desktop
 ID: BHJO:7FXM:LN3W:7I3X:XSGB:BG7Z:HCRR:4T7K:QYJU:QSNH:RO6H:C2Z3
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

[fsiegrist]

If you add the variables directly in the docker-compose.yaml, docker-compose does not complain:

environment:
  - my-variable=value

Seems not to be consistent with the behavior of .env files.

[ndeloof]

Tested on v2.0.0-rc.2, can't reproduce. Could you please give it a try?

[fsiegrist]

I use Docker Desktop for Mac. The latest version (3.6.0.5487) includes docker-compose v2.0.0-rc.1 If the problem is fixed in rc2, I deactivate the experimental feature "Use Docker Compose V2 release candidate" and wait until Docker Desktop for Mac comes with docker-compose v2.0.0-rc.2

[ndeloof]

Could you please download rc2 from https://github.com/docker/compose-cli/releases and install under $HOME/.docker/cli-plugins to confirm you can't reproduce this issue anymore with rc2 ?

[fsiegrist]

Still getting the same error:

$ docker-compose --version
Docker Compose version v2.0.0-rc.2
$ docker-compose up
invalid variable name "backend-host"

[ndeloof]

Was able to reproduce, investigating

[ndeloof]

issue is caused by https://github.com/ulyssessouza/godotenv/commit/d3051d55ab344e4765ca2a8f8c47a9996b658905 cc @ulyssessouza

[andyg101]

Also experiencing this issue. Are these variable names actually invalid or will docker compose be updated to accomodate them? If docker compose is going to enforce this validation going forward we will need to spend a lot of time updating our existing projects environment variables.

[ndeloof]

Are these variable names actually invalid

Documentation is unclear, and it seems we will need to align with de facto compose V1 behavior regarding this ambiguous syntax

[matschaffer]

Looks like env vars with dots (for example xpack.security.enabled) are still disallowed in .env for Docker Compose version 2.0.1

I made https://gist.github.com/matschaffer/9a3e6f00e5e94a725b74fb8ea7287a12 as a simple validation.

If you remove the .env file you can see hyphens and dots in env var names reaching the container env just fine.

# docker-compose installed from homebrew on macos

❯ ~/.docker/cli-plugins/docker-compose --version
Docker Compose version 2.0.1

❯ docker-compose up
[+] Running 2/0
 ⠿ Network docker-compose-check_default  Created                                                                                                                                                              0.0s
 ⠿ Container docker-compose-check-env-1  Created                                                                                                                                                              0.0s
Attaching to docker-compose-check-env-1
docker-compose-check-env-1  | inline.dots=ok
docker-compose-check-env-1  | inline-hypens=ok
docker-compose-check-env-1  | HOSTNAME=73b63ec39810
docker-compose-check-env-1  | PWD=/
docker-compose-check-env-1  | HOME=/root
docker-compose-check-env-1  | SHLVL=0
docker-compose-check-env-1  | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
docker-compose-check-env-1  | _=/usr/bin/env
docker-compose-check-env-1 exited with code 0

But docker-compose rejects them with this message if hyphens or dots appear in the env file.

unexpected character "-" in variable name near "env-hyphens=ok\nenv.dots=ok\n"

[nevergone]

Related commit: https://github.com/compose-spec/godotenv/blob/655269e2565760ac457e473904300398c6150d47/parser.go#L113

[nevergone]

If the variable name contains a dot in the env_file key file, this will cause an error in docker-compose 2.x. If the same variable is below environment key, it works. This worked perfectly fine under 1.29.x, please reset it. Why does docker-compose need to handle this? Why doesn't the container shell do this?

[mcfriend99]

Is anyone working on a fix for this??

[ndeloof]

dot env file parser has been integrated in compose-go so we have more flexibility on supported format, and rules have been relaxed to support dots and comas https://github.com/compose-spec/compose-go/tree/master/dotenv