some .env values are ignored

[heidricha]

Describe the bug COMPOSE_FILE value from .env is always overwritten with the default value.

To Reproduce

$ cat .env 
COMPOSE_FILE=docker-compose.yaml
COMPOSE_PROJECT_NAME=blahblah
$ cat docker-compose.yaml 
version: "2.4"
services:
  srv:
    image: alpine
sagemcom@prod-test:~/test$ cat docker-compose.override.yaml 
services:
  srv2:
    image: alpine

$ ~/podman-compose/podman_compose.py ps
podman-compose version: 1.0.4
['podman', '--version', '']
using podman version: 3.0.1
 ** merged:
 {
  "version": "2.4",
  "services": {
    "srv": {
      "image": "alpine"
    },
    "srv2": {
      "image": "alpine"
    }
  },
  "_dirname": "/home/sagemcom/test"
}
podman ps -a --filter label=io.podman.compose.project=test
CONTAINER ID  IMAGE   COMMAND  CREATED  STATUS  PORTS   NAMES
exit code: 0
$ export COMPOSE_FILE=docker-compose.yaml # defined the same value, 
$ ~/podman-compose/podman_compose.py ps
podman-compose version: 1.0.4
['podman', '--version', '']
using podman version: 3.0.1
podman ps -a --filter label=io.podman.compose.project=test
CONTAINER ID  IMAGE   COMMAND  CREATED  STATUS  PORTS   NAMES
exit code: 0

Expected behavior .env values should be used, if neither command line arg, nor environment variable of the same name is present

Actual behavior some variables are always overwritten (compared to the value in .env file), even if documented logic says those shouldn't

Output

$ podman-compose --version
['podman', '--version', '']
using podman version: 3.0.1
podman-composer version  1.0.3
podman --version 
podman version 3.0.1
exit code: 0

Environment:

• OS: Linux / WSL / Mac
• podman version: 3.0.1
• podman compose version: (1.0.3)

Additional context

The problem is the applied logic in _parse_compose_file(self): It should be something like this:

• .env file should be used as a set of default values for all variables in the file
• then comes the os.environ, which possibly overrides the defaults, if the same var is present
• then comes the command line argument, if provided, which has the highest precedence

[Octobug]

I ran into the same problem as yours. My versions:

$ podman-compose --version
podman-compose version: 1.0.4
['podman', '--version', '']
using podman version: 4.0.2

The var COMPOSE_PROJECT_NAME= within .env does not take effect。

• OS: macOS Monterey v12.2.1
• podman version: 4.0.2
• podman compose version: (1.0.4)

[tstaig]

I believe the precedence should comply with docker-compose precedence: https://docs.docker.com/compose/environment-variables/envvars-precedence/

I haven't checked podman-compose's current behavior thoroughly, but at least the precedence order is not compliant for shell environment > .env file.

Minimal test:

> tree -a .
.
├── docker-compose.yml
├── Dockerfile
└── .env

Dockerfile:

FROM quay.io/centos/centos:stream8
ARG TEST_VAR
RUN echo TEST_VAR: ${TEST_VAR}

docker-compose.yml:

version: "3.4"

services:
   test:
      image: testimage
      build:
         args:
            TEST_VAR: ${TEST_VAR}

.env:

IMAGE_OS=test
#TEST_VAR=.ENV

Build:

> podman-compose build --no-cache test
STEP 3/3: RUN echo TEST_VAR: ${TEST_VAR}
TEST_VAR:

Build with shell variable:

> TEST_VAR=SHELL podman-compose build --no-cache test
STEP 3/3: RUN echo TEST_VAR: ${TEST_VAR}
TEST_VAR: SHELL

Build with .env variable: .env:

IMAGE_OS=test
TEST_VAR=.ENV

TEST_VAR=SHELL podman-compose build --no-cache test
STEP 3/3: RUN echo TEST_VAR: ${TEST_VAR}
TEST_VAR: .ENV

As you can see, the behavior described in docker-compose's documentation is not respected. I'm using podman-compose 1.0.3:

> podman-compose -v
['podman', '--version', '']
using podman version: 4.2.0
podman-composer version  1.0.3
podman --version 
podman version 4.2.0
exit code: 0

[jhoffmann]

It's still not working in 1.0.6:

> ls docker-compose*
docker-compose-local-proxy.yml	
docker-compose-local.yml  
docker-compose-stock.yml  
docker-compose-win.yml  
docker-compose.yml

> more .env
COMPOSE_FILE=docker-compose-local.yml

> docker-compose config --services
elasticsearch79
mail
mysql
redis
web

> podman-compose config --services
podman-compose version: 1.0.6
['podman', '--version', '']
using podman version: 4.6.1
web

One workaround I did find was to switch from a .env file to a .envrc file and use direnv, which will automatically load the contents when you switch to the directory. Obviously wouldn't work for automation or production deployments, but works fine for my local dev scenario.

> more .envrc
layout php
layout node

source_up_if_exists
export COMPOSE_FILE=docker-compose-local.yml
export COMPOSE_PROJECT_NAME=devsite

[hlovdal]

I am also hit by this. Using monkey.yaml and monkey.override.yaml from #371 it works with docker-compose:

$ cat .env
COMPOSE_PATH_SEPARATOR=:
COMPOSE_FILE=monkey.yaml:monkey.override.yaml
$ docker-compose up
[+] Running 2/2
 ✔ Network env-test_default  Created                                                                0.1s
 ✔ Container env-test-one-1  Created                                                                0.1s
Attaching to env-test-one-1
env-test-one-1  | monkey 2
env-test-one-1 exited with code 0

$

but fails with podman-compose:

$ podman-compose up
podman-compose version: 1.0.6
['podman', '--version', '']
using podman version: 4.4.1
no compose.yaml, docker-compose.yml or container-compose.yml file found, pass files with -f
$