[Bug]: Env variables set in coolify are overriden by default variables in .env file

[noximo]

Description

In the last day or two, my builds started failing due to already used ports.

But I have ports defined in the compose.yml using the env variable.

I've noticed that plenty of envs I've set up display this message: This variable is not found in the compose file, so it won't be used.

That's problematic for several reasons:

1. Some of the variables actually are present in the compose.yml. From what I gathered, only variables used in the environment: section of the configuration of the service are detected:

services:
    php:
        image: ${IMAGES_PREFIX:-}app-php
        restart: unless-stopped
        environment:
            SERVER_NAME: ${SERVER_NAME:-localhost}, php:80 # <-- FOUND
        ports:
            # HTTP
            -   target: 80
                published: ${HTTP_PORT:-80} # <-- NOT FOUND
                protocol: tcp

2. I have several compose files and it looks like only the main one gets analyzed.

I'm starting the containers with this command: docker compose -f compose.yml -f compose.prod.yml up -d

and compose.prod.yml looks (simplified) like:

services:
    php:
        build:
            context: .
            target: frankenphp_prod
        environment:
            APP_SECRET: ${APP_SECRET} # <-- NOT FOUND

3. Not all variables are relevant to the compose.yml.

I have variables set up that aren't and won't be part of the compose.yml. I need them present during build step because one of the build steps of Dockerfile is calling symfony command that dumps all env variables into php file. That worked well so far. ,

Minimal Reproduction (if possible, example repository)

Probably any env variable that's not present in the main compose.yml under environment key. I will investigate further.

Exception or Error

No response

Version

v4.0.0-beta.262

[noximo]

I've tried to reference all variables in the compose.yml explicitly in hopes that it'll make them present. It didn't work, even though that notice went away.

So the error probably isn't related to the notice but may be the same error as referenced here: https://github.com/coollabsio/coolify/issues/1882#issuecomment-2061434692

[Kocal]

Oh, that's why my Symfony project is failing too! Env vars defined in Coolify's UI are not passed down to the PHP container, either by using printenv or cat .env.local.php :/

[Kocal]

Hum, can this be caused by https://github.com/coollabsio/coolify/commit/9c3b75757878b02033b72ee112b1f65c04e5c735? In Symfony app we commit the .env, but we expect it to be overridden by server's env vars

[noximo]

That looks like it. I've changed the name of the issue to reflect the real problem.

[Kocal]

As I workaround, I've downgraded Coolify (I don't think that's a good thing, but I don't have critical things on Coolify) to the latest working version for us (https://github.com/coollabsio/coolify/releases/tag/v4.0.0-beta.258) with this way:

cd /data/coolify/source
sh upgrade.sh 4.0.0-beta.258

Which outputs:

root@nasus:/data/coolify/source# sh upgrade.sh 4.0.0-beta.258
latest: Pulling from coollabsio/coolify-helper
Digest: sha256:4770886f7214920a14c97f408c0719b1f4a39c6839dbe0d152a33ca243ebea8b
Status: Image is up to date for ghcr.io/coollabsio/coolify-helper:latest
time="2024-04-17T21:49:53Z" level=warning msg="/data/coolify/source/docker-compose.yml: `version` is obsolete"
time="2024-04-17T21:49:53Z" level=warning msg="/data/coolify/source/docker-compose.prod.yml: `version` is obsolete"
 postgres Pulling 
 redis Pulling 
 soketi Pulling 
 coolify Pulling 
 coolify Pulled 
 redis Pulled 
 postgres Pulled 
 soketi Pulled 
 Container coolify-realtime  Recreate
 Container coolify-db  Recreate
 Container coolify-redis  Recreate
 Container coolify-db  Recreated
 Container coolify-redis  Recreated
 Container coolify  Recreate
 Container coolify-realtime  Recreated
 Container coolify  Recreated
 Container coolify-realtime  Starting
 Container coolify-db  Starting
 Container coolify-redis  Starting
 Container coolify-redis  Started
 Container coolify-db  Started
 Container coolify-redis  Waiting
 Container coolify-db  Waiting
 Container coolify-realtime  Started
 Container coolify-redis  Healthy
 Container coolify-db  Healthy
 Container coolify  Starting
 Container coolify  Started
root@nasus:/data/coolify/source#

And it nearly works, my Symfony app is accessible again, except that I can't access the project's env vars :D

[andrasbacsai]

Hm, I will revert those changes until further investigation.

[andrasbacsai]

I reverted in v264, can you please check now?

[Kocal]

Hi @andrasbacsai, I'm trying right now

[Kocal]

The v264 fixes the issue, env vars from Coolify UI are nicely passed down to the container, and my Symfony app is working!

EDIT : by the way, the env vars interpolation in other env vars is a really nice feature, thanks :)