[BUG] Environment Variables not resolved in dockerfile for Service

[839998664]

Description

I have an .env file with an environment variable for the compose.yaml file.

HOSTDIR=/mnt/jenkins-test

.env file

name: test-jenkins
services:
  jenkins-controller:    
    working_dir: /var/jenkins_home
    user: root
    env_file: ./.env.container
    build: .
    ports:
      - "8080:8080"
    volumes:
      - ${HOSTDIR}:/var/jenkins_home

compose.yaml

This variable is resolved correctly. I have another file called .env.container for a service listed in the compose file. The service has its own dockerfile which consumes the environment variables from .env.container.

CASC_JENKINS_CONFIG=/var/jenkins_home/
JAVA_OPTS="-Djenkins.install.runSetupWizard=false"
JENKINS_ENV="Test"
JENKINS_THEME="blue"

.env.container

FROM jenkins/jenkins:lts-jdk11
COPY plugins.txt .
RUN jenkins-plugin-cli --plugin-file plugins.txt
COPY config/*.yaml ./
COPY css/style-${JENKINS_ENV}.css ./userContent/layout/style.css
COPY css/jenkins-material-theme-${JENKINS_THEME}.css ./userContent/layout/jenkins-material-theme.css

dockerfile

The docker compose config command correctly lists the environment variables from the .env.container file. However, when I run docker compose up -d it errors out with empty strings instead of values of the Environment variables from the .env.container file

Steps To Reproduce

1. In this environment Run docker compose config
2. Should list the environment variables from the .env.container file in the Environment section of the output.
3. Next, run docker compose up -d
4. Results in error message

failed to solve: failed to compute cache key: failed to calculate checksum of ref 3d92caec-cd9c-45d4-9bad-c2e0a8287271::xddxlfbf1d51f41yvil9r5mmz: "/css/jenkins-material-theme-.css": not found

Compose Version

Docker Compose version v2.20.2-desktop.1

Docker Environment

Client:
 Version:    24.0.5
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-buildx.exe
  compose: Docker Compose (Docker Inc.)
    Version:  v2.20.2-desktop.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-compose.exe
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-dev.exe
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     C:\Program Files\Docker\cli-plugins\docker-extension.exe
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.6
    Path:     C:\Program Files\Docker\cli-plugins\docker-init.exe
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-sbom.exe
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scan.exe
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  0.20.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server:
 Containers: 38
  Running: 19
  Paused: 0
  Stopped: 19
 Images: 22
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
 Kernel Version: 5.10.102.1-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 7.636GiB
 Name: docker-desktop
 ID: 3d92caec-cd9c-45d4-9bad-c2e0a8287271
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
WARNING: daemon is not using the default seccomp profile

Anything else?

No response

[rltas]

Using env_file with compose is the equivalent of using --env-file with docker. As such it makes environment variables available in running containers. It's NOT the same as having .env and afaik there's no way to specify an alternative to .env. The variables are neither interpolated in compose.yaml nor available at build stage. The docs aren't wrong per se, but an additional warning about how it works would imo be justified considering how many people miss that (myself included). https://docs.docker.com/compose/compose-file/05-services/#env_file

[839998664]

What I was looking for was a way to pass build time params into the service dockerfiles ... Found an excellent article clearing the air and setting things straight, in my head at least, at --> https://vsupalov.com/docker-arg-env-variable-guide/

[ndeloof]

... a way to pass build time params into the service dockerfiles

You should use build args for this purpose

build: 
   context: .
   args:
      - JENKINS_ENV
      - JENKINS_THEME

value will be set by environment variables available in your environment or .env file, and you also can pass --env-file ./.env.container to get them loaded from your existing env file. Unfortunately there's no args-file support you could use here.

CASC_JENKINS_CONFIG nice to see my baby in use here :)

[rltas]

Which makes it even more confusing that compose's env_file and --env-file and docker's --env-file aren't the same.

[ndeloof]

yes, indeed. But that's our legacy

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.