docker
${VAR?:err} not erroring in docker compose v2 candidate for build args when VAR is null
Description
Using the Docker Compose v2 release candidate (Docker Compose version v2.0.0-rc.1), docker compose commands do not return an error for build args when a docker compose variable is flagged to error if null or empty when there exists another docker compose variable at a higher level.
Steps to reproduce the issue:
- Enable the Docker Compose v2 release candidate (though simply running
docker composeinstead ofdocker-composefromdocker-compose version 1.29.2, build 5becea4cwill also reproduce the error. - In a new project folder create the following files
.env
DOCKER_PORT="4545"
Dockerfile
FROM ubuntu:latest
ARG VAR
RUN echo $VAR
docker-compose.yml
version: "3"
services:
test-service:
build:
args:
VAR: ${VAR?:err}
image: test-service:latest
tty: true
ports:
- ${DOCKER_PORT:?err}:${DOCKER_PORT:?err}
# These will be uncommented in further steps
# - ${DOCKER_PORT}:${DOCKER_PORT}
# - 4545:4545
- Any
docker composecommand should work but the simplest way to reproduce the error is to build the service withdocker compose buildat which point the error should fire becauseVARis not set but instead it will proceed to build the container. - Comment out
- ${DOCKER_PORT:?err}:${DOCKER_PORT:?err}, uncomment- ${DOCKER_PORT}:${DOCKER_PORT} - Rerun
docker compose build. You should see the same behavior. - Comment out
- ${DOCKER_PORT}:${DOCKER_PORT}and uncomment4545:4545 - Rerun
docker compose build - This time it should error with the expected behavior
Invalid template: "required variable VAR is missing a value: :err"
Describe the results you received:
Currently it will allow the command to proceed without throwing an error and exiting. After completing the build running docker compose config confirms that the value for VAR will be null
Describe the results you expected:
Running docker compose build when a Docker Compose variable is null should result in an error that looks something like this:
Invalid template: "required variable VAR is missing a value: :err"
Additional information you deem important (e.g. issue happens only occasionally):
- I did confirm that the error is thrown properly for all of the above situations in
docker-compose version 1.29.2, build 5becea4c. - Also this applies to having a docker compose variable at any level higher than
build->args. For example the following config will result in the same behavior
services:
test-service:
build:
args:
VAR: ${VAR?:err}
image: ${IMAGE-default}
tty: true
ports:
- 4545:4545
Output of docker compose version:
Docker Compose version v2.0.0-rc.1
Output of docker info:
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
compose: Docker Compose (Docker Inc., v2.0.0-rc.1)
scan: Docker Scan (Docker Inc., v0.8.0)
Server:
Containers: 14
Running: 13
Paused: 0
Stopped: 1
Images: 35
Server Version: 20.10.8
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2
Default Runtime: runc
Init Binary: docker-init
containerd version: e25210fe30a0a703442421b0f60afac609f950a3
runc version: v1.0.1-0-g4144b63
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.10.47-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 15.64GiB
Name: docker-desktop
ID: 7A3S:4KBZ:ETMY:2QS4:Q2NU:NNQZ:FHSU:FAZA:PYXW:VCOV:2UP3:JFWH
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Additional environment details:
I actually have a similar error with compose 2.1.0. I suspect it's related.
My project consist of the following files.
Dockerfile
FROM alpine
ARG USERNAME
ARG API_KEY
ENV USERNAME="${USERNAME}"
ENV API_KEY="${API_KEY}"
CMD ["sh", "-c", "echo Username: ${USERNAME:?}; echo Api Key: ${API_KEY:?}"]
docker-compose.yml
services:
helloworld:
image: "helloworld"
build:
dockerfile: "Dockerfile"
args:
USERNAME: "${USERNAME}"
API_KEY: "${API_KEY}"
I get the expected behavior if I build and run this with docker directly:
docker build --build-arg USERNAME=user --build-arg API_KEY=api-key -t helloworld --no-cache . && docker run --rm -it helloworld
Sending build context to Docker daemon 3.072kB
Step 1/6 : FROM alpine
---> 14119a10abf4
Step 2/6 : ARG USERNAME
---> Running in a706e7e5b71a
Removing intermediate container a706e7e5b71a
---> 3b005ff5c826
Step 3/6 : ARG API_KEY
---> Running in f3c482d834a9
Removing intermediate container f3c482d834a9
---> 3231801ad056
Step 4/6 : ENV USERNAME="${USERNAME}"
---> Running in 5ccbe745e870
Removing intermediate container 5ccbe745e870
---> e54fffb578b8
Step 5/6 : ENV API_KEY="${API_KEY}"
---> Running in fc8ddeffa188
Removing intermediate container fc8ddeffa188
---> 977bf207173e
Step 6/6 : CMD ["sh", "-c", "echo Username: ${USERNAME:?}; echo Api Key: ${API_KEY:?}"]
---> Running in 31e86d20454f
Removing intermediate container 31e86d20454f
---> 9520d4541538
Successfully built 9520d4541538
Successfully tagged helloworld:latest
Username: user
Api Key: api-key
With compose 2.0.1, it works but I do get a weird warning about API_KEY not being set, even though it is:
docker compose build --build-arg USERNAME=user --build-arg API_KEY=api-key --no-cache && docker compose up
WARN[0000] The "API_KEY" variable is not set. Defaulting to a blank string.
[+] Building 0.1s (5/5) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 0.0s
=> CACHED [1/1] FROM docker.io/library/alpine 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:4034f022e8f7c7853e729b33cb03f8b54af68a3fbc379d31f707472767805bd7 0.0s
=> => naming to docker.io/library/helloworld 0.0s
WARN[0000] The "API_KEY" variable is not set. Defaulting to a blank string.
[+] Running 2/2
⠿ Network docker_default Created 0.0s
⠿ Container docker-helloworld-1 Created 0.1s
Attaching to docker-helloworld-1
docker-helloworld-1 | Username: user
docker-helloworld-1 | Api Key: api-key
docker-helloworld-1 exited with code 0
And with compose 2.1.0, the same warning is present, but API_KEY ends up being set to the same value than USERNAME:
docker compose build --build-arg USERNAME=user --build-arg API_KEY=api-key --no-cache && docker compose up
WARN[0000] The "API_KEY" variable is not set. Defaulting to a blank string.
Sending build context to Docker daemon 341B
Step 1/6 : FROM alpine
---> 14119a10abf4
Step 2/6 : ARG USERNAME
---> Running in e0303785613a
---> 6051064e83a9
Step 3/6 : ARG API_KEY
---> Running in 908d21513afb
---> d0eeb2ec9d3b
Step 4/6 : ENV USERNAME="${USERNAME}"
---> Running in 3c8059627189
---> e654eece605c
Step 5/6 : ENV API_KEY="${API_KEY}"
---> Running in c7055290391b
---> 1711ddf0a27e
Step 6/6 : CMD ["sh", "-c", "echo Username: ${USERNAME:?}; echo Api Key: ${API_KEY:?}"]
---> Running in b8608c722a71
---> e36d908a54b7
Successfully built e36d908a54b7
Successfully tagged helloworld:latest
WARN[0000] The "API_KEY" variable is not set. Defaulting to a blank string.
[+] Running 2/2
⠿ Network docker_default Created 0.0s
⠿ Container docker-helloworld-1 Created 0.1s
Attaching to docker-helloworld-1
docker-helloworld-1 | Username: user
docker-helloworld-1 | Api Key: user
docker-helloworld-1 exited with code 0
Edit: Forcing the build to use buildkit fixes the issue of both variables being set with the same value, but there is still the warning about API_KEY being not set when it is set.
@Swajalisduf Thank you for the issue.
Looks like your example has a typo.
The syntax for error messages on missing variables is :? and not ?:
But even with this, it looks like the error is shown as expected. Could you please retest with the latest version?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
@mathieu-lemay
as you use API_KEY: "${API_KEY}" in your compose file for build.args, you refer to an actual environment variable, not a build arg, so you should run API_KEY=api-key docker compose build ...
-- build-args flag on docker compose build is passed to all build, even those with no matching args definition in the compose file.
About original issue, I can confirm using the expected :?err syntax works as exepected:
required variable VAR is missing a value: err
