GoogleContainerTools
Regression since v0.19.0 using --cache=true with shell parameter expansion in RUN instruction of Dockerfile
Actual behavior
Starting with debug-v0.19.0 (reproduced with debug-v1.0.0), building using --cache=true, a Dockerfile having RUN instruction with ENV or ARG variable and shell parameter expansion is not working anymore with the following error:
error building image: error building stage: failed to optimize instructions: failed to process "RUN : \"${BINARY_NAME:?Build argument needs to be set and non-empty.}\"": unsupported modifier (?) in substitution
$ docker run --rm --entrypoint "" -v /host/path/to/kaniko/config.json:/kaniko/.docker/config.json -v /host/path/to/dockerfile/directory/kaniko/20200825-001/build1:/workspace gcr.io/kaniko-project/executor:debug-v0.19.0 /kaniko/executor --cache=true --registry-mirror index.docker.io --context /workspace --dockerfile /workspace/Dockerfile --destination index.docker.io/tanguydelignieres/kaniko_bugs_20200925-001_build1:debug-v0.19.0
INFO[0002] Resolved base name alpine:3.9 to alpine:3.9
INFO[0002] Resolved base name alpine:3.9 to alpine:3.9
INFO[0002] Retrieving image manifest alpine:3.9
INFO[0004] Retrieving image manifest alpine:3.9
INFO[0006] Built cross stage deps: map[]
INFO[0006] Retrieving image manifest alpine:3.9
INFO[0008] Retrieving image manifest alpine:3.9
error building image: error building stage: failed to optimize instructions: failed to process "RUN : \"${BINARY_NAME:?Build argument needs to be set and non-empty.}\"": unsupported modifier (?) in substitution
$
It might be related to #1085
With debug-v0.18.0, building using --cache=true, a Dockerfile having RUN instruction with ENV or ARG variable and shell parameter expansion is working:
(Do not pay attention to the --registry-mirror flag, it's only for dockerhub hostname resolution -- see #1209, nor the cache layer push error at the end, they do not prevent test the case or reproduce the issue)
$ docker run --rm --entrypoint "" -v /host/path/to/kaniko/config.json:/kaniko/.docker/config.json -v /host/path/to/dockerfile/directory/kaniko/20200825-001/build1:/workspace gcr.io/kaniko-project/executor:debug-v0.18.0 /kaniko/executor --cache=true --registry-mirror index.docker.io --context /workspace --dockerfile /workspace/Dockerfile --destination index.docker.io/tanguydelignieres/kaniko_bugs_20200925-001_build1:debug-v0.18.0
INFO[0002] Resolved base name alpine:3.9 to alpine:3.9
INFO[0002] Resolved base name alpine:3.9 to alpine:3.9
INFO[0002] Retrieving image manifest alpine:3.9
INFO[0005] Retrieving image manifest alpine:3.9
INFO[0008] Built cross stage deps: map[]
INFO[0008] Retrieving image manifest alpine:3.9
INFO[0010] Retrieving image manifest alpine:3.9
INFO[0011] Checking for cached layer index.docker.io/tanguydelignieres/kaniko_bugs_20200925-001_build1/cache:f76715517b6fb892b2ff84dd41aa7d430b9c14dd18db39815391c649fac302e0...
INFO[0014] No cached layer found for cmd RUN : "${BINARY_NAME:?Build argument needs to be set and non-empty.}"
INFO[0014] Unpacking rootfs as cmd RUN : "${BINARY_NAME:?Build argument needs to be set and non-empty.}" requires it.
INFO[0018] Taking snapshot of full filesystem...
INFO[0018] Resolving paths
INFO[0018] ENV BINARY_NAME=abinaryname
INFO[0018] No files changed in this command, skipping snapshotting.
INFO[0018] RUN : "${BINARY_NAME:?Build argument needs to be set and non-empty.}"
INFO[0018] cmd: /bin/sh
INFO[0018] args: [-c : "${BINARY_NAME:?Build argument needs to be set and non-empty.}"]
INFO[0018] Taking snapshot of full filesystem...
INFO[0018] Resolving paths
INFO[0018] No files were changed, appending empty layer to config. No layer added to image.
INFO[0018] Pushing layer index.docker.io/tanguydelignieres/kaniko_bugs_20200925-001_build1/cache:f76715517b6fb892b2ff84dd41aa7d430b9c14dd18db39815391c649fac302e0 to cache now
WARN[0024] error uploading layer to cache: failed to push to destination index.docker.io/tanguydelignieres/kaniko_bugs_20200925-001_build1/cache:f76715517b6fb892b2ff84dd41aa7d430b9c14dd18db39815391c649fac302e0: unsupported status code 401
$
Expected behavior
With >= debug-v0.19.0 building using --cache=true, a Dockerfile having RUN instruction with ENV or ARG variable and shell parameter expansion should still work as in <= debug-v0.18.0.
To Reproduce
- Use a Dockerfile with a
RUNinstruction withENVorARGvariable and shell parameter expansion - Build using kaniko executor >=
debug-v0.19.0, with--cache=true
Additional Information
- Dockerfile
FROM alpine:3.9
ENV BINARY_NAME=abinaryname
# Check for mandatory build arguments
RUN : "${BINARY_NAME:?Build argument needs to be set and non-empty.}"
- Build Context No specific Build Context
- Kaniko Image (fully qualified with digest)
gcr.io/kaniko-project/executor:debug-v0.18.0 sha256:fee68fd4aacd3b1efea1d57d04426baf14e8961db41c4aa95ba47f7fa8e45fa5gcr.io/kaniko-project/executor:debug-v0.19.0 sha256:0d0e34396f47ec6d5fd75aebb9772147a78d96ed2bbb16ec892bd178efdc8307
Triage Notes for the Maintainers
| Description | Yes/No |
|---|---|
| Please check if this a new feature you are proposing |
|
| Please check if the build works in docker but not in kaniko |
|
Please check if this error is seen when you use --cache flag |
|
| Please check if your dockerfile is a multistage dockerfile |
|
actually we run into this aswell, we use a multilayer image and our build image needs two args (NPMRC credentials and NUGET credentials)
