Nginx Multi-Stage Dockerfile Fails to Build due to Missing Files from Builder Stage

[SC-CTS]

Actual behavior Building the following Dockerfile: https://github.com/nginxinc/docker-nginx/blob/master/modules/Dockerfile.alpine (and https://github.com/nginxinc/docker-nginx/blob/master/modules/Dockerfile) does not work without specifying --ignore-path=/tmp/packages/.

It fails with the following error:

+ BUILT_MODULES=' headers-more'
+ echo 'BUILT_MODULES=" headers-more"'
INFO[0026] Taking snapshot of full filesystem...
INFO[0030] Deleting filesystem...
INFO[0030] Retrieving image manifest nginx:mainline-alpine
INFO[0030] Returning cached image manifest
INFO[0030] Executing 0 build triggers
INFO[0030] Building stage 'nginx:mainline-alpine' [idx: '1', base-idx: '-1']
INFO[0030] Unpacking rootfs as cmd RUN --mount=type=bind,target=/tmp/packages/,source=/tmp/packages/,from=builder     . /tmp/packages/modules.env     && for module in $BUILT_MODULES; do            apk add --no-cache --allow-untrusted /tmp/packages/nginx-module-${module}-${NGINX_VERSION}*.apk;        done requires it.
INFO[0033] RUN --mount=type=bind,target=/tmp/packages/,source=/tmp/packages/,from=builder     . /tmp/packages/modules.env     && for module in $BUILT_MODULES; do            apk add --no-cache --allow-untrusted /tmp/packages/nginx-module-${module}-${NGINX_VERSION}*.apk;        done
INFO[0033] Initializing snapshotter ...
INFO[0033] Taking snapshot of full filesystem...
INFO[0033] Cmd: /bin/sh
INFO[0033] Args: [-c . /tmp/packages/modules.env     && for module in $BUILT_MODULES; do            apk add --no-cache --allow-untrusted /tmp/packages/nginx-module-${module}-${NGINX_VERSION}*.apk;        done]
INFO[0033] Running: [/bin/sh -c . /tmp/packages/modules.env     && for module in $BUILT_MODULES; do            apk add --no-cache --allow-untrusted /tmp/packages/nginx-module-${module}-${NGINX_VERSION}*.apk;        done]
/bin/sh: .: line 0: can't open '/tmp/packages/modules.env': No such file or directory
error building image: error building stage: failed to execute command: waiting for process to exit: exit status 2

Full Log: log.txt

Expected behavior The build should succeed without specifying --ignore-path=/tmp/packages/.

To Reproduce Steps to reproduce the behavior:

1. docker run -v .:/workspace gcr.io/kaniko-project/executor:v1.20.1 --dockerfile /workspace/Dockerfile.alpine --context dir:///workspace/ --build-arg ENABLED_MODULES="headers-more" --no-push --tar-path=/workspace/img.tar
2. Wait until the error happens
3. Add --ignore-path=/tmp/packages/
4. Build again and it will succeed

Additional Information

• Dockerfile

ARG NGINX_FROM_IMAGE=nginx:mainline-alpine
FROM ${NGINX_FROM_IMAGE} as builder

ARG ENABLED_MODULES

SHELL ["/bin/ash", "-exo", "pipefail", "-c"]

RUN if [ "$ENABLED_MODULES" = "" ]; then \
        echo "No additional modules enabled, exiting"; \
        exit 1; \
    fi

COPY ./ /modules/

RUN apk update \
    && apk add linux-headers openssl-dev pcre2-dev zlib-dev openssl abuild \
               musl-dev libxslt libxml2-utils make mercurial gcc unzip git \
               xz g++ coreutils \
    # allow abuild as a root user \
    && printf "#!/bin/sh\\nSETFATTR=true /usr/bin/abuild -F \"\$@\"\\n" > /usr/local/bin/abuild \
    && chmod +x /usr/local/bin/abuild \
    && hg clone -r ${NGINX_VERSION}-${PKG_RELEASE} https://hg.nginx.org/pkg-oss/ \
    && cd pkg-oss \
    && mkdir /tmp/packages \
    && for module in $ENABLED_MODULES; do \
        echo "Building $module for nginx-$NGINX_VERSION"; \
        if [ -d /modules/$module ]; then \
            echo "Building $module from user-supplied sources"; \
            # check if module sources file is there and not empty
            if [ ! -s /modules/$module/source ]; then \
                echo "No source file for $module in modules/$module/source, exiting"; \
                exit 1; \
            fi; \
            # some modules require build dependencies
            if [ -f /modules/$module/build-deps ]; then \
                echo "Installing $module build dependencies"; \
                apk update && apk add $(cat /modules/$module/build-deps | xargs); \
            fi; \
            # if a module has a build dependency that is not in a distro, provide a
            # shell script to fetch/build/install those
            # note that shared libraries produced as a result of this script will
            # not be copied from the builder image to the main one so build static
            if [ -x /modules/$module/prebuild ]; then \
                echo "Running prebuild script for $module"; \
                /modules/$module/prebuild; \
            fi; \
            /pkg-oss/build_module.sh -v $NGINX_VERSION -f -y -o /tmp/packages -n $module $(cat /modules/$module/source); \
            BUILT_MODULES="$BUILT_MODULES $(echo $module | tr '[A-Z]' '[a-z]' | tr -d '[/_\-\.\t ]')"; \
        elif make -C /pkg-oss/alpine list | grep -E "^$module\s+\d+" > /dev/null; then \
            echo "Building $module from pkg-oss sources"; \
            cd /pkg-oss/alpine; \
            make abuild-module-$module BASE_VERSION=$NGINX_VERSION NGINX_VERSION=$NGINX_VERSION; \
            apk add $(. ./abuild-module-$module/APKBUILD; echo $makedepends;); \
            make module-$module BASE_VERSION=$NGINX_VERSION NGINX_VERSION=$NGINX_VERSION; \
            find ~/packages -type f -name "*.apk" -exec mv -v {} /tmp/packages/ \;; \
            BUILT_MODULES="$BUILT_MODULES $module"; \
        else \
            echo "Don't know how to build $module module, exiting"; \
            exit 1; \
        fi; \
    done \
    && echo "BUILT_MODULES=\"$BUILT_MODULES\"" > /tmp/packages/modules.env

FROM ${NGINX_FROM_IMAGE}
RUN --mount=type=bind,target=/tmp/packages/,source=/tmp/packages/,from=builder \
    . /tmp/packages/modules.env \
    && for module in $BUILT_MODULES; do \
           apk add --no-cache --allow-untrusted /tmp/packages/nginx-module-${module}-${NGINX_VERSION}*.apk; \
       done

• Build Context Only https://github.com/nginxinc/docker-nginx/blob/master/modules/Dockerfile.alpine or https://github.com/nginxinc/docker-nginx/blob/master/modules/Dockerfile are required
• Kaniko Image (fully qualified with digest) gcr.io/kaniko-project/executor:v1.20.1

Triage Notes for the Maintainers

Description	Yes/No
Please check if this a new feature you are proposing	- [ ]
Please check if the build works in docker but not in kaniko	- [Yes]
Please check if this error is seen when you use --cache flag	- [?]
Please check if your dockerfile is a multistage dockerfile	- [Yes]