moby
COPY multiple files with wildcard should fail when destination is a file
Description
Dockerfile reference for COPY states: If multiple <src> resources are specified, either directly or due to the use of a wildcard, then <dest> must be a directory, and it must end with a slash /.
I have seen this work correctly in the past, but recently I noticed it is no longer failing as it should
Reproduce
echo test1 > test1.txt echo test2 > test2.txt
Dockerfile:
FROM scratch
COPY *.txt test.txt
docker build .
Expected behavior
Docker build should fail because <src> (*.txt) resolves to multiple files but <dest> (test.txt) is not a directory. Instead, the build succeeds and a single file is copied (for me, it is test2.txt).
docker version
Client: Docker Engine - Community
Version: 24.0.7
API version: 1.43
Go version: go1.20.10
Git commit: afdd53b
Built: Thu Oct 26 09:08:17 2023
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 24.0.7
API version: 1.43 (minimum version 1.12)
Go version: go1.20.10
Git commit: 311b9ff
Built: Thu Oct 26 09:08:17 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.26
GitCommit: 3dd1e886e55dd695541fdcd67420c2888645a495
runc:
Version: 1.1.10
GitCommit: v1.1.10-0-g18a0cb0
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker info
Client: Docker Engine - Community
Version: 24.0.7
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.11.2
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.21.0
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 3
Running: 0
Paused: 0
Stopped: 3
Images: 3
Server Version: 24.0.7
Storage Driver: overlay2
Backing Filesystem: btrfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: false
userxattr: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2
Default Runtime: runc
Init Binary: docker-init
containerd version: 3dd1e886e55dd695541fdcd67420c2888645a495
runc version: v1.1.10-0-g18a0cb0
init version: de40ad0
Security Options:
seccomp
Profile: builtin
Kernel Version: 6.1.60-08594-g03a802b9a072
Operating System: Debian GNU/Linux 11 (bullseye)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 6.485GiB
Name: penguin
ID: de47c6d0-1e57-42b6-9fce-995fa823c673
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Additional Info
No response
Thanks for reporting! I was able to reproduce this, using the following steps;
mkdir -p repro-47064 && cd repro-47064
echo 'hello' > hello.txt
echo 'world' > world.txt
cat > Dockerfile <<'EOF'
# syntax=docker/dockerfile:1
FROM alpine
COPY *.txt test.txt
RUN cat test.txt
EOF
Run the build with BuildKit enabled (default). I use "progress=plain" so that the output of the shell commands shows up;
docker build --no-cache --progress=plain .
# ...
moby/moby#10 [2/3] COPY *.txt test.txt
moby/moby#10 DONE 0.0s
moby/moby#11 [3/3] RUN cat test.txt
moby/moby#11 0.105 world
moby/moby#11 DONE 0.1s
It looks like BuildKit considers COPY *.txt test.txt here to be the equivalent of;
COPY hello.txt test.txt
COPY world.txt test.txt
And the result is the result of the last COPY (test.txt contains the content of world.txt ("world"))
Here's the same with BuildKit disabled (DOCKER_BUILDKIT=0), which produces an error;
DOCKER_BUILDKIT=0 docker build --no-cache .
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
BuildKit is currently disabled; enable it by removing the DOCKER_BUILDKIT=0
environment-variable.
Sending build context to Docker daemon 4.096kB
Step 1/3 : FROM alpine
---> 34871e729050
Step 2/3 : COPY *.txt test.txt
When using COPY with more than one source file, the destination must be a directory and end with a /
Out of curiosity, I also made a comparison with a shell; here's what a shell produces when doing the equivalent;
# syntax=docker/dockerfile:1
FROM alpine
WORKDIR /repro-47064
RUN echo 'hello' > hello.txt \
&& echo 'world' > world.txt
RUN cp *.txt test.txt
RUN cat test.txt
docker build --no-cache --progress=plain .
# ...
moby/moby#8 [3/5] RUN echo 'hello' > hello.txt && echo 'world' > world.txt
moby/moby#8 DONE 0.1s
moby/moby#9 [4/5] RUN cp *.txt test.txt
moby/moby#9 0.259 cp: can't create 'test.txt/hello.txt': No such file or directory
moby/moby#9 0.259 cp: can't create 'test.txt/world.txt': No such file or directory
moby/moby#9 ERROR: process "/bin/sh -c cp *.txt test.txt" did not complete successfully: exit code: 1
------
> [4/5] RUN cp *.txt test.txt:
0.259 cp: can't create 'test.txt/hello.txt': No such file or directory
0.259 cp: can't create 'test.txt/world.txt': No such file or directory
------
Dockerfile:7
--------------------
5 | RUN echo 'hello' > hello.txt \
6 | && echo 'world' > world.txt
7 | >>> RUN cp *.txt test.txt
8 | RUN cat test.txt
9 |
--------------------
ERROR: failed to solve: process "/bin/sh -c cp *.txt test.txt" did not complete successfully: exit code: 1
So it looks like this is a bug / change in behavior in BuildKit; let me transfer this ticket to the Buildkit issue tracker
Description
Dockerfile reference for COPY states:
If multiple <src> resources are specified, either directly or due to the use of a wildcard, then <dest> must be a directory, and it must end with a slash /.I have seen this work correctly in the past, but recently I noticed it is no longer failing as it should
Reproduce
echo test1 > test1.txt echo test2 > test2.txt
Dockerfile:
FROM scratch COPY *.txt test.txtdocker build .
Expected behavior
Docker build should fail because
<src>(*.txt) resolves to multiple files but<dest>(test.txt) is not a directory. Instead, the build succeeds and a single file is copied (for me, it is test2.txt).docker version
Client: Docker Engine - Community Version: 24.0.7 API version: 1.43 Go version: go1.20.10 Git commit: afdd53b Built: Thu Oct 26 09:08:17 2023 OS/Arch: linux/amd64 Context: default Server: Docker Engine - Community Engine: Version: 24.0.7 API version: 1.43 (minimum version 1.12) Go version: go1.20.10 Git commit: 311b9ff Built: Thu Oct 26 09:08:17 2023 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.6.26 GitCommit: 3dd1e886e55dd695541fdcd67420c2888645a495 runc: Version: 1.1.10 GitCommit: v1.1.10-0-g18a0cb0 docker-init: Version: 0.19.0 GitCommit: de40ad0docker info
Client: Docker Engine - Community Version: 24.0.7 Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.11.2 Path: /usr/libexec/docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: v2.21.0 Path: /usr/libexec/docker/cli-plugins/docker-compose Server: Containers: 3 Running: 0 Paused: 0 Stopped: 3 Images: 3 Server Version: 24.0.7 Storage Driver: overlay2 Backing Filesystem: btrfs Supports d_type: true Using metacopy: false Native Overlay Diff: false userxattr: true Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 1 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc io.containerd.runc.v2 Default Runtime: runc Init Binary: docker-init containerd version: 3dd1e886e55dd695541fdcd67420c2888645a495 runc version: v1.1.10-0-g18a0cb0 init version: de40ad0 Security Options: seccomp Profile: builtin Kernel Version: 6.1.60-08594-g03a802b9a072 Operating System: Debian GNU/Linux 11 (bullseye) OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 6.485GiB Name: penguin ID: de47c6d0-1e57-42b6-9fce-995fa823c673 Docker Root Dir: /var/lib/docker Debug Mode: false Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false WARNING: bridge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabledAdditional Info
No response
g
