symlink in Dockerfile empties target dir when building on remote DOCKER_HOST

[mlechner]

Description

When creating a symlink within a Dockerfile the target dir may be emtied when build is done on a remote DOCKER_HOST

Reproduce

Dockerfile.txt

Use the attached Dockerfile to build the Image once locally once on a remote DOCKER_HOST:

$ docker build --no-cache --pull -t docker_symlinktest .
$ DOCKER_HOST=docker.mydomain.com:2375 docker build --no-cache --pull -t docker_symlinktest .

the output when building in the remote DOCKER_HOST shows an emptied (only the generated symlink exists) folder:

...
Step 5/9 : RUN ls -hal $HTTPD_PREFIX/conf/extra/
 ---> Running in 0348861c4020
total 76K
drwxr-xr-x 2 root root 4.0K Jun 13  2023 .
drwxr-xr-x 4 root root 4.0K Jun 13  2023 ..
-rw-r--r-- 1 root root 2.9K Jun 13  2023 httpd-autoindex.conf
-rw-r--r-- 1 root root 1.8K Jun 13  2023 httpd-dav.conf
-rw-r--r-- 1 root root 2.9K Jun 13  2023 httpd-default.conf
-rw-r--r-- 1 root root 1.1K Jun 13  2023 httpd-info.conf
-rw-r--r-- 1 root root 5.0K Jun 13  2023 httpd-languages.conf
-rw-r--r-- 1 root root 1.4K Jun 13  2023 httpd-manual.conf
-rw-r--r-- 1 root root 4.4K Jun 13  2023 httpd-mpm.conf
-rw-r--r-- 1 root root 2.2K Jun 13  2023 httpd-multilang-errordoc.conf
-rw-r--r-- 1 root root  13K Jun 13  2023 httpd-ssl.conf
-rw-r--r-- 1 root root  694 Jun 13  2023 httpd-userdir.conf
-rw-r--r-- 1 root root 1.5K Jun 13  2023 httpd-vhosts.conf
-rw-r--r-- 1 root root 3.1K Jun 13  2023 proxy-html.conf
Removing intermediate container 0348861c4020
 ---> 46e1421c00a7
Step 6/9 : RUN ln -sf $PWD/custom-vhosts.conf $HTTPD_PREFIX/conf/extra/httpd-vhosts.conf
 ---> Running in 22fa4be83d12
Removing intermediate container 22fa4be83d12
 ---> 28f4bcdf8518
Step 7/9 : RUN echo $HTTPD_PREFIX
 ---> Running in b53f662c24f3
/usr/local/apache2
Removing intermediate container b53f662c24f3
 ---> 06013e2d3ed6
Step 8/9 : RUN ls -hal $HTTPD_PREFIX/conf/extra/
 ---> Running in 08a783165c52
total 8.0K
drwxr-xr-x 1 root root 4.0K Feb  7 17:23 .
drwxr-xr-x 1 root root 4.0K Jun 13  2023 ..
lrwxrwxrwx 1 root root   37 Feb  7 17:23 httpd-vhosts.conf -> /usr/local/apache2/custom-vhosts.conf
Removing intermediate container 08a783165c52
 ---> e6083dd1b566
...

Expected behavior

The directory shouldn't be emptied.

Same bahaviour than on local build is expected.

docker version

identic on local and remote DOCKER_HOST:

Client: Docker Engine - Community
 Version:           19.03.11-ol
 API version:       1.40
 Go version:        go1.16.2
 Git commit:        9bb540d
 Built:             Fri Jul 23 01:33:55 2021
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.11-ol
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.16.2
  Git commit:       9bb540d
  Built:            Fri Jul 23 01:32:08 2021
  OS/Arch:          linux/amd64
  Experimental:     false
  Default Registry: docker.io
 containerd:
  Version:          v1.4.8
  GitCommit:        7eba5930496d9bbe375fdf71603e610ad737d2b2
 runc:
  Version:          1.1.4
  GitCommit:        5fd4c4d
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

docker info

Client:
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc., v2.0.1)

Server:
 Containers: 53
  Running: 46
  Paused: 0
  Stopped: 7
 Images: 1066
 Server Version: 19.03.11-ol
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7eba5930496d9bbe375fdf71603e610ad737d2b2
 runc version: 5fd4c4d
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.1.12-124.73.2.el7uek.x86_64
 Operating System: Oracle Linux Server 7.9
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 62.66GiB
 Name: docker.example.com
 ID: LYTB:YKI2:BHOJ:LMVQ:NS4F:GDUL:7GTF:LCSI:PGSC:UAPR:ON4F:3OYJ
 Docker Root Dir: /data/docker
 Debug Mode: false
 Username: foo
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  ...
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

No response

[mlechner]

Ok. This bug is not symlink specific, but even occurs on a simple "touch":

` FROM httpd:2.4.43 MAINTAINER [email protected]

RUN echo $HTTPD_PREFIX RUN ls -hal $HTTPD_PREFIX/ RUN ls -hal $HTTPD_PREFIX/conf/ RUN ls -hal $HTTPD_PREFIX/conf/extra/ RUN touch $HTTPD_PREFIX/conf/extra/ RUN echo $HTTPD_PREFIX RUN ls -hal $HTTPD_PREFIX/ RUN ls -hal $HTTPD_PREFIX/conf/ RUN ls -hal $HTTPD_PREFIX/conf/extra/

CMD ["httpd-foreground"] `

But it just seems to occur only on httpd: Image and only older ones (buster, bullseye). Not reproducible with httpd:bookworm. So I guess this issue has to closed here :-(

[mlechner]

Closed. See previous comment.

[mlechner]

This behaviour seems to be conected with the used Kernelversion. Machines upgraded from 4.1.12 (I know it's old, but its OEL7/RHEL7) to 4.14.35 build the Images without problem! I could not find any documenttaion/reference for that yet. Therefore I reopen this issue hoping that anyone can give sone reference to documentations of kernel changes, ... pointing out which exact Kernel-Versions may be safe.