[WSL2] Fail to mount file/directory on windows startup (trying to start container too early ?)

[Tristan-MyAnaPro]

Description

Hi,

I encounter an error each time docker start on logon where it fail to mount bind-mounts volumes. Stopping the containers and restarting them fix the problem, so this is not a problem from the files themselves.

The stack is as follow:

• Docker for windows
• WSL2
  • wsl --version:

    •   WSL version: 2.0.14.0
        Kernel version: 5.15.133.1-1
        WSLg version: 1.0.59
        MSRDC version: 1.2.4677
        Direct3D version: 1.611.1-81528511
        DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
        Windows version: 10.0.22631.3155
      

• All files are inside the WSL2 env
• Compose stack with bind mounts as follow

  • volumes:
      - ../:/var/www/site
    

• All containers have restart: unless-stopped so they start when docker start

It happen to me and my colleague BUT it wasn't happening to him with is older computer which was way less powerful... It seems like to reproduce you need a powerful computer, as if docker was starting "too fast" and the mount in /mnt/wsl is not totally ready.

My computer is the MSI Stealth 15M B12UE-042 for spec references.

I don't really know which log to look at to debug, I'm creating this issue with the hope to have maintainers / community help me reduce the scope until we can find a consistent way to reproduce.

Reproduce

This steps give me a 100% reproduction rate on my computer !

Have docker configured to start on windows logon.

In WSL2, execute:

• mkdir -p ~/docker-win-issue && cd ~/docker-win-issue \
  	&& echo "test" > test-file \
  	&& mkdir test-directory && echo "test" > test-directory/test-file-inside-directory \
  	&& cat > docker-compose.yml << EOF
  version: '3.7'
  services:
    apache:
      image: httpd:latest
      volumes:
        - ./test-directory:/var/www/test-directory
        - ./test-file:/var/www/test-file
      restart: unless-stopped
  EOF
  

• Start the stack: docker compose up -d
• Without shutting down the container, docker or wsl, shutdown the computer (classic end of workday, I don't want to have to shutdown my 7 compose stacks and docker and wsl.....)
• Boot up and login
• After a bit, docker start
• If the mounting of the file fail, we got an error like "Error": "failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting \"/run/desktop/mnt/host/wsl/docker-desktop-bind-mounts/Ubuntu/19f14e3a7e077143cb6a2752001de60bd840e54322e03be340183c88e743959a\" to rootfs at \"/var/www/test-file\": mount /run/desktop/mnt/host/wsl/docker-desktop-bind-mounts/Ubuntu/19f14e3a7e077143cb6a2752001de60bd840e54322e03be340183c88e743959a:/var/www/test-file (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type",
• If the mounting of a directory fail, we got no error but the directory inside the container is empty

If I check the aforementioned file on the filesystem, it's correctly here with the right content:

Expected behavior

The container start and the bind-mounts are properly mounted.

docker version

Client:
 Cloud integration: v1.0.35+desktop.11
 Version:           25.0.3
 API version:       1.44
 Go version:        go1.21.6
 Git commit:        4debf41
 Built:             Tue Feb  6 21:13:00 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Desktop
 Engine:
  Version:          25.0.3
  API version:      1.44 (minimum version 1.24)
  Go version:       go1.21.6
  Git commit:       f417435
  Built:            Tue Feb  6 21:14:25 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    25.0.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1-desktop.4
    Path:     /usr/local/lib/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.6-desktop.1
    Path:     /usr/local/lib/docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container. (Docker Inc.)
    Version:  0.0.24
    Path:     /usr/local/lib/docker/cli-plugins/docker-debug
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /usr/local/lib/docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.22
    Path:     /usr/local/lib/docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.4
    Path:     /usr/local/lib/docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.0.1
    Path:     /usr/local/lib/docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /usr/local/lib/docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.5.0
    Path:     /usr/local/lib/docker/cli-plugins/docker-scout
WARNING: Plugin "/usr/local/lib/docker/cli-plugins/docker-scan" is not valid: failed to fetch metadata: fork/exec /usr/local/lib/docker/cli-plugins/docker-scan: no such file or directory

Server:
 Containers: 19
  Running: 14
  Paused: 0
  Stopped: 5
 Images: 71
 Server Version: 25.0.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 5.15.133.1-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 20
 Total Memory: 15.48GiB
 Name: docker-desktop
 ID: bae87a06-003d-4712-9668-e1d65f4eda05
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

687632F8-FD33-48ED-8E81-B7A273C2A133/20240303145659

Additional Info

I tested and reproduced the problem even with only one stack starting, itself containing only one container

[Tristan-MyAnaPro]

Updated to 4.29.0, problem is still here

[CharlesChou73]

I found an issue committed in 2017 with the same problem #584 . Someone official explained that host-mount volumes are not available in a few seconds after docker starts. It is tracked bu not fixed yet, so I think they will never fix it.

An alternative solution is to give up docker's autostart funtion, and use windows task scheduler instead.

Create a task to start your containers with WSL console after logging on, remember to set a delay to ensure docker is started completely before the task triggered.

Here is a command for starting all containers: wsl "docker" container list -aq | wsl "xargs" docker container start

[Tristan-MyAnaPro]

@CharlesChou73 Well, I came up with a similar solution some weeks ago... Dirty but working for me: In WSL /etc/wsl.conf file, I added a "command" to be run when WSL start which point to a simple script:

#!/bin/bash

while true; do
    if [ "$(docker info -f "{{.OSType}}" 2> /dev/null)" ]; then
        echo "Docker running"
        break
    fi
    sleep 1
done

echo "Restarting"
for service in $(docker compose ls --format=json | jq -rc ".[] .ConfigFiles"); do
    docker compose --file "$service" restart &
done

A while true loop wait for docker to be running and available inside WSL and then restart all docker compose stacks. I found that having the docker command responding a 0 exit code is enough for it to be ready to properly mount volumes.

It's hacky as fuck, but working for me ^^

[CharlesChou73]

@Tristan-MyAnaPro That's instructive! I think your solution is more elegant ヾ(≧▽≦*)o

[lazebird]

FYI, i just downgrade docker desktop to 4.24.2, and the bug gone, refer to https://github.com/docker/for-win/issues/13985

[CharlesChou73]

FYI, i just downgrade docker desktop to 4.24.2, and the bug gone, refer to #13985

Downgraded to 4.24.2, and it works well, thanks a lot!