Can't mount `docker.sock` when using `virtiofs`

[pauldthomson]

trafficstars

Description

Something about the use of virtiofs causes dockerd in the guest to receive an Operation not supported rather than File exists when it performs a mkdir -p (as it always does on a bind mount source directory) and the source is the docker socket.

Version

❯ colima version && limactl --version && qemu-img --version
colima version 0.6.8
git commit: 9b0809d0ed9ad3ff1e57c405f27324e6298ca04f

runtime: docker
arch: aarch64
client: v25.0.3
server: v24.0.9
limactl version 0.20.1
qemu-img version 8.2.1
Copyright (c) 2003-2023 Fabrice Bellard and the QEMU Project developers

Operating System

• [ ] macOS Intel <= 13 (Ventura)
• [ ] macOS Intel >= 14 (Sonoma)
• [ ] Apple Silicon <= 13 (Ventura)
• [X] Apple Silicon >= 14 (Sonoma)
• [ ] Linux

Output of colima status

❯ colima status
INFO[0000] colima is running using macOS Virtualization.Framework
INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: sshfs
INFO[0000] socket: unix:///Users/pthomson/.colima/default/docker.sock

Reproduction Steps

1. Start Colima using virtiofs (default when using vz over qemu: colima start -t vz
2. Either:
   • docker run --rm -it -v "${HOME}/.colima/default/docker.sock":/var/run/docker.sock ubuntu; or
   • colima ssh then mkdir -p /Users/{username}/.colima/default/docker.sock and see Operation not supported
3. Stop colima, then start with colima start -t vz --mount-type sshfs
4. Repeat 2 and see that it "works" (and error for mkdir -p is File exists (which is swallowed by dockerd/moby)

Expected behaviour

Socket is mounted

Additional context

This is not a Colima problem per se, but rather something about virtiofs which is well out of my wheelhouse. I only raise it here as it seems that Docker Desktop has fixed the issue somehow, indicating that there is something that can be done. Whilst the workaround is just to use sshfs, it seems that virtiofs is much more performant. I've exhausted the (small) extent of my expertise in file permissions (and MacOS specifically) to no avail. Trying here in case anyone has ideas. Have also noticed there seems to be a few issues pertaining to permissions and virtiofs.

This comment seems to allude to the problem: https://forums.docker.com/t/permission-for-v-var-run-docker-sock/132976/5, particularly:

It will not work and you will probably get “Unsupported operation”. The reason is that when you mount /var/run/docker.sock you actually mount the socket from the virtual machine, not from the host operating system (in this case macOS). That socket is only for the host and if I am not mistaken, only for compatibility reason. So you can’t mount the socket from the host.

But not enough detail to know what to do...

Interestingly enough, creating a symlink (on the host) from ${HOME}/.colima/default/docker.sock to say, /var/run/docker.sock seems to make it work, i.e. it's mountable and performing a mkdir -p results in the "handled" File exists error. Wondering if this is something specific to MacOS and the home dir?

[sanimalp]

I have this problem as well, with all the same details as the above. I worked around it by going back to using sshfs instead of virtiofs using the following command:

colima start -t vz --mount-type sshfs

as mentioned above.