for-mac Bind Volumes cut data and do not update

Description

Files shared via bind volume cut end of file, when inserting lines via host. Only after doing e.g. ls -la command in the folder of the file it gets updated and shows correct new content.

Tested with images:

ubuntu:latest
alpine:latest

Tested with docker versions:

4.35.0
4.34.3

Also reinstalled docker a couple of times.

Specs: Apple M3 Max 15.0.1 (24A348)

Reproduce

Files

# app/testfile 
some content

# docker-compose.yml
services:
  some-docker:
    container_name: some-docker
    hostname: some-docker
    image: ubuntu:latest # or e.g. alpine:latest
    volumes:
      - some-volume:/opt
    command: tail -f /dev/null
volumes:
  some-volume:
    driver: local
    driver_opts:
      type: bind
      o: bind,rw
      device: ${PWD}/app

Commands and reproduction

docker compose build && docker compose up -d
docker exec -it some-docker watch -n1 cat /opt/testfile

# add lines to the beginning of testfile

# optional: 
docker exec -it some-docker ls -la /opt #while watching testfile

Expected behavior

Proper file rw bind between host and docker container with updates.

docker version

Client: Version: 27.3.1 API version: 1.47 Go version: go1.22.7 Git commit: ce12230 Built: Fri Sep 20 11:38:18 2024 OS/Arch: darwin/arm64 Context: desktop-linux

Server: Docker Desktop 4.35.0 (172550) Engine: Version: 27.3.1 API version: 1.47 (minimum version 1.24) Go version: go1.22.7 Git commit: 41ca978 Built: Fri Sep 20 11:41:19 2024 OS/Arch: linux/arm64 Experimental: false containerd: Version: 1.7.21 GitCommit: 472731909fa34bd7bc9c087e4c27943f9835f111 runc: Version: 1.1.13 GitCommit: v1.1.13-0-g58aa920 docker-init: Version: 0.19.0 GitCommit: de40ad0

AS WELL AS

Client: Version: 27.2.0 API version: 1.47 Go version: go1.21.13 Git commit: 3ab4256 Built: Tue Aug 27 14:14:45 2024 OS/Arch: darwin/arm64 Context: desktop-linux

Server: Docker Desktop 4.34.3 (170107) Engine: Version: 27.2.0 API version: 1.47 (minimum version 1.24) Go version: go1.21.13 Git commit: 3ab5c7d Built: Tue Aug 27 14:15:41 2024 OS/Arch: linux/arm64 Experimental: false containerd: Version: 1.7.20 GitCommit: 8fc6bcff51318944179630522a095cc9dbf9f353 runc: Version: 1.1.13 GitCommit: v1.1.13-0-g58aa920 docker-init: Version: 0.19.0 GitCommit: de40ad0

docker info

Client: Version: 27.3.1 Context: desktop-linux Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.17.1-desktop.1 Path: /Users/christianluis/.docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: v2.29.7-desktop.1 Path: /Users/christianluis/.docker/cli-plugins/docker-compose debug: Get a shell into any image or container (Docker Inc.) Version: 0.0.37 Path: /Users/christianluis/.docker/cli-plugins/docker-debug desktop: Docker Desktop commands (Alpha) (Docker Inc.) Version: v0.0.15 Path: /Users/christianluis/.docker/cli-plugins/docker-desktop dev: Docker Dev Environments (Docker Inc.) Version: v0.1.2 Path: /Users/christianluis/.docker/cli-plugins/docker-dev extension: Manages Docker extensions (Docker Inc.) Version: v0.2.27 Path: /Users/christianluis/.docker/cli-plugins/docker-extension feedback: Provide feedback, right in your terminal! (Docker Inc.) Version: v1.0.5 Path: /Users/christianluis/.docker/cli-plugins/docker-feedback init: Creates Docker-related starter files for your project (Docker Inc.) Version: v1.3.0 Path: /Users/christianluis/.docker/cli-plugins/docker-init sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.) Version: 0.6.0 Path: /Users/christianluis/.docker/cli-plugins/docker-sbom scout: Docker Scout (Docker Inc.) Version: v1.14.0 Path: /Users/christianluis/.docker/cli-plugins/docker-scout

Server: Containers: 5 Running: 3 Paused: 0 Stopped: 2 Images: 6 Server Version: 27.3.1 Storage Driver: overlayfs driver-type: io.containerd.snapshotter.v1 Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: 472731909fa34bd7bc9c087e4c27943f9835f111 runc version: v1.1.13-0-g58aa920 init version: de40ad0 Security Options: seccomp Profile: unconfined cgroupns Kernel Version: 6.10.11-linuxkit Operating System: Docker Desktop OSType: linux Architecture: aarch64 CPUs: 16 Total Memory: 7.653GiB Name: docker-desktop ID: 27fb130f-6042-45a2-830f-5d5ed016df43 Docker Root Dir: /var/lib/docker Debug Mode: false HTTP Proxy: http.docker.internal:3128 HTTPS Proxy: http.docker.internal:3128 No Proxy: hubproxy.docker.internal Labels: com.docker.desktop.address=unix:///Users/christianluis/Library/Containers/com.docker.docker/Data/docker-cli.sock Experimental: false Insecure Registries: hubproxy.docker.internal:5555 127.0.0.0/8 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

AS WELL AS

Client: Version: 27.2.0 Context: desktop-linux Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.16.2-desktop.1 Path: /Users/christianluis/.docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: v2.29.2-desktop.2 Path: /Users/christianluis/.docker/cli-plugins/docker-compose debug: Get a shell into any image or container (Docker Inc.) Version: 0.0.34 Path: /Users/christianluis/.docker/cli-plugins/docker-debug desktop: Docker Desktop commands (Alpha) (Docker Inc.) Version: v0.0.15 Path: /Users/christianluis/.docker/cli-plugins/docker-desktop dev: Docker Dev Environments (Docker Inc.) Version: v0.1.2 Path: /Users/christianluis/.docker/cli-plugins/docker-dev extension: Manages Docker extensions (Docker Inc.) Version: v0.2.25 Path: /Users/christianluis/.docker/cli-plugins/docker-extension feedback: Provide feedback, right in your terminal! (Docker Inc.) Version: v1.0.5 Path: /Users/christianluis/.docker/cli-plugins/docker-feedback init: Creates Docker-related starter files for your project (Docker Inc.) Version: v1.3.0 Path: /Users/christianluis/.docker/cli-plugins/docker-init sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.) Version: 0.6.0 Path: /Users/christianluis/.docker/cli-plugins/docker-sbom scout: Docker Scout (Docker Inc.) Version: v1.13.0 Path: /Users/christianluis/.docker/cli-plugins/docker-scout

Server: Containers: 5 Running: 5 Paused: 0 Stopped: 0 Images: 6 Server Version: 27.2.0 Storage Driver: overlayfs driver-type: io.containerd.snapshotter.v1 Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: 8fc6bcff51318944179630522a095cc9dbf9f353 runc version: v1.1.13-0-g58aa920 init version: de40ad0 Security Options: seccomp Profile: unconfined cgroupns Kernel Version: 6.10.4-linuxkit Operating System: Docker Desktop OSType: linux Architecture: aarch64 CPUs: 16 Total Memory: 7.654GiB Name: docker-desktop ID: 27fb130f-6042-45a2-830f-5d5ed016df43 Docker Root Dir: /var/lib/docker Debug Mode: false HTTP Proxy: http.docker.internal:3128 HTTPS Proxy: http.docker.internal:3128 No Proxy: hubproxy.docker.internal Labels: com.docker.desktop.address=unix:///Users/christianluis/Library/Containers/com.docker.docker/Data/docker-cli.sock Experimental: false Insecure Registries: hubproxy.docker.internal:5555 127.0.0.0/8 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

933FA4A8-92F1-4911-BB17-87A4F6F9D852/20241027140138

Additional Info

No response

Oct 27 '24 14:10 christianluis

Found a solution and did some further testing:

Solution:

# Files are updated correctly.
volumes:
  some-volume:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: ./app

Tests

# Error response from daemon: failed to populate volume
volumes:
  some-volume:
    driver: local
    driver_opts:
      type: none
      o: bind, rw # || o: bind, ro
      device: ./app

# Error response from daemon: failed to populate volume
volumes:
  some-volume:
    driver: local
    driver_opts:
      type: bind
      o: bind, rw # || o: bind, ro
      device: ./app

# Loads OK but has updated error as initially described
volumes:
  some-volume:
    driver: local
    driver_opts:
      type: bind
      o: bind, rw 
      device: ${PWD}/app

Oct 27 '24 15:10 christianluis

Curious; is there a specific reason for your use case to define the bind mount as a volume, instead of a direct bind-mount under the service? e.g.;

services:
  some-docker:
    container_name: some-docker
    hostname: some-docker
    image: ubuntu:latest # or e.g. alpine:latest
    volumes:
      - ./:/opt
    command: tail -f /dev/null

If doing the above, does the issue also reproduce?

(wondering if the extra indirect through defining it as a pseudo-volume causes some of the systems in docker desktop to treat it different)

Oct 27 '24 16:10 thaJeztah

Your example also works fine. There is no specific reason other than the naming of the volumes.

Oct 27 '24 17:10 christianluis

I have the same problem. In my case the mapping of the code is done using a named volume like this:

services:
...
    volumes:
      - source-data:/var/www

I had a hard time figuring out what was happening, because I'm using docker to develop and I got compilation errors and, of course, I was not able to see those errors in the code I was editing in the host. I got a breakthrough when I restarted docker and the compilation errors magically disappeared. Then I realised it wasn't my fault and I found this issue.

I'm using Apple M1 Max and MacOs 15.1 (24B83)

If I change the mapping like @thaJeztah said in the docker-compose.yaml It works fine, but I cannot commit that change because the volume is used in the production environment.

Mar 24 '25 11:03 mariosanroman