compose [BUG] Compose-created anonymous volumes are not marked as anonymous

Description

Post-https://github.com/moby/moby/pull/44216, anonymous volumes are annotated with the label com.docker.volume.anonymous, e.g.:

[
    {
        "CreatedAt": "2023-07-20T04:52:35Z",
        "Driver": "local",
        "Labels": {
            "com.docker.volume.anonymous": ""
        },
        "Mountpoint": "/var/lib/docker/volumes/2ec6337464768820a3076200c69c80e4d6d7dc363cfcde6821dc3430ba709450/_data",
        "Name": "2ec6337464768820a3076200c69c80e4d6d7dc363cfcde6821dc3430ba709450",
        "Options": null,
        "Scope": "local"
    }
]

This facilitates the new behavior of not pruning named but unused volumes by default. It appears that anonymous volumes defined at the Compose-file level do not result in an annotated anonymous volume, and thus fail to docker volume prune unless the -a (include named volumes) flag is added.

Steps To Reproduce

services:
  hw:
    image: hello-world
    volumes:
      - /foo/bar

docker inspect the container, and then the anonymous volume at /foo/bar. It will be missing the annotation:

[
    {
        "CreatedAt": "2023-07-20T07:48:38+03:00",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/fd9b04deff814f6def0a8108c42c02aac65e1f7ea6ee0a5193203197ae0d2eea/_data",
        "Name": "fd9b04deff814f6def0a8108c42c02aac65e1f7ea6ee0a5193203197ae0d2eea",
        "Options": null,
        "Scope": "local"
    }
]

Compose Version

Docker Compose version v2.20.0

Docker Environment

Client:
 Version:    24.0.4
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.1
    Path:     /Users/neersighted/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.20.0
    Path:     /Users/neersighted/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/neersighted/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/neersighted/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.6
    Path:     /Users/neersighted/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/neersighted/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/neersighted/.docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  0.16.1
    Path:     /Users/neersighted/.docker/cli-plugins/docker-scout

Server:
 Containers: 4
  Running: 3
  Paused: 0
  Stopped: 1
 Images: 10
 Server Version: 24.0.4-3-gecd494abf3
 Storage Driver: stargz
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.49-linuxkit-pr
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 12
 Total Memory: 11.68GiB
 Name: docker-desktop
 ID: 24db6515-ca45-4274-8dba-a59d6fb3a368
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

This looks possibly similar to https://github.com/moby/moby/pull/45147 (cc @cpuguy83); without knowing how the implementation works however, I can only guess that one of the following is true:

Compose "fakes" anonymous volumes by generating a random name for them, so the engine does not set the annotation.
Compose does not pass a name so the engine is generating one, but somehow there is an edge case between the API and volume creation and the annotation is not being applied.

Originally reported by/credit to @FlowSwift.

Jul 20 '23 11:07 neersighted

I have the same issue on Macbook M1 with the docker:

docker system info
Client:
 Version:    27.0.3
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.15.1-desktop.1
    Path:     /Users/dmitri/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.28.1-desktop.1
    Path:     /Users/dmitri/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.32
    Path:     /Users/dmitri/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.14
    Path:     /Users/dmitri/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/dmitri/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.25
    Path:     /Users/dmitri/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/dmitri/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     /Users/dmitri/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/dmitri/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.10.0
    Path:     /Users/dmitri/.docker/cli-plugins/docker-scout

Every time I run docker compose up the new "unnamed" volume is created. And it cannot be deleted with docker volume prune. Since my projects has the anonymous volumes for node_modules the volume size is big (300MB+) and after a few months the available space goes to 0.

The docker inspect shows that these volumes have no labels!

compose.yaml

services:
  app:
    build:
      context: .
    ports:
      - 8088:8088
    volumes:
      - ./:/usr/src/app
      - /usr/src/app/node_modules

as workaround I can change the volume to named volume with the com.docker.volume.anonymous label.

services:
  app:
    build:
      context: .
    ports:
      - 8088:8088
    volumes:
      - ./:/usr/src/app
      - modules:/usr/src/app/node_modules

volumes:
  modules:
    name: app_node_modules
    labels:
     - com.docker.volume.anonymous

But I don't like that idea.

Jul 04 '24 18:07 Dumk0

Compose converts short syntax - /foo/bar into a volume declaration, which relies on mount API, not bind where this label is automatically added by engine. For some reason, engine does not execute the same logic when using the mount API. You can confirm this running:

$ docker run --mount="type=volume,target=/foo/bar" hello-world
...
$ docker inspect bf0de046413d4d3905c1efc921f5ec75e239e24e2cc8df66191ed62ad97fa648 # anonymous volume created by engine
[
    {
        "CreatedAt": "2024-10-24T15:36:48Z",
        "Driver": "local",
        "Labels": null,

I created https://github.com/moby/moby/issues/48748 for this to be fixed Closing as "not a compose issue"

Oct 24 '24 15:10 ndeloof