[BUG] Renew Anon Volumes doesn't work as expected compared to v1

[ryanneufeld]

Description

-V, --renew-anon-volumes Recreate anonymous volumes instead of retrieving data from the previous containers.

This functions differently than in V1

Steps To Reproduce

With docker-compose v1 it was possible to run docker-compose up -dV and have all the containers restarted with fresh anonymous volumes.

Currently this no longer works expected and up simply updates any containers with new versions fetched via pull

version: "3"
services:
  mysql:
    restart: unless-stopped
    image: mysql:5.7
    ports:
      - "3306:3306"
    expose:
      - "3306"
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: "yes"

Compose Version

Docker Compose version v2.18.1

Docker Environment

MacOS Ventura 13.4.1 (22F82)

Client:
 Version:    24.0.2
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.5
    Path:     /Users/rneufeld/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.18.1
    Path:     /Users/rneufeld/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/rneufeld/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.19
    Path:     /Users/rneufeld/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.4
    Path:     /Users/rneufeld/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/rneufeld/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/rneufeld/.docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  v0.12.0
    Path:     /Users/rneufeld/.docker/cli-plugins/docker-scout

Server:
 Containers: 19
  Running: 1
  Paused: 0
  Stopped: 18
 Images: 17
 Server Version: 24.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.49-linuxkit-pr
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 5.805GiB
 Name: docker-desktop
 ID: a21ae72f-a439-4ca4-b8f2-4a3afa43de1e
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

The example use-case I have is a mysql container that comes up and populates the database with pre-installed SQL. With docker-compose v1, I was able to run docker-compose up -dV and the mysql container would be restarted with a fresh anon volume, and the DB would be imported freshly. With V2 I must down the pool to achieve the same outcome.

[dolzenko]

I'm seeing the similar issue

Client: Docker Engine - Community Version: 24.0.5 Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.11.2 Path: /usr/libexec/docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: v2.20.2 Path: /home/ev/.docker/cli-plugins/docker-compose scan: Docker Scan (Docker Inc.) Version: v0.23.0 Path: /usr/libexec/docker/cli-plugins/docker-scan

Server: Containers: 1 Running: 0 Paused: 0 Stopped: 1 Images: 285 Server Version: 24.0.5 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Using metacopy: false Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8 runc version: v1.1.7-0-g860f061 init version: de40ad0 Security Options: apparmor seccomp Profile: builtin cgroupns Kernel Version: 6.2.0-27-generic Operating System: Ubuntu 23.04 OSType: linux Architecture: x86_64 CPUs: 16 Total Memory: 15.32GiB Name: katana ID: eb11b5d1-b704-4e65-a495-26c4ac1d77f5 Docker Root Dir: /var/lib/docker Debug Mode: false Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false

Docker Compose version v2.20.2

[ndeloof]

That's indeed not a strict backward compatibility here, but with Docker Compose v2 in the wild for some time I'd prefer we don't change this behavior now

As a simple workaround, you can use --force-recreate to get the desired behavior:

$ docker compose up -d
[+] Running 2/2
 ✔ Network chose_default    Created                                        0.0s 
 ✔ Container chose-mysql-1  Started                                        0.5s 

$ docker inspect chose-mysql-1 --format '{{.Mounts}}'
[{volume 624433dd888107e5259f01ebb29c9ce74166c96604690713ebfd3f203e3369a9 /var/lib/docker/volumes/624433dd888107e5259f01ebb29c9ce74166c96604690713ebfd3f203e3369a9/_data /var/lib/mysql local  true }]

$ docker compose up --force-recreate -d
[+] Running 1/1
 ✔ Container chose-mysql-1  Started                                        0.9s 

 $ docker inspect chose-mysql-1 --format '{{.Mounts}}'
[{volume 624433dd888107e5259f01ebb29c9ce74166c96604690713ebfd3f203e3369a9 /var/lib/docker/volumes/624433dd888107e5259f01ebb29c9ce74166c96604690713ebfd3f203e3369a9/_data /var/lib/mysql local z true }]
# Same volume has been (re)used with recreated container

$  docker compose up --force-recreate -dV
[+] Running 1/1
 ✔ Container chose-mysql-1  Started                                        1.6s 

$ docker inspect chose-mysql-1 --format '{{.Mounts}}'
[{volume 0e49be586b09dbfde54782822bdd529af9b988d012eecbf3944820e7b9291d8b /var/lib/docker/volumes/0e49be586b09dbfde54782822bdd529af9b988d012eecbf3944820e7b9291d8b/_data /var/lib/mysql local  true }]
 # A fresh new volume has been created while re-creating container