`remove_volumes` in docker_compose module doesn't remove volumes if containers have already been removed.

[chrisshroba]

SUMMARY

remove_volumes in docker_compose module doesn't remove volumes if containers have already been removed.

ISSUE TYPE

• Bug Report

COMPONENT NAME

docker_compose

ANSIBLE VERSION

ansible [core 2.12.4]
  config file = /Users/chrisshroba/dev/projects/health-checker/deploy/ansible.cfg
  configured module search path = ['/Users/chrisshroba/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/chrisshroba/.local/pipx/venvs/ansible/lib/python3.10/site-packages/ansible
  ansible collection location = /Users/chrisshroba/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/chrisshroba/.local/bin/ansible
  python version = 3.10.2 (main, Feb  2 2022, 05:51:25) [Clang 13.0.0 (clang-1300.0.29.3)]
  jinja version = 3.1.1
  libyaml = True

COLLECTION VERSION

# /Users/chrisshroba/.local/pipx/venvs/ansible/lib/python3.10/site-packages/ansible_collections
Collection       Version
---------------- -------
community.docker 2.3.0

CONFIGURATION

DEFAULT_HOST_LIST(/Users/chrisshroba/dev/projects/health-checker/deploy/ansible.cfg) = ['/Users/chrisshroba/dev/projects/health-checker/deploy/hosts.yml']
DEFAULT_STDOUT_CALLBACK(/Users/chrisshroba/dev/projects/health-checker/deploy/ansible.cfg) = debug
DEFAULT_VAULT_PASSWORD_FILE(/Users/chrisshroba/dev/projects/health-checker/deploy/ansible.cfg) = /Users/chrisshroba/dev/projects/health-checker/deploy/.redacted

OS / ENVIRONMENT

macOS 12.0 deploying to Ubuntu 20.04 (LTS) x64

STEPS TO REPRODUCE

This playbook completely reproduces the issue:

- name: Example to illustrate issue
  hosts: shrobaserver
  become: yes
  vars:
    deploy_to_dir: "/root/ansible-deployed/example-issue/"
    docker_compose_definition:
      services:
        my_container:
          image: nginx
          volumes:
            - my_volume:/my_volume
      volumes:
        my_volume:
  tasks:
    - name: Create directory to deploy to if not exists
      file:
        path: "{{ deploy_to_dir }}"
        state: directory

    - name: Deploy the service with one volume
      community.docker.docker_compose:
        project_name: "example-issue"
        definition: "{{ docker_compose_definition }}"

    - name: "Confirm that volume exists."
      ansible.builtin.shell:
        cmd: "docker volume ls | grep example-issue_my_volume"

    - name: Bring down the service but don't delete volumes
      community.docker.docker_compose:
        project_name: "example-issue"
        definition: "{{ docker_compose_definition }}"
        state: absent
        remove_orphans: yes
      tags:
        - tagged

    - name: "Confirm that volume still exists."
      ansible.builtin.shell:
        cmd: "docker volume ls | grep example-issue_my_volume"

    - name: Delete the volume too.
      community.docker.docker_compose:
        project_name: "example-issue"
        definition: "{{ docker_compose_definition }}"
        state: absent
        remove_orphans: yes
        remove_volumes: yes

    - name: "Confirm that volume does not exist."
      ansible.builtin.shell:
        cmd: "docker volume ls | grep example-issue_my_volume"
      register: output
      failed_when: "output.failed is false"

EXPECTED RESULTS

When run by ansible-playbook issue.yml, this playbook should complete successfully, because the volume should be correctly removed.

ACTUAL RESULTS

When run by ansible-playbook issue.yml, the playbook fails on the Confirm that volume does not exist. task, unless the Bring down the service but don't delete volumes task is omitted. I tagged that task with the tag tagged to illustrate that the playbook completes successfully when that task is omitted.

Running all tasks

➜  deploy git:(main) ✗ ansible-playbook issue.yml

PLAY [Example to illustrate issue] ********************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************
ok: [shrobaserver]

TASK [Create directory to deploy to if not exists] ****************************************************************************
ok: [shrobaserver]

TASK [Deploy the service with one volume] *************************************************************************************
ok: [shrobaserver]

TASK [Confirm that volume exists.] ********************************************************************************************
changed: [shrobaserver]

TASK [Bring down the service but don't delete volumes] ************************************************************************
changed: [shrobaserver]

TASK [Confirm that volume still exists.] **************************************************************************************
changed: [shrobaserver]

TASK [Delete the volume too.] *************************************************************************************************
ok: [shrobaserver]

TASK [Confirm that volume does not exist.] ************************************************************************************
fatal: [shrobaserver]: FAILED! => {
    "changed": true,
    "cmd": "docker volume ls | grep example-issue_my_volume",
    "delta": "0:00:00.107942",
    "end": "2022-04-07 18:01:15.058116",
    "failed_when_result": true,
    "rc": 0,
    "start": "2022-04-07 18:01:14.950174"
}

STDOUT:

local     example-issue_my_volume

PLAY RECAP ********************************************************************************************************************
shrobaserver               : ok=7    changed=3    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Omitting the Bring down the service but don't delete volumes task:

➜  deploy git:(main) ✗ ansible-playbook issue.yml --skip-tags tagged

PLAY [Example to illustrate issue] ********************************************************************************************

TASK [Create directory to deploy to if not exists] ****************************************************************************
ok: [shrobaserver]

TASK [Deploy the service with one volume] *************************************************************************************
changed: [shrobaserver]

TASK [Confirm that volume exists.] ********************************************************************************************
changed: [shrobaserver]

TASK [Confirm that volume still exists.] **************************************************************************************
changed: [shrobaserver]

TASK [Delete the volume too.] *************************************************************************************************
changed: [shrobaserver]

TASK [Confirm that volume does not exist.] ************************************************************************************
changed: [shrobaserver]

PLAY RECAP ********************************************************************************************************************
shrobaserver               : ok=6    changed=5    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

[felixfontein]

The remove_volumes flag is passed on to docker-compose (down command), but only if there is at least one container that's up. (Not sure what happens if you call down if the service is already down though; also I'm not sure how change detection should cover remove_volumes.)

Maybe it's best to adjust the documentation of remove_volumes so that it accurately documents what remove_volumes actually does?

If someone wants to implement the currently implied behavior, that's of course also possible. Might be a lot harder though. (On the other hand, the Python docker-compose is dead and thus frozen, so any code written now cannot be broken by newer versions of docker-compose...)

[chrisshroba]

I meant to include in my original post that running the equivalent docker-compose commands manually does the right thing: Running docker-compose down -v after docker-compose down does remove the volumes despite there being no running containers.

I haven't had a chance to test this but I think something like this 2 line change might solve the problem by causing docker-compose to run the down command when remove_volumes is specified and volumes still exist:

https://github.com/chrisshroba/community.docker/commit/fbf1f430daad49e2c0a0ef39d1410434597caaf7

[felixfontein]

Ah, that should indeed solve it. (But then I don't use docker-compose, so I'm not very familiar with its code and the module's code.) Do you want to create a PR for this? (If you do, please add a changelog fragment.)