Host debian 11 running molecule on Debian 11 (Failed to connect to bus: No such file or directory)

[staticdev]

Issue Type

• Bug report

Molecule and Ansible details

ansible --version && molecule --version
ansible [core 2.13.2]
  config file = None
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/user/.local/lib/python3.9/site-packages/ansible
  ansible collection location = /home/user/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/user/.local/bin/ansible
  python version = 3.9.2 (default, Feb 28 2021, 17:03:44) [GCC 10.2.1 20210110]
  jinja version = 3.1.2
  libyaml = True
molecule 4.0.1 using python 3.10 
    ansible:2.13.2
    delegated:4.0.1 from molecule
    docker:2.0.0 from molecule_docker requiring collections: community.docker>=3.0.0-a2

Molecule installation method (one of):

• pip

Ansible installation method (one of):

• pip

Detail any linters or test runners used: yamllint and ansible-lint

Desired Behavior

I try to run a molecule that works on GA (on Debian 11), it works there on Debian11.

Actual Behaviour

When I try locally it does not work:

Example:

git clone [email protected]:geerlingguy/ansible-role-mysql.git && cd ansible-role-mysql
# change `molecule.yml` image to `geerlingguy/docker-debian11-ansible:latest`
molecule test

I get:

TASK [geerlingguy.mysql : Ensure MySQL is started and enabled on boot.] ********
fatal: [instance]: FAILED! => {"changed": false, "cmd": "/bin/systemctl", "msg": "Failed to connect to bus: No such file or directory", "rc": 1, "stderr": "Failed to connect to bus: No such file or directory\n", "stderr_lines": ["Failed to connect to bus: No such file or directory"], "stdout": "", "stdout_lines": []}

I tried:

1. Running directly the example above
2. Updating molecule.yml command: ${MOLECULE_DOCKER_COMMAND:-""} to /sbin/init. Same result.
3. Changing volumes to /sys/fs/cgroup:/sys/fs/cgroup:rw instead of ro as described here. Same result.
4. Adding to molecule.yml all from the documentation.

platforms:
  - name: instance
    image: geerlingguy/docker-debian11
    command: /sbin/init
    tmpfs:
      - /run
      - /tmp
    volumes:
      - /sys/fs/cgroup:/sys/fs/cgroup:ro

Same result.

Output of 1. with molecule --debug test:

o.txt

[zhan9san]

Hi @staticdev

I cannot re-produce this issue on Ubuntu server. BTW, container with systemd cannot be used on MacOS Docker Desktop.

$ git diff
diff --git a/molecule/default/molecule.yml b/molecule/default/molecule.yml
index 44ed195..ae860b9 100644
--- a/molecule/default/molecule.yml
+++ b/molecule/default/molecule.yml
@@ -6,7 +6,7 @@ driver:
   name: docker
 platforms:
   - name: instance
-    image: "geerlingguy/docker-${MOLECULE_DISTRO:-centos7}-ansible:latest"
+    image: "geerlingguy/docker-debian11-ansible:latest"
     command: ${MOLECULE_DOCKER_COMMAND:-""}
     volumes:
       - /sys/fs/cgroup:/sys/fs/cgroup:ro

$ git remote -v
origin	https://github.com/geerlingguy/ansible-role-mysql.git (fetch)
origin	https://github.com/geerlingguy/ansible-role-mysql.git (push)

TASK [geerlingguy.mysql : Ensure MySQL is started and enabled on boot.] ********
changed: [instance]

Detailed molecule log

$ molecule test
INFO     default scenario test matrix: dependency, lint, cleanup, destroy, syntax, create, prepare, converge, idempotence, side_effect, verify, cleanup, destroy
INFO     Performing prerun with role_name_check=1...
INFO     Set ANSIBLE_LIBRARY=/home/x/.cache/ansible-compat/8b2ffe/modules:/home/x/.ansible/plugins/modules:/usr/share/ansible/plugins/modules
INFO     Set ANSIBLE_COLLECTIONS_PATH=/home/x/.cache/ansible-compat/8b2ffe/collections:/home/x/.ansible/collections:/usr/share/ansible/collections
INFO     Set ANSIBLE_ROLES_PATH=/home/x/.cache/ansible-compat/8b2ffe/roles:/home/x/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles
INFO     Using /home/x/.cache/ansible-compat/8b2ffe/roles/geerlingguy.mysql symlink to current repository in order to enable Ansible to find the role using its expected full name.
INFO     Running default > dependency
WARNING  Skipping, missing the requirements file.
WARNING  Skipping, missing the requirements file.
INFO     Running default > lint
INFO     Lint is disabled.
INFO     Running default > cleanup
WARNING  Skipping, cleanup playbook not configured.
INFO     Running default > destroy
INFO     Sanity checks: 'docker'

PLAY [Destroy] *****************************************************************

TASK [Destroy molecule instance(s)] ********************************************
changed: [localhost] => (item=instance)

TASK [Wait for instance(s) deletion to complete] *******************************
FAILED - RETRYING: [localhost]: Wait for instance(s) deletion to complete (300 retries left).
ok: [localhost] => (item=instance)

TASK [Delete docker networks(s)] ***********************************************

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0

INFO     Running default > syntax

playbook: /mnt/src/github/ansible-role-mysql/molecule/default/converge.yml
INFO     Running default > create

PLAY [Create] ******************************************************************

TASK [Log into a Docker registry] **********************************************
skipping: [localhost] => (item=None)
skipping: [localhost]

TASK [Check presence of custom Dockerfiles] ************************************
ok: [localhost] => (item={'command': '', 'image': 'geerlingguy/docker-debian11-ansible:latest', 'name': 'instance', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']})

TASK [Create Dockerfiles from image names] *************************************
skipping: [localhost] => (item={'command': '', 'image': 'geerlingguy/docker-debian11-ansible:latest', 'name': 'instance', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']})

TASK [Discover local Docker images] ********************************************
ok: [localhost] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': {'command': '', 'image': 'geerlingguy/docker-debian11-ansible:latest', 'name': 'instance', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']}, 'ansible_loop_var': 'item', 'i': 0, 'ansible_index_var': 'i'})

TASK [Build an Ansible compatible image (new)] *********************************
skipping: [localhost] => (item=molecule_local/geerlingguy/docker-debian11-ansible:latest)

TASK [Create docker network(s)] ************************************************

TASK [Determine the CMD directives] ********************************************
ok: [localhost] => (item={'command': '', 'image': 'geerlingguy/docker-debian11-ansible:latest', 'name': 'instance', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']})

TASK [Create molecule instance(s)] *********************************************
changed: [localhost] => (item=instance)

TASK [Wait for instance(s) creation to complete] *******************************
FAILED - RETRYING: [localhost]: Wait for instance(s) creation to complete (300 retries left).
changed: [localhost] => (item={'failed': 0, 'started': 1, 'finished': 0, 'ansible_job_id': '887383259427.3021464', 'results_file': '/home/x/.ansible_async/887383259427.3021464', 'changed': True, 'item': {'command': '', 'image': 'geerlingguy/docker-debian11-ansible:latest', 'name': 'instance', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']}, 'ansible_loop_var': 'item'})

PLAY RECAP *********************************************************************
localhost                  : ok=5    changed=2    unreachable=0    failed=0    skipped=4    rescued=0    ignored=0

INFO     Running default > prepare
WARNING  Skipping, prepare playbook not configured.
INFO     Running default > converge

PLAY [Converge] ****************************************************************

TASK [Gathering Facts] *********************************************************
ok: [instance]

TASK [geerlingguy.mysql : include_tasks] ***************************************
included: /mnt/src/github/ansible-role-mysql/tasks/variables.yml for instance

TASK [geerlingguy.mysql : Include OS-specific variables.] **********************
ok: [instance] => (item=/mnt/src/github/ansible-role-mysql/vars/Debian-11.yml)

TASK [geerlingguy.mysql : Define mysql_packages.] ******************************
ok: [instance]

TASK [geerlingguy.mysql : Define mysql_daemon.] ********************************
ok: [instance]

TASK [geerlingguy.mysql : Define mysql_slow_query_log_file.] *******************
ok: [instance]

TASK [geerlingguy.mysql : Define mysql_log_error.] *****************************
ok: [instance]

TASK [geerlingguy.mysql : Define mysql_syslog_tag.] ****************************
ok: [instance]

TASK [geerlingguy.mysql : Define mysql_pid_file.] ******************************
ok: [instance]

TASK [geerlingguy.mysql : Define mysql_config_file.] ***************************
ok: [instance]

TASK [geerlingguy.mysql : Define mysql_config_include_dir.] ********************
ok: [instance]

TASK [geerlingguy.mysql : Define mysql_socket.] ********************************
ok: [instance]

TASK [geerlingguy.mysql : Define mysql_supports_innodb_large_prefix.] **********
ok: [instance]

TASK [geerlingguy.mysql : include_tasks] ***************************************
skipping: [instance]

TASK [geerlingguy.mysql : include_tasks] ***************************************
included: /mnt/src/github/ansible-role-mysql/tasks/setup-Debian.yml for instance

TASK [geerlingguy.mysql : Check if MySQL is already installed.] ****************
ok: [instance]

TASK [geerlingguy.mysql : Update apt cache if MySQL is not yet installed.] *****
ok: [instance]

TASK [geerlingguy.mysql : Ensure MySQL Python libraries are installed.] ********
changed: [instance]

TASK [geerlingguy.mysql : Ensure MySQL packages are installed.] ****************
changed: [instance]

TASK [geerlingguy.mysql : Ensure MySQL is stopped after initial install.] ******
ok: [instance]

TASK [geerlingguy.mysql : Delete innodb log files created by apt package after initial install.] ***
changed: [instance] => (item=ib_logfile0)
ok: [instance] => (item=ib_logfile1)

TASK [geerlingguy.mysql : include_tasks] ***************************************
skipping: [instance]

TASK [geerlingguy.mysql : Check if MySQL packages were installed.] *************
ok: [instance]

TASK [geerlingguy.mysql : include_tasks] ***************************************
included: /mnt/src/github/ansible-role-mysql/tasks/configure.yml for instance

TASK [geerlingguy.mysql : Get MySQL version.] **********************************
ok: [instance]

TASK [geerlingguy.mysql : Copy my.cnf global MySQL configuration.] *************
changed: [instance]

TASK [geerlingguy.mysql : Verify mysql include directory exists.] **************
skipping: [instance]

TASK [geerlingguy.mysql : Copy my.cnf override files into include directory.] ***

TASK [geerlingguy.mysql : Create slow query log file (if configured).] *********
skipping: [instance]

TASK [geerlingguy.mysql : Create datadir if it does not exist] *****************
ok: [instance]

TASK [geerlingguy.mysql : Set ownership on slow query log file (if configured).] ***
skipping: [instance]

TASK [geerlingguy.mysql : Create error log file (if configured).] **************
skipping: [instance]

TASK [geerlingguy.mysql : Set ownership on error log file (if configured).] ****
skipping: [instance]

TASK [geerlingguy.mysql : Ensure MySQL is started and enabled on boot.] ********
changed: [instance]

TASK [geerlingguy.mysql : include_tasks] ***************************************
fatal: [instance]: FAILED! => {"reason": "couldn't resolve module/action 'mysql_user'. This often indicates a misspelling, missing collection, or incorrect module path.\n\nThe error appears to be in '/mnt/src/github/ansible-role-mysql/tasks/secure-installation.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n---\n- name: Ensure default user is present.\n  ^ here\n"}

RUNNING HANDLER [geerlingguy.mysql : restart mysql] ****************************

PLAY RECAP *********************************************************************
instance                   : ok=26   changed=5    unreachable=0    failed=1    skipped=8    rescued=0    ignored=0

WARNING  Retrying execution failure 2 of: ansible-playbook --inventory /home/x/.cache/molecule/ansible-role-mysql/default/inventory --skip-tags molecule-notest,notest /mnt/src/github/ansible-role-mysql/molecule/default/converge.yml
CRITICAL Ansible return code was 2, command was: ['ansible-playbook', '--inventory', '/home/x/.cache/molecule/ansible-role-mysql/default/inventory', '--skip-tags', 'molecule-notest,notest', '/mnt/src/github/ansible-role-mysql/molecule/default/converge.yml']
WARNING  An error occurred during the test sequence action: 'converge'. Cleaning up.
INFO     Running default > cleanup
WARNING  Skipping, cleanup playbook not configured.
INFO     Running default > destroy

PLAY [Destroy] *****************************************************************

TASK [Destroy molecule instance(s)] ********************************************
changed: [localhost] => (item=instance)

TASK [Wait for instance(s) deletion to complete] *******************************
FAILED - RETRYING: [localhost]: Wait for instance(s) deletion to complete (300 retries left).
changed: [localhost] => (item=instance)

TASK [Delete docker networks(s)] ***********************************************

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=2    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0

INFO     Pruning extra files from scenario ephemeral directory

[staticdev]

@zhan9san looks like it happens only on Debian for some not yet known reason. Could you maybe try in a VM?

[zhan9san]

As you can see, I did test it in a VM(a ubuntu server).

And you can check the diff and I did use debian11

[staticdev]

As you can see, I did test it in a VM(a ubuntu server).

And you can check the diff and I did use debian11

When I said VM, I mean the host (do not use Ubuntu server but Debian). Sorry to confuse you since both host and image must be debian ;)

[zhan9san]

@staticdev

Thanks for you explanation.

It can be reproduced on Debian server.

But we can do nothing about it, let us wait for https://github.com/geerlingguy/docker-debian11-ansible/issues/4

[aussielunix]

This is due to changes in systemd introduced at systemd 248 - notably cgroupsv2 I believe.

I have molecule working fine with debian 10,11 - ubuntu 20.04,22.04 and Rockylinux 9 so far, albeit it still needs privileged mode.

See my https://github.com/geerlingguy/docker-debian11-ansible/issues/4#issuecomment-1225082481 for more details and a solution

[zhan9san]

This issue is addressed in #3700 .

You can use cgroupns_mode: host to work it around in molecule==4.0.3.

Here is an example, https://github.com/ansible-community/molecule/pull/3665#issuecomment-1254979734

Could you verify it?

[geerlingguy]

I can confirm it works, with molecule 4.0.3. Thanks!

[staticdev]

@zhan9san after some test I was able to make it work. For that 3 conditions are REQUIRED:

• Upgrade to molecule 4.0.3
• Add cgroupns_mode: host
• Change /sys/fs/cgroup:/sys/fs/cgroup:ro to /sys/fs/cgroup:/sys/fs/cgroup:rw

@geerlingguy thanks a lot for the support across repos, you are the best!

Closing here.