mitogen-hq
Unexpected failure during module execution: mitogen.core.StreamError: cannot unpickle 'ansible.utils.unsafe_proxy'/'AnsibleUnsafeText'
Hello,
I have a strange behavior in one of my playbook following the activation of mitogen, with cascading calls. Here is a simplified example, to reproduce the problem, of my use case :
This is my role "testrole" :
---
- block:
- block:
- name: "Create a workdir"
tempfile:
state: directory
prefix: "repo_tmp"
register: WorkDirectory
run_once: true
delegate_to: 127.0.0.1
always:
- name: "Delete the workdir"
file:
path: "{{ WorkDirectory.path }}"
state: absent
delegate_to: 127.0.0.1
run_once: true
...
This role is called by a first playbook "light.yml" :
---
- hosts: testserver
gather_facts: no
tasks:
- import_role:
name: testrole
...
This one is called by a second playbook "medium.yml" :
---
- import_playbook: light.yml
...
And then this second playbook is called by a third playbook "master.yml" :
---
- import_playbook: "{{ lookup('env','PWD') }}/medium.yml"
...
My inventory looks like :
[testserver]
testserver1 ansible_host=myserver ip=1.1.1.1
With mitogen strategy activated :
- The call of "light.yml" works as expected :
$ ansible-playbook light.yml -i inv.ini
PLAY [testserver] *********************************************************************************************************************************************************************************************
TASK [testrole : Create a workdir] ****************************************************************************************************************************************************************************
jeudi 07 novembre 2024 10:22:55 +0100 (0:00:00.067) 0:00:00.067 ********
changed: [testserver1 -> 127.0.0.1]
TASK [testrole : Delete a workdir] ****************************************************************************************************************************************************************************
jeudi 07 novembre 2024 10:22:56 +0100 (0:00:01.102) 0:00:01.170 ********
changed: [testserver1 -> 127.0.0.1]
PLAY RECAP ****************************************************************************************************************************************************************************************************
testserver1 : ok=2 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
- The call of "medium.yml" works as expected :
$ ansible-playbook medium.yml -i inv.ini
PLAY [testserver] *********************************************************************************************************************************************************************************************
TASK [testrole : Create a workdir] ****************************************************************************************************************************************************************************
jeudi 07 novembre 2024 10:23:26 +0100 (0:00:00.085) 0:00:00.085 ********
changed: [testserver1 -> 127.0.0.1]
TASK [testrole : Delete a workdir] ****************************************************************************************************************************************************************************
jeudi 07 novembre 2024 10:23:28 +0100 (0:00:01.094) 0:00:01.179 ********
changed: [testserver1 -> 127.0.0.1]
PLAY RECAP ****************************************************************************************************************************************************************************************************
testserver1 : ok=2 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
- The call of "master.yml" doesn't works :
$ ansible-playbook master.yml -i inv.ini
PLAY [testserver] *********************************************************************************************************************************************************************************************
TASK [testrole : Create a workdir] ****************************************************************************************************************************************************************************
jeudi 07 novembre 2024 10:24:00 +0100 (0:00:00.069) 0:00:00.069 ********
ERROR! [mux 2916640] 10:24:01.661857 E mitogen.[local.2916677]: raw pickle was: b'\x80\x02(X2\x00\x00\x00lrdevheb1.uem.lan-2916674-7f786f379740-8f5f6663e10q\x00X\x16\x00\x00\x00ansible_mitogen.targetq\x01NX\n\x00\x00\x00run_moduleq\x02)cmitogen.core\nKwargs\nq\x03}q\x04X\x06\x00\x00\x00kwargsq\x05}q\x06(X\x0b\x00\x00\x00runner_nameq\x07X\x0e\x00\x00\x00NewStyleRunnerq\x08X\x06\x00\x00\x00moduleq\tX\x08\x00\x00\x00tempfileq\nX\x04\x00\x00\x00pathq\x0bXC\x00\x00\x00/usr/local/lib/python3.10/site-packages/ansible/modules/tempfile.pyq\x0cX\t\x00\x00\x00json_argsq\rXQ\x02\x00\x00{"state": "directory", "prefix": "repo_tmp", "_ansible_check_mode": false, "_ansible_no_log": false, "_ansible_debug": false, "_ansible_diff": false, "_ansible_verbosity": 0, "_ansible_version": "2.16.10", "_ansible_module_name": "tempfile", "_ansible_syslog_facility": "LOG_USER", "_ansible_selinux_special_fs": ["fuse", "nfs", "vboxsf", "ramfs", "9p", "vfat"], "_ansible_string_conversion_action": "warn", "_ansible_socket": null, "_ansible_shell_executable": "/bin/sh", "_ansible_keep_remote_files": false, "_ansible_tmpdir": null, "_ansible_remote_tmp": "/home/D_NT_UEM/busy/.ansible/tmp"}q\x0eX\x03\x00\x00\x00envq\x0f}q\x10X\x14\x00\x00\x00interpreter_fragmentq\x11NX\t\x00\x00\x00is_pythonq\x12NX\n\x00\x00\x00module_mapq\x13}q\x14(X\x07\x00\x00\x00builtinq\x15]q\x16(X\x1a\x00\x00\x00ansible.module_utils.basicq\x17X\x1b\x00\x00\x00ansible.module_utils.commonq\x18X(\x00\x00\x00ansible.module_utils.common._json_compatq\x19X"\x00\x00\x00ansible.module_utils.common._utilsq\x1aX$\x00\x00\x00ansible.module_utils.common.arg_specq\x1bX\'\x00\x00\x00ansible.module_utils.common.collectionsq\x1cX \x00\x00\x00ansible.module_utils.common.fileq\x1dX"\x00\x00\x00ansible.module_utils.common.localeq\x1eX&\x00\x00\x00ansible.module_utils.common.parametersq\x1fX#\x00\x00\x00ansible.module_utils.common.processq X$\x00\x00\x00ansible.module_utils.common.sys_infoq!X \x00\x00\x00ansible.module_utils.common.textq"X+\x00\x00\x00ansible.module_utils.common.text.convertersq#X+\x00\x00\x00ansible.module_utils.common.text.formattersq$X&\x00\x00\x00ansible.module_utils.common.validationq%X$\x00\x00\x00ansible.module_utils.common.warningsq&X\x1b\x00\x00\x00ansible.module_utils.compatq\'X\'\x00\x00\x00ansible.module_utils.compat._selectors2q(X%\x00\x00\x00ansible.module_utils.compat.selectorsq)X#\x00\x00\x00ansible.module_utils.compat.selinuxq*X\x1b\x00\x00\x00ansible.module_utils.distroq+X#\x00\x00\x00ansible.module_utils.distro._distroq,X\x1b\x00\x00\x00ansible.module_utils.errorsq-X\x1c\x00\x00\x00ansible.module_utils.parsingq.X)\x00\x00\x00ansible.module_utils.parsing.convert_boolq/X\x1f\x00\x00\x00ansible.module_utils.pycompat24q0X\x18\x00\x00\x00ansible.module_utils.sixq1eX\x06\x00\x00\x00customq2]q3uX\x0e\x00\x00\x00py_module_nameq4X\x18\x00\x00\x00ansible.modules.tempfileq5X\r\x00\x00\x00good_temp_dirq6X \x00\x00\x00/home/D_NT_UEM/busy/.ansible/tmpq7X\x03\x00\x00\x00cwdq8cansible.utils.unsafe_proxy\nAnsibleUnsafeText\nq9X1\x00\x00\x00/home/D_NT_UEM/busy/efluid-provisioning/hebergeurq:\x85q;Rq<X\t\x00\x00\x00extra_envq=}q>X\x0b\x00\x00\x00emulate_ttyq?\x88X\x0f\x00\x00\[email protected]\n_unpickle_context\nqAK\x00N\x86qBRqCus\x85qDRqEtqF.'
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: File "<stdin>", line 899, in _find_global
fatal: [testserver1 -> 127.0.0.1]: FAILED! =>
msg: |-
Unexpected failure during module execution: mitogen.core.StreamError: cannot unpickle 'ansible.utils.unsafe_proxy'/'AnsibleUnsafeText'
File "<stdin>", line 3860, in _dispatch_one
File "<stdin>", line 3843, in _parse_request
File "<stdin>", line 998, in unpickle
File "<stdin>", line 789, in find_class
File "<stdin>", line 899, in _find_global
stdout: ''
Do you have any idea what the problem is?
Thanks !
EDIT by @moreati: Formatting and syntax highlighting
PS : If i remove {{ lookup('env','PWD') }} of a master.yml, it works. But why it doesn't works with this ?
Hi dyluem,
You appear to have skipped over the template of standard questions for new issues. They're there to help rule out common dead ends and misunderstandings. Please answer
- Which version of Ansible are you running?
- Is your version of Ansible patched in any way?
- Are you running with any custom modules, or
module_utilsloaded? - Have you tried the latest master version from Git?
- Mention your host and target OS and versions
- Mention your host and target Python versions
- If reporting any kind of problem with Ansible, please include the Ansible version along with output of "ansible-config dump --only-changed".
Additionally
- What happens if you run the failing playbook with vanilla Ansible?
Thanks, Alex
Hi Moreati,
Thanks for your answer :
- Which version of Ansible are you running?
$ ansible --version
ansible [core 2.16.7]
config file = /data/HEB/efluid-provisioning/ansible.cfg
configured module search path = ['/data/HEB/efluid-provisioning/common/ansible/library']
ansible python module location = /usr/local/lib/python3.11/site-packages/ansible
ansible collection location = /home/ansible/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.11.2 (main, Mar 6 2023, 19:02:20) [GCC 8.5.0 20210514 (Red Hat 8.5.0-16)] (/usr/local/bin/python3.11)
jinja version = 3.1.2
libyaml = True
# pip list | grep -wE "ansible |ansible-core "
ansible 9.6.0
ansible-cmdb 1.31
ansible-compat 24.6.1
ansible-core 2.16.7
ansible-inventory-grapher 2.5.0
- Is your version of Ansible patched in any way?
Classic Ansible version, installed with pip
- Are you running with any custom modules, or module_utils loaded?
No, original ansible code.
- Have you tried the latest master version from Git?
To ? Ansible or Mitogen ?
- Mention your host and target OS and versions -Mention your host and target Python versions
Host OS : RHEL 8.10 Target OS : RHEL 8.9 Host Python : 3.10 Target Python : 3.6.8
-What happens if you run the failing playbook with vanilla Ansible?
I don't know what "Vanilla Ansible" is, sorry
I Hope it can help you
Thanks for help,
Dyluem
-What happens if you run the failing playbook with vanilla Ansible?
I don't know what "Vanilla Ansible" is, sorry
Sorry, "vanilla Ansible" is a term I use for Ansible without any Mitogen, e.g. nostrategy: mitogen_linear, no references to ansible_mitogen/plugins in ansible.cfg.
I have a similar situation. It looks like mitogen prepends some garbage into the ssh hostname it tries to reach, at least in my case. If this is another problem, feel free to tell me to open a separate issue for it.
- name: create ssh_host_keys
community.crypto.openssh_keypair:
type: "{{ key.type|default(omit) }}"
size: "{{ key.size|default(omit) }}"
state: present
path: "{{ key.path }}"
loop_control:
loop_var: key
loop:
- type: rsa
size: 4096
path: /mnt/loop0p1/etc/ssh/ssh_host_rsa_key
- type: ed25519
path: /mnt/loop0p1/etc/ssh/ssh_host_ed25519_key
register: result
ignore_errors: "{{ ansible_check_mode }}"
- name: add new ssh_host_key to managementservers /home/ansible/.ssh/known_hosts
ansible.builtin.known_hosts:
hash_host: true
key: "{{ current_guest.ip }} {{ inner_item.1.public_key }}"
name: "{{ current_guest.ip }}"
state: present
path: /home/ansible/.ssh/known_hosts
loop: "{{ groups['ansible-cluster'] | product(result.results|selectattr('type', 'equalto', 'ed25519')) }}"
loop_control:
loop_var: inner_item
label: "{{ inner_item.0 }}: {{ inner_item.1.fingerprint }}"
delegate_to: "{{ inner_item.0 }}"
ignore_unreachable: true
ignore_errors: "{{ ansible_check_mode }}"
become: true
become_method: su
become_user: ansible
This is the problemactic part:
loop: "{{ groups['ansible-cluster'] | product(result.results|selectattr('type', 'equalto', 'ed25519')) }}"
It works without mitogen. Even though the contents of inner_item.0 is always the IP of the desired host (at least there is no other visible text when using debug: but mitogen (internally) garbles it.
My workaround is to extract the key with selectattr instead of using inner_item.1.public_key and to only loop "{{ groups['ansible-cluster'] }}, that works.
