ansible_mitogen blacklists third-party modules

[nradchenko]

Hi,

I'd like to use xml module in my playbooks, but lxml python module required for operation is implicitly blacklisted for some unknown reason.

root@0d2e911be079:/etc/ansible# python -V
Python 3.11.4
root@0d2e911be079:/etc/ansible# pip freeze | grep -e ansible -e mitogen
ansible==2.10.7
ansible-base==2.10.17
mitogen==0.3.4
root@0d2e911be079:/etc/ansible# cat test.yml 
- hosts: all
  tasks:
    - xml:
        path: somepath
root@0d2e911be079:/etc/ansible# ansible-playbook test.yml -i localhost, -c local -l all -CD

PLAY [all] **********************************************************************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************************************************************
ok: [localhost]

TASK [xml] **********************************************************************************************************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: 'lxml' is present in the Mitogen importer blacklist, therefore this context will not attempt to request it from the master, as the request will always be refused.
fatal: [localhost]: FAILED! => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python3"}, "changed": false, "msg": "Failed to import the required Python library (lxml) on 0d2e911be079's Python /usr/bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"}

PLAY RECAP **********************************************************************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

root@0d2e911be079:/etc/ansible# 

[moreati]

There is a bug here but I'm afraid fixing it wouldn't allow mitogen to send the lxml module. Importing lxml requires importing lxml.etree, whch is a binary module. Binary modules are too platform specific for Mitogen to copy them across.

The error message should have been

TASK [xml] **********************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: The Mitogen master process was unable to serve 'lxml.etree'. It may be a native Python extension, or it may be missing entirely. Check the importer debug logs on the master for more information.
fatal: [bertha.local]: FAILED! => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python3"}, "changed": false, "msg": "Failed to import the required Python library (lxml) on bertha's Python /usr/bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"}

[moreati]

To use the Ansible xml module you'll need to install lxml on the target host(s), regardless of Mitogen.

[moreati]

ansible_mitogen.process._setup_responder() populates mitogen.core.Importer.whitelist. However this causes the logic in mitogen.core.is_blacklisted_import() to report all non-whitelisted modules (e.g. lxml) as blacklisted. Causing misleading/false denials such as

ModuleNotFoundError: 'lxml' is present in the Mitogen importer blacklist, therefore this context will not attempt to request it from the master, as the request will always be refused.

[moreati]

Looking through old commits this behaviour appears to be by design. If so it comes as a surprise to me.