ModuleResponder sends ansible.cli package, and many others, probably unnecessarily

[moreati]

Performing an Ansible ping, with -vvv shows that Mitogen is sending the package ansible.cli to the target, and many other parts of the Ansible codebase that I would expect to be unneeded and irrelevant on a target.

[mux  19249] 09:50:45.309897 D mitogen.responder: sending ansible.cli (16.54 KiB) to local.19253

Adding some instrumentation to ModuleFinder.find_related() shows that ansible.cli is being incuded as a related import of ansible.module_utils.basic, which imports __main__. import __main__ has been present in ansible.module_utils.basic since Ansible 2.7. I t was introduced in https://github.com/ansible/ansible/commit/52449cc01a71778ef94ea0237eed0284f5d75582

diff --git a/mitogen/master.py b/mitogen/master.py
index b1e0a1de..3a7a72ad 100644
--- a/mitogen/master.py
+++ b/mitogen/master.py
@@ -1063,12 +1063,14 @@ class ModuleFinder(object):
             for which source code can be retrieved
         :type fullname: str
         """
+        LOG.info('find_related(%r)', fullname)
         stack = [fullname]
         found = set()
 
         while stack:
             name = stack.pop(0)
             names = self.find_related_imports(name)
+            LOG.info('find_related(%r) -> %r -> %r', fullname, name, names)
             stack.extend(set(names).difference(set(found).union(stack)))
             found.update(names)
 

[mux  19249] 09:50:44.800698 I mitogen: find_related('ansible')
[mux  19249] 09:50:44.801085 I mitogen: find_related('ansible') -> 'ansible' -> []
[mux  19249] 09:50:44.801424 D mitogen.responder: sending ansible (0.60 KiB) to local.19253
[mux  19249] 09:50:44.802015 D mitogen.responder: ansible.module_utils is a package at /nix/store/sd4jwfhmz53g9srmvgzkm2rz512v4lpf-python3.11-ansible-core-2.16.4/lib/python3.11/site-packages/ansible/module_utils/__init__.py with submodules ['_text', 'ansible_release', 'api', 'basic', 'common', 'compat', 'connection', 'csharp', 'distro', 'errors', 'facts', 'json_utils', 'parsing', 'powershell', 'pycompat24', 'service', 'six', 'splitter', 'urls', 'yumdnf']
[mux  19249] 09:50:44.802312 I mitogen: find_related('ansible.module_utils')
[mux  19249] 09:50:44.802639 I mitogen: find_related('ansible.module_utils') -> 'ansible.module_utils' -> ['ansible']
[mux  19249] 09:50:44.802923 I mitogen: find_related('ansible.module_utils') -> 'ansible' -> []
[mux  19249] 09:50:44.803223 D mitogen.responder: sending ansible.module_utils (0.54 KiB) to local.19253
[mux  19249] 09:50:44.813366 I mitogen: find_related('ansible.module_utils.basic')
[mux  19249] 09:50:44.821895 I mitogen: find_related('ansible.module_utils.basic') -> 'ansible.module_utils.basic' -> ['__main__', 'ansible', 'ansible.module_utils', 'ansible.module_utils._text', 'ansible.module_utils.common._json_compat', 'ansible.module_utils.common._utils', 'ansible.module_utils.common.arg_spec', 'ansible.module_utils.common.file', 'ansible.module_utils.common.locale', 'ansible.module_utils.common.parameters', 'ansible.module_utils.common.process', 'ansible.module_utils.common.sys_info', 'ansible.module_utils.common.text.converters', 'ansible.module_utils.common.text.formatters', 'ansible.module_utils.common.validation', 'ansible.module_utils.common.warnings', 'ansible.module_utils.compat', 'ansible.module_utils.errors', 'ansible.module_utils.parsing.convert_bool', 'ansible.module_utils.pycompat24', 'ansible.module_utils.six']
[mux  19249] 09:50:44.822577 I mitogen: find_related('ansible.module_utils.basic') -> '__main__' -> ['ansible.cli.playbook']

[moreati]

PoC reduces localhost ping time from 0.98 seconds to 0.66

diff --git a/mitogen/master.py b/mitogen/master.py
index b1e0a1de..cbd8a9f2 100644
--- a/mitogen/master.py
+++ b/mitogen/master.py
@@ -1048,6 +1048,7 @@ class ModuleFinder(object):
                 if sys.modules.get(name) is not None
                 and not is_stdlib_name(name)
                 and u'six.moves' not in name  # TODO: crap
+                and name != u'__main__'
             )
         ))
 

[moreati]

A code search for import __main__ shows it's not unique in Ansible, so it may need better handling than just the above PoC