mitogen icon indicating copy to clipboard operation
mitogen copied to clipboard

Document how to set `Connection.options.connect_timeout` found in "Failed to setup connection after X seconds" error

Open nhooey opened this issue 4 years ago • 3 comments

Problem

This error occurs when there is an SSH connection timeout:

[mux  787] 03:25:53.163826 D mitogen.parent: failing connection ssh.importer.staging.rsrchx.local
due to TimeoutError('Failed to setup connection after 10.00 seconds')

But it's not clear how to set that timeout duration in the Mitogen options.

Non-solutions

And due to the way Mitogen works, any SSH ConnectTimeout option that is set with SSH will have no effect on this because the timeout is scheduled in Mitogen's Python source.

Solution?

How do you set the Connection.options.connect_timeout value that's used in Connection._on_timer_expired() as shown in the above error message?

Relevant code

Where the connect_timeout option is initially set: mitogen.parent.Options.__init__():

        if connect_timeout:
            self.connect_timeout = connect_timeout
# ...
        self.connect_deadline = mitogen.core.now() + self.connect_timeout

Where the connect_timeout option is actually used: mitogen.parent.Connection._async_connect():

    def _async_connect(self):
        LOG.debug('creating connection to context %d using %s',
                  self.context.context_id, self.__class__.__module__)
        mitogen.core.listen(self._router.broker, 'shutdown',
                            self._on_broker_shutdown)
        self._timer = self._router.broker.timers.schedule(
            when=self.options.connect_deadline,
            func=self._on_timer_expired,
        )
  • Which version of Ansible are you running?
ansible 2.9.0
  config file = /code/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.7.9 (default, Sep 10 2020, 17:21:15) [GCC 8.3.0]
  • What version of Mitogen are you using? Version 0.2.9 from the website.

  • Is your version of Ansible patched in any way? No

  • Are you running with any custom modules, or module_utils loaded? No, but I have [paramiko_connection] and [ssh_connection] headings in my ansible.cfg.

  • Have you tried the latest master version from Git? No

nhooey avatar Nov 05 '20 16:11 nhooey

@s1113950 I noticed that you recently commented on another issue about options.

Do you know how to set this option?

nhooey avatar Nov 05 '20 16:11 nhooey

Sorry for the late reply, I thought I had already replied to this 🤦 .

https://github.com/dw/mitogen/blob/master/tests/parent_test.py#L96 here's the test for connect_timeout. The value should come from ansible_timeout or ansible_ssh_timeout, set from here: https://github.com/dw/mitogen/blob/master/ansible_mitogen/connection.py#L149 . Try setting one of those vars and let me know if it works! 😄

s1113950 avatar Dec 28 '20 02:12 s1113950

Sorry for the late reply, I thought I had already replied to this 🤦 .

https://github.com/dw/mitogen/blob/master/tests/parent_test.py#L96 here's the test for connect_timeout. The value should come from ansible_timeout or ansible_ssh_timeout, set from here: https://github.com/dw/mitogen/blob/master/ansible_mitogen/connection.py#L149 . Try setting one of those vars and let me know if it works! 😄

Just let you know that I try with ANSIBLE_TIMEOUT env var and it works. It doesn't work with ANSIBLE_SSH_TIMEOUT env var

Tested with ansible core 21.12.5 and mitogen v0.3.4 dev

hungpr0 avatar Oct 05 '22 08:10 hungpr0