openwisp-monitoring
openwisp-monitoring copied to clipboard
[bug] Management tunnel goes down just before iperf check was executed
Reported by @pandafy in https://github.com/openwisp/openwisp-monitoring/pull/394#pullrequestreview-1013241686
- The database reports that
DeviceConnection
is working, but the management tunnel goes down just before iperf check was executed. The code should be able to gracefully handle this and also notify users that SSH connection is not working.
Error log
[2022-06-21 14:04:53,311: ERROR/ForkPoolWorker-8] 'NoneType' object has no attribute 'open_session'
Traceback (most recent call last):
File "/home/aryaman/Desktop/Iperf-GSoC/env/lib/python3.8/site-packages/openwisp_controller/connection/connectors/ssh.py", line 174, in exec_command
stdin, stdout, stderr = self.shell.exec_command(command, timeout=timeout)
File "/home/aryaman/Desktop/Iperf-GSoC/env/lib/python3.8/site-packages/paramiko/client.py", line 508, in exec_command
chan = self._transport.open_session(timeout=timeout)
AttributeError: 'NoneType' object has no attribute 'open_session'
[2022-06-21 14:04:53,313: ERROR/ForkPoolWorker-8] Task openwisp_monitoring.check.tasks.perform_check[9b3fe0a6-e740-4bcc-abce-7c616c4837e7] raised unexpected: AttributeError("'NoneType' object has no attribute 'open_session'")
Traceback (most recent call last):
File "/home/aryaman/Desktop/Iperf-GSoC/env/lib/python3.8/site-packages/celery/app/trace.py", line 451, in trace_task
R = retval = fun(*args, **kwargs)
File "/home/aryaman/Desktop/Iperf-GSoC/env/lib/python3.8/site-packages/celery/app/trace.py", line 734, in __protected_call__
return self.run(*args, **kwargs)
File "/home/aryaman/Desktop/Iperf-GSoC/openwisp-monitoring/openwisp_monitoring/check/tasks.py", line 48, in perform_check
result = check.perform_check()
File "/home/aryaman/Desktop/Iperf-GSoC/openwisp-monitoring/openwisp_monitoring/check/base/models.py", line 86, in perform_check
return self.check_instance.check(store=True)
File "/home/aryaman/Desktop/Iperf-GSoC/openwisp-monitoring/openwisp_monitoring/check/classes/iperf.py", line 37, in check
res, exit_code = device_connection.connector_instance.exec_command(
File "/home/aryaman/Desktop/Iperf-GSoC/env/lib/python3.8/site-packages/openwisp_controller/connection/connectors/ssh.py", line 182, in exec_command
raise e
File "/home/aryaman/Desktop/Iperf-GSoC/env/lib/python3.8/site-packages/openwisp_controller/connection/connectors/ssh.py", line 174, in exec_command
stdin, stdout, stderr = self.shell.exec_command(command, timeout=timeout)
File "/home/aryaman/Desktop/Iperf-GSoC/env/lib/python3.8/site-packages/paramiko/client.py", line 508, in exec_command
chan = self._transport.open_session(timeout=timeout)
AttributeError: 'NoneType' object has no attribute 'open_session'