yandex-tank icon indicating copy to clipboard operation
yandex-tank copied to clipboard

Failed stop monitoring

Open IvanKyznetsov opened this issue 4 years ago • 2 comments

Проблема во время остановки мониторинга. На удаленной машине ubuntu 20

2020-08-30 15:09:54,687 [DEBUG] yandextank.plugins.Phantom.plugin plugin.py:195 Processed ammo count: 100/ 2020-08-30 15:09:54,687 [DEBUG] yandextank.core.tankcore tankcore.py:319 RC after: 0 2020-08-30 15:09:54,687 [INFO] yandextank.core.tankcore tankcore.py:321 Stopping monitoring 2020-08-30 15:09:54,687 [INFO] yandextank.core.tankcore tankcore.py:323 Stopping <yandextank.plugins.Telegraf.plugin.Plugin object at 0x7f0495a288d0> 2020-08-30 15:09:54,688 [INFO] yandextank.plugins.Telegraf.plugin plugin.py:233 Finishing monitoring 2020-08-30 15:09:54,688 [DEBUG] yandextank.plugins.Telegraf.collector collector.py:120 Uninstalling monitoring agents 2020-08-30 15:09:54,688 [WARNING] yandextank.plugins.Telegraf.client client.py:361 Unable to correctly stop monitoring agent - session is broken on 134.209.. Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/Telegraf/client.py", line 355, in _stop_agent self.session.send("stop\n") File "/usr/local/lib/python2.7/dist-packages/yandextank/common/util.py", line 158, in send self.session.send(data) File "/usr/local/lib/python2.7/dist-packages/paramiko/channel.py", line 801, in send return self._send(s, m) File "/usr/local/lib/python2.7/dist-packages/paramiko/channel.py", line 1198, in _send raise socket.error("Socket is closed") error: Socket is closed 2020-08-30 15:09:54,688 [INFO] yandextank.plugins.Telegraf.client client.py:404 Session was broken on 134.209., switch to kill 2020-08-30 15:09:54,752 [INFO] yandextank.common.util util.py:107 Receiving from 134.209.:[/tmp/_agent.log] to [agent_134.209..log] 2020-08-30 15:09:54,752 [DEBUG] yandextank.common.util util.py:32 Opening SSH connection to 134.209.*****:22 2020-08-30 15:09:54,908 [DEBUG] urllib3.connectionpool connectionpool.py:442 https://overload.yandex.net:443 "POST /api/job/305103/push_data.json?upload_token=0b2e1175e88841759487b01df3b90f9e&api_token=69366fa914ca43d3b98fa864b0adcdc2 HTTP/1.1" 200 16 2020-08-30 15:09:54,908 [DEBUG] yandextank.plugins.DataUploader.client client.py:511 API response: [{u'success': 1}] 2020-08-30 15:09:55,107 [INFO] yandextank.plugins.Telegraf.client client.py:346 Daemon reader stopped 2020-08-30 15:09:55,144 [DEBUG] urllib3.connectionpool connectionpool.py:442 https://overload.yandex.net:443 "POST /api/job/305103/push_data.json?upload_token=0b2e1175e88841759487b01df3b90f9e&api_token=69366fa914ca43d3b98fa864b0adcdc2 HTTP/1.1" 200 16 2020-08-30 15:09:55,144 [DEBUG] yandextank.plugins.DataUploader.client client.py:511 API response: [{u'success': 1}] 2020-08-30 15:09:55,355 [DEBUG] urllib3.connectionpool connectionpool.py:442 https://overload.yandex.net:443 "POST /api/job/305103/push_data.json?upload_token=0b2e1175e88841759487b01df3b90f9e&api_token=69366fa914ca43d3b98fa864b0adcdc2 HTTP/1.1" 200 16 2020-08-30 15:09:55,356 [DEBUG] yandextank.plugins.DataUploader.client client.py:511 API response: [{u'success': 1}] 2020-08-30 15:09:56,194 [ERROR] yandextank.plugins.Telegraf.client client.py:394 Unable to get agent artefacts Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/Telegraf/client.py", line 386, in uninstall log_filename) File "/usr/local/lib/python2.7/dist-packages/yandextank/common/util.py", line 109, in get_file result = sftp.get(remote_path, local_path, self.get_progress_logger(remote_path)) File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 802, in get size = self.getfo(remotepath, fl, callback) File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 778, in getfo file_size = self.stat(remotepath).st_size File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 493, in stat t, msg = self._request(CMD_STAT, path) File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 813, in _request return self._read_response(num) File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 865, in _read_response self._convert_status(msg) File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 894, in _convert_status raise IOError(errno.ENOENT, text) IOError: [Errno 2] No such file 2020-08-30 15:09:56,194 [DEBUG] yandextank.plugins.Telegraf.collector collector.py:133 Waiting for agent <yandextank.plugins.Telegraf.client.SSHClient object at 0x7f0495766210> reader thread to finish.

IvanKyznetsov avatar Aug 30 '20 08:08 IvanKyznetsov

Could you provide monitoring.xml and load.ini files? That would definitely help to maintainers to determine the problem.

nnugumanov avatar Sep 01 '20 15:09 nnugumanov

Hello!

load.yml

overload:
  enabled: true
  token_file: token.txt
phantom:
  address: 157.230***:80
  uris:
    - /
  load_profile:
    load_type: rps
    schedule: line(0, 1000, 20)
console:
  enabled: true
telegraf:
  enabled: true
  package: yandextank.plugins.Telegraf
  config: monitoring.xml

monitoring.xml

<Host address="157.230***" interval="1" username="root">
    <CPU/>
    <Net/>
    <Memory/>
</Host>

IvanKyznetsov avatar Sep 01 '20 17:09 IvanKyznetsov