redpanda
redpanda copied to clipboard
CI Failure (key symptom) in `RedpandaCloudSelfTest.test_healthy`
https://buildkite.com/redpanda/vtools/builds/13738
Module: rptest.redpanda_cloud_tests.cloud_self_test
Class: RedpandaCloudSelfTest
Method: test_healthy
test_id: RedpandaCloudSelfTest.test_healthy
status: FAIL
run time: 23.425 seconds
CalledProcessError(1, ['tsh', 'ssh', '--proxy=proxy.tp.redpanda.com:443', '--auth=okta', '--identity=/tmp/machine-id/identity', 'redpanda@cp24867ek221n77ef6tg-agent', './breakglass-tools.sh'], '', '\x1b[31mERROR: \x1b[0mfailed connecting to host cp24867ek221n77ef6tg-agent:0: failed to receive cluster details response\n\tfailed to dial target host\n\tTeleport proxy failed to connect to "node" agent "@local-node" over reverse tunnel:\n\n no tunnel connection found: no node reverse tunnel for 5526817b-8b2d-4f62-b5dd-d2dc8a09d12e.proxy.tp.redpanda.com found\n\nThis usually means that the agent is offline or has disconnected. Check the\nagent logs and, if the issue persists, try restarting it or re-registering it\nwith the cluster.\n\n')
Traceback (most recent call last):
File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 177, in _do_run
self.test = self.test_context.cls(self.test_context)
File "/home/ubuntu/redpanda/tests/rptest/redpanda_cloud_tests/cloud_self_test.py", line 24, in __init__
super().__init__(test_context, *args, **kwargs)
File "/home/ubuntu/redpanda/tests/rptest/tests/redpanda_cloud_test.py", line 25, in __init__
self.redpanda = make_redpanda_cloud_service(test_context)
File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 4928, in make_redpanda_cloud_service
return RedpandaServiceCloud(context,
File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 1610, in __init__
self.rebuild_pods_classes()
File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 1638, in rebuild_pods_classes
for p in self.get_redpanda_pods()
File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 1697, in get_redpanda_pods
pods = json.loads(self.kubectl.cmd('get pods -n redpanda -o json'))
File "/home/ubuntu/redpanda/tests/rptest/clients/kubectl.py", line 249, in cmd
self._install()
File "/home/ubuntu/redpanda/tests/rptest/clients/kubectl.py", line 122, in _install
self._ssh_cmd(['./breakglass-tools.sh'])
File "/home/ubuntu/redpanda/tests/rptest/clients/kubectl.py", line 232, in _ssh_cmd
return self._local_cmd(local_cmd)
File "/home/ubuntu/redpanda/tests/rptest/clients/kubectl.py", line 215, in _local_cmd
raise subprocess.CalledProcessError(process.returncode, cmd, s_out,
subprocess.CalledProcessError: Command '['tsh', 'ssh', '--proxy=proxy.tp.redpanda.com:443', '--auth=okta', '--identity=/tmp/machine-id/identity', 'redpanda@cp24867ek221n77ef6tg-agent', './breakglass-tools.sh']' returned non-zero exit status 1.
JIRA Link: CORE-2978