redpanda
redpanda copied to clipboard
CI Failure (key symptom) in `UpgradeBackToBackTest.test_upgrade_with_all_workloads`
https://buildkite.com/redpanda/vtools/builds/13742
Module: rptest.tests.upgrade_test
Class: UpgradeBackToBackTest
Method: test_upgrade_with_all_workloads
Arguments: {
"single_upgrade": false
}
test_id: UpgradeBackToBackTest.test_upgrade_with_all_workloads
status: FAIL
run time: 131.186 seconds
RemoteCommandError({'ssh_config': {'host': 'ip-172-31-3-81', 'hostname': '172.31.3.81', 'user': 'root', 'port': 22, 'password': None, 'identityfile': '/home/ubuntu/.ssh/id_rsa'}, 'hostname': 'ip-172-31-3-81', 'ssh_hostname': '172.31.3.81', 'user': 'root', 'externally_routable_ip': '35.161.143.84', '_logger': <Logger rptest.tests.upgrade_test.UpgradeBackToBackTest.test_upgrade_with_all_workloads.single_upgrade=False-822 (DEBUG)>, 'os': 'linux', '_ssh_client': <paramiko.client.SSHClient object at 0x7f3ce17ec0d0>, '_sftp_client': <paramiko.sftp_client.SFTPClient object at 0x7f3ce164b7f0>, '_custom_ssh_exception_checks': None}, 'curl -fsSL https://vectorized-public.s3.us-west-2.amazonaws.com/releases/redpanda/22.2.13/redpanda-22.2.13-amd64.tar.gz --retry 3 --retry-connrefused --retry-delay 2 --create-dir -o /opt/redpanda_installs/v22.2.13/redpanda.tar.gz && gunzip -c /opt/redpanda_installs/v22.2.13/redpanda.tar.gz | tar -xf - -C /opt/redpanda_installs/v22.2.13 && rm /opt/redpanda_installs/v22.2.13/redpanda.tar.gz', 35, b'')
Traceback (most recent call last):
File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run
data = self.run_test()
File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 276, in run_test
return self.test_context.function(self.test)
File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/mark/_mark.py", line 535, in wrapper
return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
File "/home/ubuntu/redpanda/tests/rptest/services/cluster.py", line 103, in wrapped
r = f(self, *args, **kwargs)
File "/home/ubuntu/redpanda/tests/rptest/tests/upgrade_test.py", line 224, in test_upgrade_with_all_workloads
for current_version in self.upgrade_through_versions(self.versions):
File "/home/ubuntu/redpanda/tests/rptest/tests/redpanda_test.py", line 247, in upgrade_through_versions
current_version = install_next()
File "/home/ubuntu/redpanda/tests/rptest/tests/redpanda_test.py", line 174, in install_next
self.redpanda._installer.install(self.redpanda.nodes, v)
File "/home/ubuntu/redpanda/tests/rptest/services/redpanda_installer.py", line 609, in install
self._install_unlocked(nodes, install_target)
File "/home/ubuntu/redpanda/tests/rptest/services/redpanda_installer.py", line 658, in _install_unlocked
raise e
File "/home/ubuntu/redpanda/tests/rptest/services/redpanda_installer.py", line 638, in _install_unlocked
self.wait_for_async_ssh(self._redpanda.logger,
File "/home/ubuntu/redpanda/tests/rptest/services/redpanda_installer.py", line 165, in wait_for_async_ssh
for l in ssh_out_per_node[node]:
File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/cluster/remoteaccount.py", line 687, in next
return next(self.iter_obj)
File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/cluster/remoteaccount.py", line 354, in output_generator
raise RemoteCommandError(self, cmd, exit_status, stderr.read())
ducktape.cluster.remoteaccount.RemoteCommandError: root@ip-172-31-3-81: Command 'curl -fsSL https://vectorized-public.s3.us-west-2.amazonaws.com/releases/redpanda/22.2.13/redpanda-22.2.13-amd64.tar.gz --retry 3 --retry-connrefused --retry-delay 2 --create-dir -o /opt/redpanda_installs/v22.2.13/redpanda.tar.gz && gunzip -c /opt/redpanda_installs/v22.2.13/redpanda.tar.gz | tar -xf - -C /opt/redpanda_installs/v22.2.13 && rm /opt/redpanda_installs/v22.2.13/redpanda.tar.gz' returned non-zero exit status 35.
JIRA Link: CORE-2976