redpanda
redpanda copied to clipboard
redpanda-data
CI Failure (`assert offset == o`) in `TimeQueryTest.test_timequery`
https://buildkite.com/redpanda/vtools/builds/12363
Module: rptest.tests.timequery_test
Class: TimeQueryTest
Method: test_timequery
Arguments: {
"spillover": true,
"batch_cache": false,
"cloud_storage": true
}
test_id: TimeQueryTest.test_timequery
status: FAIL
run time: 33.746 seconds
AssertionError()
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 184, in _do_run
data = self.run_test()
File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 269, in run_test
return self.test_context.function(self.test)
File "/usr/local/lib/python3.10/dist-packages/ducktape/mark/_mark.py", line 481, in wrapper
return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
File "/home/ubuntu/redpanda/tests/rptest/services/cluster.py", line 104, in wrapped
r = f(self, *args, **kwargs)
File "/home/ubuntu/redpanda/tests/rptest/tests/timequery_test.py", line 372, in test_timequery
self._do_test_timequery(cloud_storage, batch_cache, spillover)
File "/home/ubuntu/redpanda/tests/rptest/tests/timequery_test.py", line 346, in _do_test_timequery
self._test_timequery(cluster=self.redpanda,
File "/home/ubuntu/redpanda/tests/rptest/tests/timequery_test.py", line 210, in _test_timequery
assert offset == o
AssertionError
JIRA Link: CORE-1895
Note to developer: please provide a message with the assertion when addressing this issue
*https://buildkite.com/redpanda/vtools/builds/12827 *https://buildkite.com/redpanda/vtools/builds/12843
Without going into the details of the CI failure itself, whats interesting about this is that it has only been triggered in CDT over the last few months, and every report of it failing also has a number of other failed ducktapes with it- usually node operation or partition moving tests.
Most specifically, this seems to fail quite often with rptest.tests.cloud_storage_timing_stress_test::CloudStorageTimingStressTest.test_cloud_storage_with_partition_moves and rptest.tests.node_pool_migration_test::NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.
So, that leads me to ask the question, is ducktape parallelization and the interaction of these tests responsible for the failures seen in the TimeQueryTest?
Timequeries seem very stable on their own.
*https://buildkite.com/redpanda/vtools/builds/14763 *https://buildkite.com/redpanda/vtools/builds/14777
Closing older-bot-filed CI issues as we transition to a more reliable system.
