redpanda-data
PESDLC-1489 StreamVerifier service
This implements service part of the StreamVerifier that is created in this PR.
Service will supports all commands including 'produce', 'consume' and 'atomic' as well as getting current status and active command
Backports Required
- [x] none - not a bug fix
- [ ] none - this is a backport
- [ ] none - issue does not exist in previous branches
- [ ] none - papercut/not impactful enough to backport
- [ ] v24.1.x
- [ ] v23.3.x
- [ ] v23.2.x
Release Notes
- none
Most recent commit provides visibility on how very basic test structure might look like. One more thing to add is message validation option on consumption via externally pluggable validator.
new failures in https://buildkite.com/redpanda/redpanda/builds/51238#019094e8-d681-417f-a7c3-8db584f7d721:
"rptest.tests.delete_records_test.DeleteRecordsTest.test_delete_records_segment_deletion.cloud_storage_enabled=True.truncate_point=at_high_watermark"
"rptest.transactions.tx_admin_api_test.TxAdminTest.test_mark_transaction_expired"
new failures in https://buildkite.com/redpanda/redpanda/builds/51587#0190bc79-b124-448a-a387-9c4f087b5635:
"rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node"
new failures in https://buildkite.com/redpanda/redpanda/builds/51658#0190c184-0228-4a99-9546-e8430d8b45dd:
"rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node"
new failures in https://buildkite.com/redpanda/redpanda/builds/51658#0190c184-fa72-4f55-bca5-85c2cdc49e69:
"rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node"
@bharathv , updated to fit simplification. Will continue on Monday to debug and polish. Right now there is strange count mismatch happening
[INFO:2024-07-12 22:09:04,105]: RunnerClient: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_streaming_upgrade: Data: None
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node
status: PASS
run time: 1 minute 39.690 seconds
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_streaming_restart_unsafely_and_add
status: IGNORE
run time: 0.000 seconds
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_streaming_restart_safely_and_add
status: FAIL
run time: 3 minutes 10.183 seconds
...
AssertionError: Produced/Atomic message count mismatch: 1250/1251
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_streaming_upgrade
status: FAIL
run time: 2 minutes 4.507 seconds
...
AssertionError: Produced/Atomic message count mismatch: 996/69
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
============================================================================================================================================================================================================================================================================================================================
SESSION REPORT (ALL TESTS)
ducktape version: 0.8.18
session_id: 2024-07-12--012
run time: 7 minutes 1.258 seconds
tests run: 4
passed: 1
flaky: 0
failed: 2
ignored: 1
opassed: 0
ofailed: 0
opassedfips: 0
ofailedfips: 0
============================================================================================================================================================================================================================================================================================================================
Most probably due to lack of message retry to produce after KafkaErrors on node unavailability.
skipped ducktape retry in https://buildkite.com/redpanda/redpanda/builds/51451#0190a94f-5ebe-47b9-baed-e16504ea0f32: pandatriage cache was not found
skipped ducktape retry in https://buildkite.com/redpanda/redpanda/builds/51451#0190a965-6c5a-43db-9d72-64e6387d099f: pandatriage cache was not found
Sample test runs using EC2
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node
status: PASS
run time: 2 minutes 21.611 seconds
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node
status: PASS
run time: 2 minutes 41.324 seconds
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node
status: PASS
run time: 2 minutes 28.480 seconds
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node
status: PASS
run time: 2 minutes 16.879 seconds
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node
status: PASS
run time: 2 minutes 18.292 seconds
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node
status: PASS
run time: 2 minutes 24.543 seconds
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node
status: PASS
run time: 2 minutes 26.633 seconds
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node
status: PASS
run time: 2 minutes 18.435 seconds
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node
status: PASS
run time: 2 minutes 27.722 seconds
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: rptest.transactions.stream_verifier_test.StreamVerifierTest.test_simple_produce_consume_txn_with_add_node
status: PASS
run time: 2 minutes 17.179 seconds
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
=============================================================================================================================================================================================================================================================================================================================================
SESSION REPORT (ALL TESTS)
ducktape version: 0.8.18
session_id: 2024-07-15--010
run time: 24 minutes 17.711 seconds
tests run: 10
passed: 10
flaky: 0
failed: 0
ignored: 0
opassed: 0
ofailed: 0
opassedfips: 0
ofailedfips: 0
=============================================================================================================================================================================================================================================================================================================================================
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/51539#0190b813-8c1a-4c0d-b167-1183571c629f
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/51587#0190bc7b-44ac-4632-903e-0d5b8b8b747c
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/51658#0190c184-022c-4ee8-8b50-5a9c9ccd5a2d
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/51658#0190c184-fa77-428e-8ad2-74bebbe97ffa
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/51677#0190c229-84b8-4a71-a064-26caba5d1a5c
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/51799#0190ccbe-f206-445d-802f-af11e40346d8
@bharathv, yes, I keep forgetting that CDT runs in docker and there should be different set of timings
/cdt release rp_version=build skip-units dt-repeat=20 tests/rptest/transactions/stream_verifier_test.py
/cdt release rp_version=build skip-units dt-repeat=20 tests/rptest/transactions/stream_verifier_test.py