redpanda
redpanda copied to clipboard
PESDLC-1488 Implement StreamsVerifierTx
PR implements StreamVerifier as a remote python app that can be run from cli either as a message generator right away, or used as a RESTful web server to sent produce/consume commands and get current status
Examples. Cli:
# Produce
python3 stream_verifier_txn.py --brokers localhost:19092 --debug --topic-prefix topic-stream-verifier-tx --topic-count 16 produce --rps 0 --total 64
# Consume
python3 stream_verifier_txn.py --brokers localhost:19092 --debug --topic-prefix topic-stream-verifier-tx --topic-count 16 consume
python3 stream_verifier_txn.py --brokers localhost:19092 webservice --port 8090
Webservice:
# app config get
curl -s -X GET http://localhost:8090
# app config update
curl -X POST http://localhost:8090 -d "{\"topic_count\": 256, \"msg_rate\": 0, \"msg_total\": 5000, \"worker_threads\": 6}"
# produce, status
curl -s -X GET http://localhost:8090/produce
# produce, run
curl -X POST --header "Content-Type: application/json" http://localhost:8090/produce -d "{\"topic_prefix\": \"topic-stream-verifier-tx\", \"topic_count\": 128}"
# consume, status
curl -s -X GET http://localhost:8090/consume
# consume, run
curl -X POST --header "Content-Type: application/json" http://localhost:8090/consume -d "{\"topic_prefix\": \"topic-stream-verifier-tx\", \"topic_count\": 128}"
Backports Required
- [x] none - not a bug fix
- [ ] none - this is a backport
- [ ] none - issue does not exist in previous branches
- [ ] none - papercut/not impactful enough to backport
- [ ] v24.1.x
- [ ] v23.3.x
- [ ] v23.2.x
Release Notes
- none