redpanda icon indicating copy to clipboard operation
redpanda copied to clipboard

PESDLC-1488 Implement StreamsVerifierTx

Open savex opened this issue 1 year ago • 0 comments

PR implements StreamVerifier as a remote python app that can be run from cli either as a message generator right away, or used as a RESTful web server to sent produce/consume commands and get current status

Examples. Cli:

# Produce
python3 stream_verifier_txn.py --brokers localhost:19092 --debug --topic-prefix topic-stream-verifier-tx --topic-count 16 produce --rps 0 --total 64

# Consume
python3 stream_verifier_txn.py --brokers localhost:19092 --debug --topic-prefix topic-stream-verifier-tx --topic-count 16 consume

python3 stream_verifier_txn.py --brokers localhost:19092 webservice --port 8090

Webservice:

# app config get
curl -s -X GET http://localhost:8090

# app config update
curl -X POST http://localhost:8090 -d "{\"topic_count\": 256, \"msg_rate\": 0, \"msg_total\": 5000, \"worker_threads\": 6}"

# produce, status
curl -s -X GET http://localhost:8090/produce

# produce, run
curl -X POST --header "Content-Type: application/json" http://localhost:8090/produce -d "{\"topic_prefix\": \"topic-stream-verifier-tx\", \"topic_count\": 128}"

# consume, status
curl -s -X GET http://localhost:8090/consume
# consume, run
curl -X POST --header "Content-Type: application/json" http://localhost:8090/consume -d "{\"topic_prefix\": \"topic-stream-verifier-tx\", \"topic_count\": 128}"

Backports Required

  • [x] none - not a bug fix
  • [ ] none - this is a backport
  • [ ] none - issue does not exist in previous branches
  • [ ] none - papercut/not impactful enough to backport
  • [ ] v24.1.x
  • [ ] v23.3.x
  • [ ] v23.2.x

Release Notes

  • none

savex avatar Jun 05 '24 23:06 savex