google-cloud-rust
google-cloud-rust copied to clipboard
Add a subscriber throughput benchmark
We should add a throughput benchmark for the subscriber client, similar to https://cloud.google.com/blog/products/data-analytics/testing-cloud-pubsub-clients-to-maximize-streaming-performance
The benchmark should pull messages from a subscription and ack them, across N tasks.
(At least) the following should be configurable:
- test iterations
- test duration
- num clients
- num tasks
- num worker threads
- outstanding messages / bytes
- ack/nack flush duration
The benchmark should report
- the ack count for each iteration
- some sort of error rate
- (stretch) deadline expirations / message redeliveries
We need to run this thing for a while (>1 day) and report the results here.
The target throughput is around 600 MB/s on a 16 vCPU VM. I am not sure about the target error rate / redelivery rate.
I have a rough sketch of such a benchmark: https://github.com/dbolduc/google-cloud-rust/tree/1b01c325ca05f5ded4d60f0948d0cff63993ab02