client-rust
client-rust copied to clipboard
Add super batch for gRPC request and add more configures for client
In this pr, super batch mechanism was added, and this mechanism default to disabled, you can enable it by
Config::default().with_kv_allow_batch(true);
and some other configures were added for batch size control, such as
Config::default()
.with_kv_allow_batch(true)
.with_kv_max_batch_size(10)
.with_kv_overload_threshold(400)
.with_kv_max_batch_wait_time(10)
.with_kv_max_inflight_requests(10000);
I have do some benchmark in my development host with different max_batch_size
(but keeps other configs same), and the result meet the expectation.
+-------------+-----------+---------+---------+---------+----------+
| | | | | | |
| Perf\Batch | No batch | 10 max | 20 max | 50 max | 100 max |
+-------------+-----------+---------+---------+---------+----------+
| | | | | | |
| Ops/sec | 48533 | 67772 | 72658 | 75951 | 75591 |
+-------------+-----------+---------+---------+---------+----------+
| | | | | | |
| p50(ms) | 13.6 | 10.1 | 9.7 | 9.2 | 9.1 |
+-------------+-----------+---------+---------+---------+----------+
| | | | | | |
| p99(ms) | 53.5 | 35.8 | 30.4 | 30.7 | 33.8 |
+-------------+-----------+---------+---------+---------+----------+
| | | | | | |
| p99.9(ms) | 79.3 | 56.5 | 47.1 | 60.4 | 57.8 |
+-------------+-----------+---------+---------+---------+----------+
The result shows the performance has obvious improvement.
- QPS has almost 50% growth when 20 max batch request configured.
- P999 latency has 40% decrease when 20 max batch request configured.
@ekexium @iosmanthus PTAL