client-rust icon indicating copy to clipboard operation
client-rust copied to clipboard

Add super batch for gRPC request and add more configures for client

Open yongman opened this issue 1 year ago • 1 comments

In this pr, super batch mechanism was added, and this mechanism default to disabled, you can enable it by

Config::default().with_kv_allow_batch(true);

and some other configures were added for batch size control, such as

Config::default()
.with_kv_allow_batch(true)
.with_kv_max_batch_size(10)
.with_kv_overload_threshold(400)
.with_kv_max_batch_wait_time(10)
.with_kv_max_inflight_requests(10000);

I have do some benchmark in my development host with different max_batch_size (but keeps other configs same), and the result meet the expectation.

+-------------+-----------+---------+---------+---------+----------+
|             |           |         |         |         |          |
| Perf\Batch  | No batch  | 10 max  | 20 max  | 50 max  | 100 max  |
+-------------+-----------+---------+---------+---------+----------+
|             |           |         |         |         |          |
| Ops/sec     | 48533     | 67772   | 72658   | 75951   | 75591    |
+-------------+-----------+---------+---------+---------+----------+
|             |           |         |         |         |          |
| p50(ms)     | 13.6      | 10.1    | 9.7     | 9.2     | 9.1      |
+-------------+-----------+---------+---------+---------+----------+
|             |           |         |         |         |          |
| p99(ms)     | 53.5      | 35.8    | 30.4    | 30.7    | 33.8     |
+-------------+-----------+---------+---------+---------+----------+
|             |           |         |         |         |          |
| p99.9(ms)   | 79.3      | 56.5    | 47.1    | 60.4    | 57.8     |
+-------------+-----------+---------+---------+---------+----------+

The result shows the performance has obvious improvement.

  • QPS has almost 50% growth when 20 max batch request configured.
  • P999 latency has 40% decrease when 20 max batch request configured.

yongman avatar Aug 12 '22 03:08 yongman

@ekexium @iosmanthus PTAL

yongman avatar Aug 15 '22 01:08 yongman