rpc-perf icon indicating copy to clipboard operation
rpc-perf copied to clipboard

more graceful connect retry behavior

Open thinkingfish opened this issue 6 years ago • 2 comments

connect is a fairly expensive operation for the cache server, at large volume it can easily overwhelm the server thread accepting new connections temporarily, leading to test instabilities. Since it's currently impossible to coordinate different rpcperf instances at a fine granularity, I think we should introduce a few features into the connect behavior to reduce the chance of "reconnect storms":

  • allow connect to be rate-limited, and the rate can be set in the config;
  • allow timeouts to be retried with an exponential backoff, with a max cap that can be configured.

thinkingfish avatar Jul 06 '18 21:07 thinkingfish

partially implemented in #162

brayniac avatar Aug 07 '18 17:08 brayniac

#236 should also help with this by allowing for "soft timeouts"

brayniac avatar Sep 09 '19 17:09 brayniac