spring-cloud-commons
spring-cloud-commons copied to clipboard
spring-cloud
Feature: Enable request hedging for WebClient
Hedging is a mechanism for idempotent requests to automatically start a new request after some time threshold has been reached. For example, automatically firing off a new request when the latency of an existing HTTP call is at the 95th percentile. In some kind of retry mechanism, you'd have to cancel the first request. With hedging, you instead let them all run and simply use the first one that responds. The goal is to trade off some extra network traffic for reduced tail latencies. This is something we built and have been using ourselves. We thought it would be useful to contribute back to the community.
There are two specific implementation issues that I need help from the community here to answer.
First, the implementation here uses io.micrometer:micrometer.core to track the histogram of response times (to calculate the percentile latency). spring-cloud-commons doesn't have a required dependency on that library; it's optional through org.springframework.boot:spring-boot-start-actuator. I'm at a loss whether the right thing to do is to factor out this class with the dependency, add the dependency as required, or any other compromise (use a different library; note the dependency is needed to use the class).
The second is ordering. This thing adds an ExchangeFilterFunction. There's no easy way to control where this gets added in the filter list relative to the filters added by, for example, load balancing. The solutions I came up with to control this require the end user to override both LoadBalancerWebClientBuilderBeanPostProcessor and this thing's BeanPostProcessor`.
