async-http-client
async-http-client copied to clipboard
Support for DnsAddressResolverGroup to allow Netty's InflightNameResolver
For background, we attempted to use asynchttpclient with a Netty async DNS resolver some time back, but managed to quite severely DoS our cluster DNS servers. It emerged that concurrent DNS lookups for the same host were not being 'coalesced'; the effect was that any DNS cache miss would cause our services to hit DNS 1:1 with every outgoing HTTP call.
We were building a common Netty DnsNameResolver using DnsNameResolverBuilder() and passing through to each request using setNameResolver on RequestBuilderBase.
There are a few issues and completed PRs in Netty around this space. Ultimately it looks like Netty's InflightNameResolver is the solution, but it is internal to Netty, and therefore can't be supplied to asynchttpclient.
Comments such as this from @trustin suggest that the right thing to do might be for asynchttpclient to accept a DnsAddressResolverGroup. As a result, DNS lookups would go via the InflightNameResolver and would be coalesced automatically.
I'd be curious to get your views on this - is this the right approach, or are we missing an obvious simpler way? 😄
Looks like the way to go. Contributions welcome :)
I might be interested in doing this. I am currently adapting DnsAddressResolverGroup using an ugly hack, so this interests me a lot.
@astei Sure, keep'em coming :)
This seems like a pretty big API change and a complex internal change just because we want to use InflightNameResolver. The only thing preventing us from using InflightNameResolver is that it's package-private in Netty.
@slandelle what are your thoughts on just copying InflightNameResolver and using the copy? It's ~100 lines of extremely simple code.