async-http-client icon indicating copy to clipboard operation
async-http-client copied to clipboard

Support for DnsAddressResolverGroup to allow Netty's InflightNameResolver

Open rnorth opened this issue 5 years ago • 4 comments

For background, we attempted to use asynchttpclient with a Netty async DNS resolver some time back, but managed to quite severely DoS our cluster DNS servers. It emerged that concurrent DNS lookups for the same host were not being 'coalesced'; the effect was that any DNS cache miss would cause our services to hit DNS 1:1 with every outgoing HTTP call.

We were building a common Netty DnsNameResolver using DnsNameResolverBuilder() and passing through to each request using setNameResolver on RequestBuilderBase.

There are a few issues and completed PRs in Netty around this space. Ultimately it looks like Netty's InflightNameResolver is the solution, but it is internal to Netty, and therefore can't be supplied to asynchttpclient.

Comments such as this from @trustin suggest that the right thing to do might be for asynchttpclient to accept a DnsAddressResolverGroup. As a result, DNS lookups would go via the InflightNameResolver and would be coalesced automatically.

I'd be curious to get your views on this - is this the right approach, or are we missing an obvious simpler way? 😄

rnorth avatar Jul 05 '19 13:07 rnorth

Looks like the way to go. Contributions welcome :)

slandelle avatar Jul 05 '19 13:07 slandelle

I might be interested in doing this. I am currently adapting DnsAddressResolverGroup using an ugly hack, so this interests me a lot.

astei avatar Sep 28 '19 05:09 astei

@astei Sure, keep'em coming :)

slandelle avatar Sep 28 '19 12:09 slandelle

This seems like a pretty big API change and a complex internal change just because we want to use InflightNameResolver. The only thing preventing us from using InflightNameResolver is that it's package-private in Netty.

@slandelle what are your thoughts on just copying InflightNameResolver and using the copy? It's ~100 lines of extremely simple code.

n-miles avatar May 09 '20 00:05 n-miles