How can I change the time of DNS Cache

[libingnan]

Please,I want to change the time of DNS Cache~

[spencergibb]

Not sure what you mean, but gateway has no DNS cache. Please feel free to explain more and perhaps I'll reopen

[trajano]

Sorry to necro, but this is the first hit when I Google "spring cloud gateway DNS cache" I think it would be the same context.

In my case I have services that the gateway uses. These services are referenced by their DNS name (rather than IP addresses). Since Docker can manage the routing via Virtual IPs and its internal DNS could map against it.

Internally Spring Cloud Gateway uses Netty, which does provide a DefaultDNSCache that allows clients to specify how long to cache it's DNS entries for. But I can't find in the documentation any way of changing that or reinjecting it.

Also netty does not recognize the networkaddress.cache.ttl value for the DNS.

So if I remove and re-add the stack I would get Caused by: java.net.ConnectException: finishConnect(..) failed: No route to host

Although it could be the InMemoryDnsResolver is used rather than the SystemDnsResolver from HttpComponents

[trajano]

Anyway this is technically doable (in Hoxton at least through HttpClientCustomizers)

@Component
public class RemoveDnsCacheCustomizer implements HttpClientCustomizer {
    @Override
    public HttpClient customize(HttpClient httpClient) {
        DnsNameResolverBuilder dnsResolverBuilder = new DnsNameResolverBuilder()
            .channelFactory(EpollDatagramChannel::new)
            .resolveCache(new DefaultDnsCache(0, 1, 0));
        httpClient.tcpConfiguration(tcpClient -> tcpClient.resolver(new DnsAddressResolverGroup(dnsResolverBuilder)));
        return httpClient;
    }
}

[trajano]

Low hanging fruit would be to add the code snippet to the documentation. A more comprehensive one would likely be something akin to auto-configuring a DnsNameResolver if one is not present and provide two options.

1. default
2. non-caching

[salaboy]

@trajano @spencergibb I am facing this issue in Kubernetes.. i think by default.. or in Kubernetes the caching needs to be disabled by default..

[salaboy]

For the ones looking at the classes DnsNameResolverBuilder and DefaultDnsCache , they come from

<dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-resolver-dns</artifactId>
        </dependency>

Which is not added by spring-cloud-gateway by default.

[salaboy]

@trajano did you manage to remove the cache completely?

[violetagg]

All, the latest Reactor Netty release 1.0.x uses Netty DNS resolver. Here you can find the documentation https://projectreactor.io/docs/netty/release/reference/index.html#_host_name_resolution_2

This means Spring Cloud Gateway 3.0.x (Spring Cloud 2020.0.x) uses by default Netty DNS resolver

[trajano]

@salaboy no but having the TTL short is usually enough. It may overload the DNS if it is too low anyway.

[salaboy]

@violetagg is there any way to access that DNS resolver to debug it? I am trying to figure out how the spring cloud gateway (3.0.x) actually use it.. to see if I can find the guilty one.. or to see if I can manually force to empty the cache.. Also do you know if the DNS resolver does support multiple IPs for a single domain name?

@trajano are you working in Kube as well?

[violetagg]

@salaboy We just provide configuration for the Netty DNS resolver, the actual implementation is netty-resolver-dns. You can start from this code https://github.com/reactor/reactor-netty/blob/8bfb9a864dcef05e86bda67e2c7e250b7a42a4e7/reactor-netty-core/src/main/java/reactor/netty/transport/NameResolverProvider.java#L393-L427

[trajano]

@salaboy nope I'm on swarm.

[salaboy]

@trajano quite similar then :)

[salaboy]

@violetagg thanks a lot for the pointer.. I will report back

[pokusak]

@violetagg Can you please elaborate a little on how can I pass configuration from Spring to Netty DNS? I need to set searchDomain to custom value but I don't see the Spring way to do it. (Customizer doesn't seems to work. I have modified example above to not disable cache but to set searchDomains. It is called. But when reactor.netty.http.client.HttpClientConfig#resolverInternal is called it goes via reactor.netty.tcp.TcpResources#getOrCreateDefaultResolver which creates new instance which doesn't honor settings from customizer) Using Windows no Docker, no Kubernetes, ... reactor-netty-core=1.0.10 which came from Spring Cloud Gateway thanks.

[violetagg]

@pokusak Please try to specify the resolver directly

@Component
public class RemoveDnsCacheCustomizer implements HttpClientCustomizer {
    @Override
    public HttpClient customize(HttpClient httpClient) {
        DnsNameResolverBuilder dnsResolverBuilder = new DnsNameResolverBuilder()
            .channelFactory(EpollDatagramChannel::new)
            .resolveCache(new DefaultDnsCache(0, 1, 0));
        httpClient.resolver(new DnsAddressResolverGroup(dnsResolverBuilder));
        return httpClient;
    }
}

[pokusak]

@violetagg tried already yesterday with same result

[violetagg]

@pokusak Please provide a reproducible example

[salaboy]

@violetagg it never worked for me.. is it working now?

[pokusak]

please see attached app demo.zip Its app generated from initializr, added some dummy path and customization of searchDomains. @violetagg please let me know if this example works for you

[spencergibb]

resolver() does not mutate the http client

@Component
public class SearchDomainsCustomizer implements HttpClientCustomizer {
    @Override
    public HttpClient customize(HttpClient httpClient) {
        DnsNameResolverBuilder dnsResolverBuilder = new DnsNameResolverBuilder().searchDomains(Arrays.asList("com"));
        httpClient = httpClient.resolver(new DnsAddressResolverGroup(dnsResolverBuilder));
        return httpClient;
    }
}

[pokusak]

@spencergibb You are right, that did the trick. Thanks! (just for completeness channelFactory has to be also specified. it can't be omitted as I did.)

[salaboy]

@spencergibb can this be included by default or maybe by a spring-cloud-kubernetes sub-module? Not having this in Kubernetes really mess up things.

[kostya05983]

More beauty way to disable DNS cache without default 0,0,0

@Component
class RemoveCacheCustomizer: HttpClientCustomizer {
    override fun customize(httpClient: HttpClient): HttpClient {
        val dnsResolverBuilder = DnsNameResolverBuilder()
            .channelFactory { EpollDatagramChannel() }
            .resolveCache(NoopDnsCache.INSTANCE)
        return httpClient.resolver(DnsAddressResolverGroup(dnsResolverBuilder))
    }
}

[jlconde15]

@salaboy Did you be able to solve the issue using the spring cloud gateway with kubernetes? I have the same issue and the spring cloud gateway is always sending the request to the same pod. If I make a curl from inside the k8s cluster to the service the kube-proxy is working property and send the request to one of the pods associated with the service, but like a said the spring cloud gateway microservice always send the request to the same pod. If i delete that pod, then the gateway starts using the other one after it updated the DNS here the gateway logs: 2022-01-20 11:12:38.859 DEBUG 1 --- [or-http-epoll-3] reactor.netty.http.client.HttpClient : [09005325] REGISTERED 2022-01-20 11:12:38.860 DEBUG 1 --- [or-http-epoll-3] io.netty.resolver.dns.DnsQueryContext : [id: 0x77494498] WRITE: UDP, [11686: /10.0.0.10:53], DefaultDnsQuestion(product-service.default.svc.cluster.local. IN A) 2022-01-20 11:12:38.864 DEBUG 1 --- [or-http-epoll-3] io.netty.resolver.dns.DnsNameResolver : [id: 0x77494498] RECEIVED: UDP [11686: /10.0.0.10:53], DatagramDnsResponse(from: /10.0.0.10:53, 11686, QUERY(0), NoError(0), RD AA) DefaultDnsQuestion(product-service.default.svc.cluster.local. IN A) DefaultDnsRawRecord(product-service.default.svc.cluster.local. 5 IN A 4B) DefaultDnsRawRecord(OPT flags:0 udp:4096 0B)

I have also posted that issue on here https://stackoverflow.com/questions/70771033/spring-cloud-gateway-inside-k8s-routes-are-not-being-load-balanced?noredirect=1#comment125121469_70771033

[jlconde15]

Hi, I found the issue here https://learnk8s.io/kubernetes-long-lived-connections I just set the httpClient keepAlive property to false, and now it is working as expected

[bindupatnaik]

@violetagg @libingnan @trajano @kostya05983 @salaboy @jlconde15 pls guide . I am facing same problem as spring-cloud-api-gateway is remembering the hostnames of microservices apps even though they are down it is trying to reach them. api-gateway is bypassing the load balancer of microservices instead it is trying to call microservice apps by remembering their hostnames in DNS Cache. I tried both by setting the defaultDnsCache to 0,0,0 or removing the DnsCache completely like below but nothing seem to be working. Any help is much appreciated. Not sure httpClient is being invoked with below configuration. ++++++++++++++++++++ @Component public class RemoveDnsCacheCustomizer implements HttpClientCustomizer { @Override public HttpClient customize(HttpClient httpClient) { DnsNameResolverBuilder dnsResolverBuilder = new DnsNameResolverBuilder() .channelFactory(EpollDatagramChannel::new) .resolveCache(new DefaultDnsCache(0, 1, 0)); httpClient.resolver(new DnsAddressResolverGroup(dnsResolverBuilder)); return httpClient; } } +++++++++++++++++++++++ @Component class RemoveCacheCustomizer: HttpClientCustomizer { override fun customize(httpClient: HttpClient): HttpClient { val dnsResolverBuilder = DnsNameResolverBuilder() .channelFactory { EpollDatagramChannel() } .resolveCache(NoopDnsCache.INSTANCE) return httpClient.resolver(DnsAddressResolverGroup(dnsResolverBuilder)) } } +++++++++++++++++++++++++