grpc-go
grpc-go copied to clipboard
Grpc DNS resolver timing out for simple localhost lookup
What version of gRPC are you using?
1.71.1
What version of Go are you using (go version)?
1.24.2 Darwin/arm64
What operating system (Linux, Windows, …) and version?
Mac m1 Sequoia 15.5
What did you do?
This is a follow up of https://github.com/grpc/grpc-go/issues/7429#issuecomment-2905313867
Essentially tried running:
https://github.com/grpc/grpc-go/blob/master/examples/helloworld/greeter_client/main.go
But got:
rpc error: code = DeadlineExceeded desc= context deadline exceeded
This was with
localhost:4317
It worked fine with
127.0.01:4317
What did you expect to see?
I expected both localhost and 127.0.01 to work.
What did you see instead?
Only 127.0.0.1 worked
I had to remove the two DNS nameservers entries on my machine to get localhost to work. But this is not an appropriate fix because I needed root to do this and breaks other functionality that does need it.
I could have also
used passthrough://localhost:4317, but for some libraries such as OTEL exporters that use grpc it is currently not possible to pass through the scheme.
I tried net.DefaultResolver.LookupIP(ctx, 'ip', 'localhost') and it resolved with no issues quickly.
The issue seems to be that grpc DNS resolver is trying to contact the name servers even for simple /etc/hosts entries that net.DefaultResolver has no issues working with.
Ok after a deep dive into the code I can see
-
The code that does the resolution is at
https://github.com/grpc/grpc-go/blob/master/internal/resolver/dns/dns_resolver.go#L336
-
The lookupHost is working since it is using net.DefaultResolver
-
but after that it tries to do
lookupTXT on _grpc_config.localhost
That is timing out after 5 seconds with
I/o timeout
-
So question is how to disable that Txt lookup
grpc.WithDisableServiceConfig()
Is it possible to have an environment variable to turn this off, to work with problematic nameservers?
Hi @peterchenadded, there isn't a way to selectively disable service config lookups for hosts. You will need to use the WithDisableServiceConfig() dial option or provide an IP address as the target (e.g: "127.0.0.1" instead of "localhost") to bypass service config lookups. To allow users to configure gRPC channels, libraries usually accept their own options structs which they convert to gRPC DialOptions internally. GCP client libraries are an example of this pattern.
Thanks @arjan-bal problem is I don't have direct access to change the client options for the library am using e.g. otel grpc.
If there no plans to add a environment variable to disable it, we can close this issue off.
I will have to live with 127.0.0.1 for my local Mac setup.
It seems like we should probably be doing something here to help. Reopening as we discuss internally what our options are.
Thanks @dfawley, do we know what version this will be released in?
@peterchenadded we already cut the release branch for 1.74, which is happening next week. So this would make it into 1.75 which would release on/around Aug 19th.