grpc-go icon indicating copy to clipboard operation
grpc-go copied to clipboard

grpc.Dial fails to connect to unix domain socket when https_proxy is set

Open Ongy opened this issue 8 months ago • 1 comments

NOTE: if you are reporting is a potential security vulnerability or a crash, please follow our CVE process at https://github.com/grpc/proposal/blob/master/P4-grpc-cve-process.md instead of filing an issue here.

Please see the FAQ in our main README.md, then answer the questions below before submitting your issue.

What version of gRPC are you using?

google.golang.org/grpc v1.71.0

What version of Go are you using (go version)?

go version go1.23.6 linux/amd64

What operating system (Linux, Windows, …) and version?

Linux (glinux, recent)

What did you do?

If possible, provide a recipe for reproducing the error.

Repro is available in https://github.com/Ongy/go-grpc-repro/commit/3c23732ddd07aad52cd4bd16beb4d3cfe857b38f

The repro is quite minimal and requires introspection of the debug output to recognize the error.

What did you expect to see?

I expect Dial to try and connect to a unix:///... domain socket and then fail due to some error of type "file doesn't exist"

What did you see instead?

By looking at the debug output, I can determine that the proxy logic picks up the https_proxy for a URL of schema unix://. While this just leads to a different type of error in the repro, this leads to a real error in an application where the unix socket does exist.

❯ GRPC_GO_LOG_VERBOSITY_LEVEL=99 GRPC_GO_LOG_SEVERITY_LEVEL=info go run ./...
2025/03/28 15:17:16 INFO: [core] original dial target is: "unix:///var/run/test.sock"
2025/03/28 15:17:16 INFO: [core] [Channel #1]Channel created
2025/03/28 15:17:16 INFO: [core] [Channel #1]parsed dial target is: resolver.Target{URL:url.URL{Scheme:"unix", Opaque:"", User:(*url.Userinfo)(nil), Host:"", Path:"/var/run/test.sock", RawPath:"", OmitHost:false, ForceQuery:false, RawQuery:"", Fragment:"", RawFragment:""}}
2025/03/28 15:17:16 INFO: [core] [Channel #1]Channel authority set to "localhost"
2025/03/28 15:17:16 INFO: [delegating-resolver] Proxy URL detected : http://127.0.0.1
2025/03/28 15:17:16 INFO: [delegating-resolver] Addresses received from target resolver: [{Addr: "/var/run/test.sock", ServerName: "", Attributes: {"<%!p(networktype.keyType=grpc.internal.transport.networktype)>": "unix" }, }]
2025/03/28 15:17:16 INFO: [delegating-resolver] Addresses received from proxy resolver: [{Addr: "127.0.0.1:443", ServerName: "", }]
2025/03/28 15:17:16 INFO: [core] [Channel #1]Resolver state updated: {
  "Addresses": [
    {
      "Addr": "127.0.0.1:443",
      "ServerName": "",
      "Attributes": {
        "\u003c%!p(proxyattributes.keyType=grpc.resolver.delegatingresolver.proxyOptions)\u003e": "\u003c%!p(proxyattributes.Options={\u003cnil\u003e /var/run/test.sock})\u003e"
      },
      "BalancerAttributes": null,
      "Metadata": null
    }
  ],
  "Endpoints": [
    {
      "Addresses": [
        {
          "Addr": "127.0.0.1:443",
          "ServerName": "",
          "Attributes": {
            "\u003c%!p(proxyattributes.keyType=grpc.resolver.delegatingresolver.proxyOptions)\u003e": "\u003c%!p(proxyattributes.Options={\u003cnil\u003e /var/run/test.sock})\u003e"
          },
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    }
  ],
  "ServiceConfig": null,
  "Attributes": null
} (resolver returned new addresses)

Ongy avatar Mar 28 '25 14:03 Ongy

Thanks for reporting this.

@eshitachandwani probably the right fix is to also avoid the proxy from inside the delegating resolver when the target resolver produces addresses whose networktype is anything that is not "tcp".

https://github.com/grpc/grpc-go/blob/6819ed796fcd0232a46dab21c1b7826aa7f1d561/internal/transport/networktype/networktype.go#L40

That's how the transport decides to use a different dialer:

https://github.com/grpc/grpc-go/blob/6819ed796fcd0232a46dab21c1b7826aa7f1d561/internal/transport/http2_client.go#L159-L177

In this case, we shouldn't even wait for the proxy resolver to resolve before producing those addresses.

And we should make sure to add a test case for this scenario.

dfawley avatar Mar 28 '25 15:03 dfawley

As a quick fix until it's fixed you can look into grpc.WithNoProxy dial option. Also, it's possible to implement custom address resolver (a bit more complex).

anjmao avatar Mar 31 '25 15:03 anjmao

It looks like the only place that sets the network type attribute on the address is the unix resolver.

easwars avatar Apr 11 '25 16:04 easwars

gRFC A1 clearly states that it is a specification of the different TCP-level proxy setups that we support. But given that this support is driven by environment variables in Go, we should also look into clearly documenting that our support is only for TCP-level proxies. Maybe in the proxy example or in the proxy.md?

easwars avatar Apr 11 '25 17:04 easwars

@eshitachandwani we would need to cherry-pick the fix into v1.72.x for a patch release.

arjan-bal avatar May 07 '25 07:05 arjan-bal