AdguardTeam
http3 upstream connection stalls
When using an HTTP3 upstream (tested with h3://dns.google/dns-query) sometimes the dns proxy stops responding to queries.
The proxy starts working again after dns.google closes the connection due to max connection age being reached. At that point the proxy restarts a new connection to dns.google using 0-RTT.
This is happening just using this proxy as the DNS for my local machine. So far I have not been able to create a scenario for reliably reproducing the issue, but it'll usually start occurring after some time (maybe at least 20 minutes).
From the verbose logs I can see that the proxy gets stuck at: https://github.com/AdguardTeam/dnsproxy/blob/f661fdcf9edc0ddfebf6465b90d2b77314533980/upstream/doh.go#L223
@Cydox, hello and thanks for the report. Could you please collect the verbose log and goroutines profile for us? You can send both to [email protected] with the issue number in subject.
To collect the goroutines profile, enable the profiling first. This may be done either with --pprof CLI option, or by setting pprof: true within the YAML configuration file. After that, when the problem occur again, using the same host machine, request:
curl "http://127.0.0.1:6060/debug/pprof/goroutine?debug=1" > profile.txt
The same is available via any web browser.
I figured out what is going on. After a little over an hour the quic connection runs out of free streams. The reason for this is that after exactly 1 hour dns.google sends a GOAWAY frame. Despite this quic-go continues to send new requests. It looks like quic-go is not currently handling the GOAWAY frame:
https://github.com/quic-go/quic-go/blob/03a32d0daa232fef3313726e244bc7b56eb88bcc/http3/frames.go#L69
