AdGuardHome icon indicating copy to clipboard operation
AdGuardHome copied to clipboard

Timeout errors - Response recevied over udp

Open renatoyamane opened this issue 1 year ago • 13 comments

Prerequisites

Platform (OS and CPU architecture)

Linux, ARMv7

Installation

Custom package (OpenWrt, HomeAssistant, etc; please mention in the description)

Setup

On a router, DHCP is handled by the router

AdGuard Home version

0.107.51

Action

After upgrading from 0.107.50 to 0.107.51 I started to see these errors below:

Fri Jun  7 08:43:20 2024 daemon.warn dnsmasq[4731]: nameserver 127.0.0.1 refused to do a recursive query
Fri Jun  7 09:14:21 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:14:21.508565 [error] dnsproxy: 127.0.0.1:53: response received over udp: "exchanging with 127.0.0.1:53 over udp: read udp 127.0.0.1:59025->127.0.0.1:53: i/o timeout"
Fri Jun  7 09:14:21 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:14:21.508706 [error] dnsproxy: upstream 127.0.0.1:53 failed to exchange ;150.6.168.192.in-addr.arpa.	IN	 SOA in 2.001038427s: exchanging with 127.0.0.1:53 over udp: read udp 127.0.0.1:59025->127.0.0.1:53: i/o timeout
Fri Jun  7 09:14:21 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:14:21.508565 [error] dnsproxy: 127.0.0.1:53: response received over udp: "exchanging with 127.0.0.1:53 over udp: read udp 127.0.0.1:56506->127.0.0.1:53: i/o timeout"
Fri Jun  7 09:14:21 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:14:21.508776 [error] dnsproxy: upstream 127.0.0.1:53 failed to exchange ;150.6.168.192.in-addr.arpa.	IN	 SOA in 2.001723775s: exchanging with 127.0.0.1:53 over udp: read udp 127.0.0.1:56506->127.0.0.1:53: i/o timeout
Fri Jun  7 09:14:24 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:14:24.433914 [error] dnsproxy: [::1]:53: response received over udp: "exchanging with [::1]:53 over udp: read udp [::1]:46063->[::1]:53: i/o timeout"
Fri Jun  7 09:14:24 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:14:24.434041 [error] dnsproxy: upstream [::1]:53 failed to exchange ;150.6.168.192.in-addr.arpa.	IN	 SOA in 2.001156477s: exchanging with [::1]:53 over udp: read udp [::1]:46063->[::1]:53: i/o timeout
Fri Jun  7 09:14:24 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:14:24.434826 [error] dnsproxy: [::1]:53: response received over udp: "exchanging with [::1]:53 over udp: read udp [::1]:42492->[::1]:53: i/o timeout"
Fri Jun  7 09:14:24 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:14:24.434916 [error] dnsproxy: upstream [::1]:53 failed to exchange ;150.6.168.192.in-addr.arpa.	IN	 SOA in 2.00137525s: exchanging with [::1]:53 over udp: read udp [::1]:42492->[::1]:53: i/o timeout
Fri Jun  7 09:15:51 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:15:51.106272 [error] dnsproxy: [::1]:53: response received over udp: "exchanging with [::1]:53 over udp: read udp [::1]:55499->[::1]:53: i/o timeout"
Fri Jun  7 09:15:51 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:15:51.106426 [error] dnsproxy: upstream [::1]:53 failed to exchange ;150.6.168.192.in-addr.arpa.	IN	 SOA in 2.000838985s: exchanging with [::1]:53 over udp: read udp [::1]:55499->[::1]:53: i/o timeout
Fri Jun  7 09:15:51 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:15:51.106272 [error] dnsproxy: 127.0.0.1:53: response received over udp: "exchanging with 127.0.0.1:53 over udp: read udp 127.0.0.1:44697->127.0.0.1:53: i/o timeout"
Fri Jun  7 09:15:51 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:15:51.106504 [error] dnsproxy: upstream 127.0.0.1:53 failed to exchange ;150.6.168.192.in-addr.arpa.	IN	 SOA in 2.001481132s: exchanging with 127.0.0.1:53 over udp: read udp 127.0.0.1:44697->127.0.0.1:53: i/o timeout
Fri Jun  7 09:15:53 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:15:53.255717 [error] dnsproxy: [::1]:53: response received over udp: "exchanging with [::1]:53 over udp: read udp [::1]:33618->[::1]:53: i/o timeout"
Fri Jun  7 09:15:53 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:15:53.255815 [error] dnsproxy: upstream [::1]:53 failed to exchange ;150.6.168.192.in-addr.arpa.	IN	 SOA in 2.001736635s: exchanging with [::1]:53 over udp: read udp [::1]:33618->[::1]:53: i/o timeout
Fri Jun  7 09:15:53 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:15:53.256082 [error] dnsproxy: 127.0.0.1:53: response received over udp: "exchanging with 127.0.0.1:53 over udp: read udp 127.0.0.1:49044->127.0.0.1:53: i/o timeout"
Fri Jun  7 09:15:53 2024 user.notice AdGuardHome[9325]: 2024/06/07 08:15:53.256130 [error] dnsproxy: upstream 127.0.0.1:53 failed to exchange ;150.6.168.192.in-addr.arpa.	IN	 SOA in 2.001394808s: exchanging with 127.0.0.1:53 over udp: read udp 127.0.0.1:49044->127.0.0.1:53: i/o timeout

Note also the IP address looks reversed: What I see on the log: 150.6.168.192 What probably should be: 192.168.6.150

Upstream DNS servers (Load balancing mode):

https://dns.quad9.net/dns-query
https://doh.opendns.com/dns-query
https://security.cloudflare-dns.com/dns-query
tls://security.cloudflare-dns.com
quic://dns.adguard-dns.com
quic://zero.dns0.eu
https://dns.adguard-dns.com/dns-query
tls://dns.adguard-dns.com
h3://cloudflare-dns.com/dns-query
# Cisco OpenDNS:
sdns://AQAAAAAAAAAADjIwOC42Ny4yMjAuMjIwILc1EUAgbyJdPivYItf9aR6hwzzI1maNDL4Ev6vKQ_t5GzIuZG5zY3J5cHQtY2VydC5vcGVuZG5zLmNvbQ
# Quad9:
sdns://AQMAAAAAAAAADDkuOS45Ljk6ODQ0MyBnyEe4yHWM0SAkVUO-dWdG3zTfHYTAC4xHA2jfgh2GPhkyLmRuc2NyeXB0LWNlcnQucXVhZDkubmV0
# AdGuard:
sdns://AgMAAAAAAAAADDk0LjE0MC4xNC4xNCCaOjT3J965vKUQA9nOnDn48n3ZxSQpAcK6saROY1oCGQw5NC4xNDAuMTQuMTQKL2Rucy1xdWVyeQ
# Comodo Secure DNS:
sdns://AQAAAAAAAAAACjguMjAuMjQ3LjIg0sJUqpYcHsoXmZb1X7yAHwg2xyN5q1J-zaiGG-Dgs7AoMi5kbnNjcnlwdC1jZXJ0LnNoaWVsZC0yLmRuc2J5Y29tb2RvLmNvbQ
quic://x-hagezi-proplus.freedns.controld.com
quic://x-1hosts-lite.freedns.controld.com

Bootstrap DNS servers:

208.67.222.222
1.1.1.1
208.67.220.220
9.9.9.9
8.8.8.8
149.112.112.10
94.140.15.15
2a10:50c0::ad1:ff
94.140.14.14
2a10:50c0::ad2:ff
[2a10:50c0::ad1:ff]:5443
2001:4860:4860:0:0:0:0:6464
76.76.2.42
45.90.28.128

Bug 6897 is still happening on this version

Expected result

No erros

Actual result

Erros reported above

Additional information and/or screenshots

No response

renatoyamane avatar Jun 07 '24 15:06 renatoyamane

I'm having the exact same problem if I have Unbound as my Upstream DNS. Watchtower container updated my ADH and after that, everything stopped working. So I'm not sure what version it was previously.

This issue started yesterday. I've tried to roll back 1 version but it didn't help. I'm having to use a public dns server in "Upstream DNS Server" in order to have ADH to work.

diegoweb avatar Jun 08 '24 22:06 diegoweb

Hi @renatoyamane

The IPs don't look reversed, this is how PTR records (reverse DNS lookups) are declared. If you look at the log, you'll notice that the timeout is actually coming from 127.0.0.1:53 which I assume you've declared in the "Private reverse DNS servers" section.

The thing is that something is quering for the SOA registry, which I guess is not implemented in the DNS Server that you're using as private reverse resolver ending with a timout of the query.

Pushing SOA requests to the upstream private resolvers whas added in v0.107.49 so this entries have probably been showing up in the log since that release.

Cebeerre avatar Jun 10 '24 06:06 Cebeerre

I'm having the exact same problem if I have Unbound as my Upstream DNS. Watchtower container updated my ADH and after that, everything stopped working. So I'm not sure what version it was previously.

This issue started yesterday. I've tried to roll back 1 version but it didn't help. I'm having to use a public dns server in "Upstream DNS Server" in order to have ADH to work.

Your issue sounds like a completely different thing. If you're starting unbound as a container as well, you might be having conectivity issues between the two containers. Check how you configured the networks if you're using the internal docker DNS resolver to declare Unbound in the AGH config.

Cebeerre avatar Jun 10 '24 06:06 Cebeerre

Hi @renatoyamane

The IPs don't look reversed, this is how PTR records (reverse DNS lookups) are declared. If you look at the log, you'll notice that the timeout is actually coming from 127.0.0.1:53 which I assume you've declared in the "Private reverse DNS servers" section.

The thing is that something is quering for the SOA registry, which I guess is not implemented in the DNS Server that you're using as private reverse resolver ending with a timout of the query.

Pushing SOA requests to the upstream private resolvers whas added in v0.107.49 so this entries have probably been showing up in the log since that release.

Thanks. My settings are: reverse

So, should I change the empty box (Private reverse DNS servers) to the IP of the router where AdGuard is installed (like: 192.168.1.1)?

renatoyamane avatar Jun 13 '24 10:06 renatoyamane

I am seeing the exact same error in my logs (only with different IPs) as well (AGH 0.107.52 on a Raspi 4b, our DSL router's DNS pointing to AGH, and AGH querying DOH servers from AdGuard directly). If this race starts, AGH becomes unresponsive on the DNS side (can still reach the web frontend), AGH spams the log file, and DNS queries start to fail. Currently I have to restart AGH every five minutes to avoid this behaviour. Is there a fix?

jensb avatar Sep 15 '24 08:09 jensb

also seeing this issue in v0.107.52 though more of a singular issue with

adguard               | 2024/09/18 18:32:41.404793 [error] dnsproxy: 192.168.178.1:53: response received over udp: "exchanging with 192.168.178.1:53 over udp: read udp 192.168.178.5:59000->192.168.178.1:53: i/o timeout"
adguard               | 2024/09/18 18:32:41.405103 [error] dnsproxy: 192.168.178.1:53: response received over udp: "exchanging with 192.168.178.1:53 over udp: read udp 192.168.178.5:38472->192.168.178.1:53: i/o timeout"
adguard               | 2024/09/18 18:32:41.408214 [error] dnsproxy: 192.168.178.1:53: response received over udp: "exchanging with 192.168.178.1:53 over udp: read udp 192.168.178.5:57142->192.168.178.1:53: i/o timeout"
adguard               | 2024/09/18 18:32:41.417714 [error] dnsproxy: 192.168.178.1:53: response received over udp: "exchanging with 192.168.178.1:53 over udp: read udp 192.168.178.5:39264->192.168.178.1:53: i/o timeout"
adguard               | 2024/09/18 18:32:41.423475 [error] dnsproxy: 192.168.178.1:53: response received over udp: "exchanging with 192.168.178.1:53 over udp: read udp 192.168.178.5:39804->192.168.178.1:53: i/o timeout"
adguard               | 2024/09/18 18:32:41.425151 [error] dnsproxy: 192.168.178.1:53: response received over udp: "exchanging with 192.168.178.1:53 over udp: read udp 192.168.178.5:41646->192.168.178.1:53: i/o timeout"
adguard               | 2024/09/18 18:32:41.427218 [error] dnsproxy: 192.168.178.1:53: response received over udp: "exchanging with 192.168.178.1:53 over udp: read udp 192.168.178.5:37605->192.168.178.1:53: i/o timeout"
adguard               | 2024/09/18 18:32:41.437899 [error] dnsproxy: 192.168.178.1:53: response received over udp: "exchanging with 192.168.178.1:53 over udp: read udp 192.168.178.5:36153->192.168.178.1:53: i/o timeout"
adguard               | 2024/09/18 18:32:41.439622 [error] dnsproxy: 192.168.178.1:53: response received over udp: "exchanging with 192.168.178.1:53 over udp: read udp 192.168.178.5:48392->192.168.178.1:53: i/o timeout"
adguard               | 2024/09/18 18:32:41.450249 [error] dnsproxy: 192.168.178.1:53: response received over udp: "exchanging with 192.168.178.1:53 over udp: read udp 192.168.178.5:52336->192.168.178.1:53: i/o timeout"

the routers dns being on .1:53 and adguard+unbound on .5 (unbound port 5053, adguard 53)

AlexanderSalge avatar Sep 18 '24 16:09 AlexanderSalge

The issue is still there, disabling private DNS resolvers helps.

hydroid7 avatar Jan 06 '25 10:01 hydroid7

Possibly. However, for our LAN disabling private resolvers is unfortunately not an option. Please fix this even when using private resolvers.

jensb avatar Jan 06 '25 12:01 jensb

Very glad I found this, disabling private DNS resolvers worked for me. However, I would love to see this fixed, not sure how to help with data tho

Cilenco avatar Feb 21 '25 18:02 Cilenco

I had the same error. I checked my upstream dot dns servers (i.e. dot.website.com) and realized that my bootstrap dns servers have the same IP than the upstream servers. After switching bootstrap servers to non-upstream server the errors are gone . :)

domini1000-git avatar Mar 21 '25 21:03 domini1000-git

I had the same error. I checked my upstream dot dns servers (i.e. dot.website.com) and realized that my bootstrap dns servers have the same IP than the upstream servers. After switching bootstrap servers to non-upstream server the errors are gone . :)

I've noticed the same, but I'm struggling to understand why this happens. Do we get throttled by the upstream server due to a burst or requests?

gshpychka avatar May 30 '25 14:05 gshpychka

I had the same error. I checked my upstream dot dns servers (i.e. dot.website.com) and realized that my bootstrap dns servers have the same IP than the upstream servers. After switching bootstrap servers to non-upstream server the errors are gone . :)

Could you explain what you mean and how you fixed it? Sorry I have been having what I think is the same issue but struggling to work out whats going on!

toeachbyeach avatar Jun 09 '25 10:06 toeachbyeach

I had the same error. I checked my upstream dot dns servers (i.e. dot.website.com) and realized that my bootstrap dns servers have the same IP than the upstream servers. After switching bootstrap servers to non-upstream server the errors are gone . :)

Could you explain what you mean and how you fixed it? Sorry I have been having what I think is the same issue but struggling to work out whats going on!

Made sure the bootstrap server IPs aren't used as any of the main upstream servers.

gshpychka avatar Jun 09 '25 10:06 gshpychka