DNS rewrites BROKEN since months

[Lovely-Maisonette]

Rewrites are ignored and queries are sent upstream instead:

DNS Query (CLI) [FAILS]

DNS rewrite config and version (docker) [OK]

Adguard log entry [Answer from upstream!!!)]

Testing from within the GUI [OK!!!]

Something in the logic got broken around v107

[fernvenue]

Maybe you can turn on verbose log and try delete this rewrite then add it again to see what's happen.

[agneevX]

Cannot reproduce this on v0.107.5.

[Lovely-Maisonette]

Maybe you can turn on verbose log and try delete this rewrite then add it again to see what's happen.

stopped, deleted all rewrites, set verbose, restarted, added just one rewrite my.test 1.2.3.4

This is the nslookup from client 192.168.1.191 (added the DOT to prevent expansion of localdomain)

PS C:\> nslookup.exe my.test. dns.casa
Server:  DNS.casa
Address:  192.168.1.10

*** DNS.casa can't find my.test.: Non-existent domain

This is the verbose log

Adguard    | 2022/04/13 10:34:37.802695 21#167 [debug] github.com/AdguardTeam/dnsproxy/proxy.(*Proxy).udpHandlePacket(): Start handling new UDP packet from 192.168.1.191:51817
Adguard    | 2022/04/13 10:34:37.802797 21#167 [debug] github.com/AdguardTeam/dnsproxy/proxy.(*Proxy).logDNSMessage(): IN: ;; opcode: QUERY, status: NOERROR, id: 1
Adguard    | ;; flags: rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
Adguard    |
Adguard    | ;; QUESTION SECTION:
Adguard    | ;10.1.168.192.in-addr.arpa.        IN       PTR
Adguard    |
Adguard    | 2022/04/13 10:34:37.803099 21#167 [debug] 192.168.1.1:53: sending request PTR 10.1.168.192.in-addr.arpa.
Adguard    | 2022/04/13 10:34:37.804852 21#167 [debug] 192.168.1.1:53: response: ok
Adguard    | 2022/04/13 10:34:37.805007 21#167 [debug] time.Duration.Milliseconds(): upstream 192.168.1.1:53 successfully finished exchange of ;10.1.168.192.in-addr.arpa.      IN       PTR. Elapsed 1.934675ms.
Adguard    | 2022/04/13 10:34:37.805118 21#167 [debug] github.com/AdguardTeam/dnsproxy/proxy.(*Proxy).replyFromUpstream(): RTT: 2.085088ms
Adguard    | 2022/04/13 10:34:37.805200 21#167 [debug] client ip: 192.168.1.191
Adguard    | 2022/04/13 10:34:37.805304 21#167 [debug] github.com/AdguardTeam/dnsproxy/proxy.(*Proxy).logDNSMessage(): OUT: ;; opcode: QUERY, status: NOERROR, id: 1
Adguard    | ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
Adguard    |
Adguard    | ;; QUESTION SECTION:
Adguard    | ;10.1.168.192.in-addr.arpa.        IN       PTR
Adguard    |
Adguard    | ;; ANSWER SECTION:
Adguard    | 10.1.168.192.in-addr.arpa. 43200   IN      PTR     DNS.casa.
Adguard    |
Adguard    | 2022/04/13 10:34:37.810543 21#168 [debug] github.com/AdguardTeam/dnsproxy/proxy.(*Proxy).udpHandlePacket(): Start handling new UDP packet from 192.168.1.191:51818
Adguard    | 2022/04/13 10:34:37.810635 21#168 [debug] github.com/AdguardTeam/dnsproxy/proxy.(*Proxy).logDNSMessage(): IN: ;; opcode: QUERY, status: NOERROR, id: 2
Adguard    | ;; flags: rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
Adguard    |
Adguard    | ;; QUESTION SECTION:
Adguard    | ;my.test.  IN       A
Adguard    |
Adguard    | 2022/04/13 10:34:37.810895 21#168 [debug] serving response from general cache
Adguard    | 2022/04/13 10:34:37.810968 21#168 [debug] client ip: 192.168.1.191
Adguard    | 2022/04/13 10:34:37.811099 21#168 [debug] github.com/AdguardTeam/dnsproxy/proxy.(*Proxy).logDNSMessage(): OUT: ;; opcode: QUERY, status: NXDOMAIN, id: 2
Adguard    | ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
Adguard    |
Adguard    | ;; QUESTION SECTION:
Adguard    | ;my.test.  IN       A
Adguard    |
Adguard    | ;; AUTHORITY SECTION:
Adguard    | .  86371   IN      SOA     a.root-servers.net. nstld.verisign-grs.com. 2022041300 1800 900 604800 86400
Adguard    |
Adguard    | 2022/04/13 10:34:37.816465 21#169 [debug] github.com/AdguardTeam/dnsproxy/proxy.(*Proxy).udpHandlePacket(): Start handling new UDP packet from 192.168.1.191:51819
Adguard    | 2022/04/13 10:34:37.816654 21#169 [debug] github.com/AdguardTeam/dnsproxy/proxy.(*Proxy).logDNSMessage(): IN: ;; opcode: QUERY, status: NOERROR, id: 3
Adguard    | ;; flags: rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
Adguard    |
Adguard    | ;; QUESTION SECTION:
Adguard    | ;my.test.  IN       AAAA
Adguard    |
Adguard    | 2022/04/13 10:34:37.816758 21#169 [debug] IPv6 is disabled. Reply with NoError to my.test. AAAA request
Adguard    | 2022/04/13 10:34:37.816837 21#169 [debug] github.com/AdguardTeam/dnsproxy/proxy.(*Proxy).logDNSMessage(): OUT: ;; opcode: QUERY, status: NOERROR, id: 3
Adguard    | ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
Adguard    |
Adguard    | ;; QUESTION SECTION:
Adguard    | ;my.test.  IN       AAAA
Adguard    |
Adguard    | ;; AUTHORITY SECTION:
Adguard    | my.test.   10      IN      SOA     fake-for-negative-caching.adguard.com. hostmaster.my.test. 100500 1800 60 604800 86400
Adguard    |

And this from the GUI

And this from the GUI filter test, which works as expected

There must be something taking precedence before the filter is applied, when the queries come from port 53 I remember something "rewrite" code being introduced around v106->v107, and then rolled back due to problems, and that rollback never fixed my issues.

I never changed my installation, only pulling regularly the latest version from < v106 times from https://hub.docker.com/r/adguard/adguardhome

[fernvenue]

@Lovely-Maisonette Does this client use the global settings and upstream?

[Lovely-Maisonette]

I use my router as upstream for internal name resolution (is not enough, an therefore I need the rewrite to add CNAMES. What is not internal should go outside. This is relevant config:

upstream_dns:
  - '[/casa/]192.168.1.1'
  - '[/168.192.in-addr.arpa/]192.168.1.1'
  - https://dns.cloudflare.com/dns-query
  - 1.1.1.1
  upstream_dns_file: ""
  bootstrap_dns:
  - 9.9.9.10
  - 149.112.112.10
  - 2620:fe::10
  - 2620:fe::fe:10

But the rewrite should bypass any upstreams.

I am not sure what you mean by "global settings"

You think there is a missconfiguration? The rewrites stopped working after an update around one year ago I would say, and without changing the configuration.

[fernvenue]

the rewrite should precede any upstreams.

Of course :)

I am not sure what you mean by "global settings"

Settings - Client settings

[Lovely-Maisonette]

yes, global settings. I have not defined any specific client settings for this or any other ip. I moved to adguard/adguardhome:edge long ago, in the hope a fix would roll in.

[ainar-g]

@Lovely-Maisonette, we cannot reproduce that, but we've had several similar issues. A few questions that would help us diagnose the issue:

1. Do you have the “Block domains using filters and hosts files” checkmark checked on the Settings → General settings page?
2. I've noticed that you're using Windows. Have you tried the same nslookup but with a dot at the end of the hostname? E.g. nslookup my.test..
3. Does your router send DHCP Option 119 aka Domain Suffix Search?
4. If you're using AdGuard Home as your DHCP server, what's the value of the local_domain_name parameter?

Thanks.

[Lovely-Maisonette]

Hi @ainar-g,

I will speak in the past, because I changed some things drastically in the last couple of hours.

1. Yes, I used the blocking lists as provided by Adward out of-the-box.
2. Normally I am on Linux (so did not work there), but to build the report today I was on windows. Yes, I did both, with (second post) and without dot (first post) to minimize having a transient localdoman expansion.

My router is my DHCP server and main name resolution for local hosts. The DHCP server tells the clients to use the domain casa (it is now a TLD, but it was not when I started using it. I use it for my 192.168. and 10. networks. Adguard is my main DNS server (router the secondary/fallback) Adguard retrievs external names from outside servers, and internal names from the DNS/DHCP router.

I have internal services that should be isolated from each other, those get their own networks and the router does not know about them. Some of them are reachable through a reverse proxy (frontend endpoints only). The reverse proxy needs to be reached by it's own name and a multitude of CNAMES, to route requests to those internal services. I used the rewrites to provide those aliases. I do not know if there is a way to automate that the same way I did with the other servers on 192.168. but did not care, they are in the dozens, so I could maintain them by hand in the rewrite page (that page is missing some EDIT button).

Today I could not wait any longer and started looking for a fix or an alternative. At the end I deleted everything and started over. In parallel I learned from something you wrote elsewhere about dnsrewrite and made it work using the "Custom filtering rules" (that name fooled me, never looked in there before) instead of the "rewrites" (intuitively, the right place to look for creating CNAMES).

Now I have:

elastic.casa^$dnsrewrite=NOERROR;CNAME;web.casa portainer.casa^$dnsrewrite=NOERROR;CNAME;web.casa foo.casa^$dnsrewrite=NOERROR;CNAME;web.casa bar.casa^$dnsrewrite=NOERROR;CNAME;web.casa ...

This works and is good enough for my needs. Unfortunately, trying to make it work, I did not think about keeping the old instance and config to help you debug this further, sorry for my shortsightedness.

I was not able to remove the .casa from there (changing the domain on the router will break it up) but as long as |I do not need to scale into the hundreds of server, I do not mind, though all I need is a simple elastic, portainer, foo, bar -> web

The rewrite bug is still there, but maybe we need to close this ticket as "cannot reproduce" until further notice.

[ainar-g]

I see, thanks for the feedback. We are actually planning to eventually remove the legacy DNS rewrites completely, and we've been recommending users to switch to dnsrewrite rules for a while now. Perhaps we should add a UI tip about that.

[ElStupid]

How to do dnsrewrite rules?

I'm experiencing the same kind of problems as mentioned here, although it did work for short time. I only have 1 DNS server defined, which is AdGuard Home. Now when I query for a DNS rewrite address (which is something like sub.domain.local) the logging show it's upstreamed and NXDOMAIN. Even when caching is switched off by setting it to 0 size. I'm running v0.107.5 on my NAS in a Docker container btw.

If exchanging the legacy DNS rewrite for dnsrewrite rules makes life easier then please do so. But the interface is quite intuitive and I would be sad to see it go...

[Lovely-Maisonette]

I am not part of the team but,

How to do dnsrewrite rules?

What was not working (being set upstream, instead of resolving like in the internal "check filter") for me was the Filters/DNS Rewrites.

If I understand correctly, that is what @ainar-g calls "legacy" and to be removed sooner or later (hopefully not the other way around, not clear to me. @ainar-g?) I discovered a second way to make rewrites a few days ago (after years of using Adward, not intuitive). in the Filters/Custom Filtering Rules (With than name sounds like the exception, not the rule) I would say, it needs better documentation. I stopped when this worked for me:

portainer.casa^$dnsrewrite=NOERROR;CNAME;web.casa

Do not ask me to document it, or to know what happens if I remove CNAME, NOERROR, or ^$ which looks like an empty regular expression. IT worked for me and I moved on. Hopefully someone can expand on this: https://github.com/AdguardTeam/AdGuardHome/wiki/Hosts-Blocklists

Still not clear why it is documented in the Blocklist. Probably GUI and Docs need revamping there.

[ElStupid]

Exactly. I guess your finding in the custom filtering rules is the "dnswrite" @ainar-g is talking about. With some very good documentation that would be doable also (I don't make lots of them and they don't change often). But it would be great to do it more intuitive through the current interface.

Thanks for pointing it out, I will give it a try.

[RossComputerGuy]

I think I'm having the same issue when doing DNS rewrites with Adguard. I get responses when I query a rewritten URL but when I use Tailscale, I get NXDOMAIN. Is this a known issue and is it related to this? I'm running Adguard Home off TrueNAS.

[dnnspaul]

It worked for several weeks after installation, but it "suddendly" stopped working for now multiple months. I read this whole thread and enabling the option for filtering the domains with filters and hosts-file made it working again:

(sorry, but I'm using the german version. It's the first setting at "Settings -> General".) I'm using the linuxserver adguard docker image.

[RossComputerGuy]

@dnnspaul I had the same problem where it just suddenly stopped working even without using Tailscale but your solution fixed it.

[zfedoran]

Same issue here, worked for a while, then stopped working for one rewrite after another. At this point half work and half do not. All work when tested through the adguard web GUI test but not when testing from the terminal or other device on the same network. These used to work but have stopped working.

@dnnspaul fix didn't work as I already had this setting enabled. Flipping it on and off doesn't do anything either.

[kvervo]

I was about to write a comment for this issue with details and steps to reproduce it :) And then I found that in my network I had the AdBlock feature from the router vendor enabled on the router. This caused the issue and although all clients reported that DNS server was configured properly from DHCP, still the router somehow override that and forced it's own instance of DNS to take over for all client requests.

Once the feature was disable everything got back to normal.

[zfedoran]

@kvervo which type of router are you using? I think it would be helpful to share in case anyone else has that issue

[dominikhoebert]

Same for me, it worked fine until a few days ago and now most of them don't work anymore. All of them work through the AG web gui test. Did somebody find a solution in the meantime?

[wehrstedt]

Got the same problem today. Im running adguard in a docker container. Today, dns rewrite stopped working and a lot of my local domains could not be resolved. I did not changed anything.

I opened the adguard dashboard to debug this issue. First thing i noticed was that the query log were disabled. 2 days ago the query log were enabled. I did not disabled it by myself. After enabling the query log i could see my local dns queries were forwarded to my upstream dns servers even dns rewrites were active and well configured.

After a while i diceded to launch a new, fresh docker container. DNS rewrite was working again. I relaunched my old container and noticed that dns blocking also does not work. I than rechecked the configuration and noticed that the option "Block domains using filters and host files" were disabled. After reactivating this option everything worked again.

I dont know what happened to the configuration. I did not changed anything at all.

[wehrstedt]

In the adguard logs:

2023/03/16 22:49:04.687319 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Receive-Buffer-Size for details.

Maybe there is a memory leak which causes this issue?

[Cybolic]

I was having the same problem: DNS rewrites worked fine on the local network (i.e. nslookup myhost.local.net 192.168.x.x worked), but failed whenever I used Tailscale (i.e. nslookup myhost.local.net 100.x.x.x => NXDOMAIN).

What finally made it work was adding Tailscale's network to the dns.private_networks config setting:

dns:
  private_networks:
    # RFC 1918
    - 10.0.0.0/8
    - 172.16.0.0/12
    - 192.168.0.0/16
    # Tailscale
    - 100.64.0.0/10

For reference, I saw no difference between using DNS rewrites vs custom filtering rules and the "Block domains using filters and hosts files" setting didn't have any effect either.

[Wetzel402]

It worked for several weeks after installation, but it "suddendly" stopped working for now multiple months. I read this whole thread and enabling the option for filtering the domains with filters and hosts-file made it working again:

(sorry, but I'm using the german version. It's the first setting at "Settings -> General".) I'm using the linuxserver adguard docker image.

This was my problem! Thanks!