pdns recursor: return root-hints for IN NS . queries without rd bit for the sake of dig +trace

Program: Recursor
Issue type: Feature request

Short description

Consider returning root-hins for questions for IN NS . without the rd bit set if nothing is in cache for the sake of dig +trace.

Usecase

dig +trace sld.tld. does not work reliably with newer PowerDNS recursor versions (>5) even if allow_no_rd is set.

Description

On PowerDNS recursors that see low usage the initial query that dig sld.tld. +trace does - which seems to be essentially dig ns . +norecurse for some versions of dig (others apparently do set the rd bit) - no answer section is returned even if the allow_no_rd option is set causing the trace to fail. This seems to be dependent on the contents of the cache so returning root-hints would be preferred in those cases to make dig +trace work reliably.

May 30 '24 13:05 ZaphodB

IMO, the initial query done by dig +trace should be directed to a server authoritative for root. So there's a very simple workaround: use dig @some-letter.root-servers.net +trace some.name.

May 30 '24 13:05 omoerbeek

Thank you, that is a good workaround. The dig default is however to use the systems recursive nameservers and i fear some people might not realize this option. It would certainly be a convenience feature.

May 30 '24 13:05 ZaphodB

On PowerDNS recursors that see low usage the initial query that dig sld.tld. +trace does - which seems to be essentially dig ns . +norecurse for some versions of dig (others apparently do set the rd bit) - no answer section is returned even if the allow_no_rd option is set causing the trace to fail. This seems to be dependent on the contents of the cache so returning root-hints would be preferred in those cases to make dig +trace work reliably.

You can reliably reproduce this? It seems like it should be extremely rare that your resolver doesn't have the root cached/primed. Even if you wipe '.' before a query this query, the cache is reprimed and the correct answer provided.

May 30 '24 20:05 phonedph1

dig +trace was fixed in BIND 9.15.1. https://gitlab.isc.org/isc-projects/bind9/-/issues/1028

I'd like to hope most systems are on 9.16 (which is itself going EOL) or newer, but idk.

May 30 '24 23:05 mnordhoff

➜  ~ dig -v
DiG 9.10.6

on MacOS 14.5.

I am able to semi-reliably reproduce the issue. When it occurs after restart of the process It seems to persist until the first recursive queries hit the recursor + TTL of the packet cache.

In a diff -au between dump-cache outputs from when the issue occurs (-) vs. after it (+) seems that the root zone is not known to the cache when the issue occurs:

+; Zone .
+. 86357 IN NSEC aaa. NS SOA RRSIG NSEC DNSKEY ZONEMD
+- RRSIG NSEC 8 0 86400 20240613050000 20240531040000 5613 . aw+AKvv4BHeQ5YIrxJt9xKazcxrxIvmDgjcRnuqlrdrUhi/pYRKg1e1bORaoyQgYsm+a7kJDjKJwgyKkhDr1A6Bk+cae3Om8M/iV22iXMIRNBYUnKuUCwhlVPn+Z7uAnUcX/ltqYPBQvDAOu8KK630e5tuus/IwvxpBIqy+LNOkXk+lWiKXhJkabi12HIFRZqsHVBR7ZsFrxPfw5MACBdNcUylWIWZX0DaPs8Htr+KOa36VDA060/G6fDIux2qDnrWbfkYV+L4uomI+g50I6qtZQRfiMD0PJ7ZTVp+Qq56XGT3BWet396+zGs5M59bVUotyFXhczNx29VRuKnnFdgg==

which seems to coincide with the root getting refreshed.

 ; record cache shard 200; size 0
-; record cache shard 201; size 2
-. 86400 86383 IN NS a.root-servers.net. ; (Secure) auth=1 zone=. from=192.58.128.30 nm= rtag= ss=0
-. 86400 86383 IN NS b.root-servers.net. ; (Secure) auth=1 zone=. from=192.58.128.30 nm= rtag= ss=0
-. 86400 86383 IN NS c.root-servers.net. ; (Secure) auth=1 zone=. from=192.58.128.30 nm= rtag= ss=0
-. 86400 86383 IN NS d.root-servers.net. ; (Secure) auth=1 zone=. from=192.58.128.30 nm= rtag= ss=0
-. 86400 86383 IN NS e.root-servers.net. ; (Secure) auth=1 zone=. from=192.58.128.30 nm= rtag= ss=0
-. 86400 86383 IN NS f.root-servers.net. ; (Secure) auth=1 zone=. from=192.58.128.30 nm= rtag= ss=0
-. 86400 86383 IN NS g.root-servers.net. ; (Secure) auth=1 zone=. from=192.58.128.30 nm= rtag= ss=0
-. 86400 86383 IN NS h.root-servers.net. ; (Secure) auth=1 zone=. from=192.58.128.30 nm= rtag= ss=0
-. 86400 86383 IN NS i.root-servers.net. ; (Secure) auth=1 zone=. from=192.58.128.30 nm= rtag= ss=0
-. 86400 86383 IN NS j.root-servers.net. ; (Secure) auth=1 zone=. from=192.58.128.30 nm= rtag= ss=0
-. 86400 86383 IN NS k.root-servers.net. ; (Secure) auth=1 zone=. from=192.58.128.30 nm= rtag= ss=0
-. 86400 86383 IN NS l.root-servers.net. ; (Secure) auth=1 zone=. from=192.58.128.30 nm= rtag= ss=0
-. 86400 86383 IN NS m.root-servers.net. ; (Secure) auth=1 zone=. from=192.58.128.30 nm= rtag= ss=0
-. 86400 86383 IN RRSIG NS 8 0 518400 20240613050000 20240531040000 5613 . ZZBvE/SXQlUxxVB7GIom11v38bXbqUgrpIpzZ5kVBVaSgZhwAp3wypThnD8LoxRcS91IT7zw1N6r4+mD8FT6vwdVRfzx8TkaLgbvzo0zzi8JbVefcdi2oHpqEdTKWG2cG6UwO5P5FGj8PSLnF7mW2tsJq13z2HmCCyoXk6UYSwfKmTC7oHCc8cQsdVqFzsIBDYp5FvQF5LLRsXZgXyD5PBI72ca6LIXDYrdwUpLE2Jgd9q426aVJV3ig6svG5Do8pqTC5hvrihx6wHIu2lJVpeqYUkRNOIpEGWtzZk6MvDJBV08oiG5CzDStmDHCyTa5qLS6fJS7n7DrBZCLZVirtg== ;
-. 86400 86383 IN DNSKEY 257 3 8 AwEAAaz/tAm8yTn4Mfeh5eyI96WSVexTBAvkMgJzkKTOiW1vkIbzxeF3+/4RgWOq7HrxRixHlFlExOLAJr5emLvN7SWXgnLh4+B5xQlNVz8Og8kvArMtNROxVQuCaSnIDdD5LKyWbRd2n9WGe2R8PzgCmr3EgVLrjyBxWezF0jLHwVN8efS3rCj/EWgvIWgb9tarpVUDK/b58Da+sqqls3eNbuv7pr+eoZG+SrDK6nWeL3c6H5Apxz7LjVc1uTIdsIXxuOLYA4/ilBmSVIzuDWfdRUfhHdY6+cn8HFRm+2hM8AnXGXws9555KrUB5qihylGa8subX2Nn6UwNR1AkUTV74bU= ; (Secure) auth=1 zone=. from=202.12.27.33 nm= rtag= ss=0
-. 86400 86383 IN DNSKEY 256 3 8 AwEAAZBALoOFImwcJJg9Iu7Vy7ZyLjhtXfvO1c9k4vHjOpf9i7U1kKtrBvhnwsOni1sb50gkUayRtMDTUQqvljMMf4bpkyEtcE5evCzhHbFLq1coL5QOix3mfJm++FvIMaAt52nOvAdqR/luuI11bA1AmSCIJKAUx147DcfOHYKg3as+dznn3Iah4cWBMVzDe7PPsFS1AO6gU8EpmiRJ9VMNA09fOyDuq9+d6sw8UUnJRMAFAuPLhUFjUAOuWOw74BC9lOtMQpbLMz8pX0CDKdOXDHjyj61nxSSWxPdUjeoxI17lQTpSPRtqRHFn5Fgj2e+9BVwhhWGDQN8kUVSJHZtQiI0= ; (Secure) auth=1 zone=. from=202.12.27.33 nm= rtag= ss=0
-. 86400 86383 IN RRSIG DNSKEY 8 0 172800 20240621000000 20240531000000 20326 . axzhlvsjvXefBmyQv7pZSJsEJ5BW2poXC95oU5D4itI8yejNJmehpcjBlVodnaqSQ0nNRVnvhVqC0eGzwOc/A+DCIS8tt+w93GIHhMAIkydLuxGofblreMIdD2/WNP4QQgCf4H7ANDNURozqDaFg9rq+GEHBBucHL4jt1t3dxcDcOyKaEo+2Z28NjsucYdqAENVm0Gcy9AJ4bd/a5N7DuxzdIn9F1jdOxDUbjx4Cp6+701fbnScjW7Ebd2MXcESaTwMWr021uF16p3evTJHupQi8dGfpQdbn9Jlja2LhHD83oYT5bRArjw/ayMnDYcV4KPZjidzagV9sL7R/gbU+QA== ;
+; record cache shard 201; size 4
+. 86400 86357 IN NSEC aaa. NS SOA RRSIG NSEC DNSKEY ZONEMD ; (Secure) auth=1 zone=. from=193.0.14.129 nm= rtag= ss=0
+. 86400 86357 IN RRSIG NSEC 8 0 86400 20240613050000 20240531040000 5613 . aw+AKvv4BHeQ5YIrxJt9xKazcxrxIvmDgjcRnuqlrdrUhi/pYRKg1e1bORaoyQgYsm+a7kJDjKJwgyKkhDr1A6Bk+cae3Om8M/iV22iXMIRNBYUnKuUCwhlVPn+Z7uAnUcX/ltqYPBQvDAOu8KK630e5tuus/IwvxpBIqy+LNOkXk+lWiKXhJkabi12HIFRZqsHVBR7ZsFrxPfw5MACBdNcUylWIWZX0DaPs8Htr+KOa36VDA060/G6fDIux2qDnrWbfkYV+L4uomI+g50I6qtZQRfiMD0PJ7ZTVp+Qq56XGT3BWet396+zGs5M59bVUotyFXhczNx29VRuKnnFdgg== ;
+. 86400 86327 IN DNSKEY 256 3 8 AwEAAZBALoOFImwcJJg9Iu7Vy7ZyLjhtXfvO1c9k4vHjOpf9i7U1kKtrBvhnwsOni1sb50gkUayRtMDTUQqvljMMf4bpkyEtcE5evCzhHbFLq1coL5QOix3mfJm++FvIMaAt52nOvAdqR/luuI11bA1AmSCIJKAUx147DcfOHYKg3as+dznn3Iah4cWBMVzDe7PPsFS1AO6gU8EpmiRJ9VMNA09fOyDuq9+d6sw8UUnJRMAFAuPLhUFjUAOuWOw74BC9lOtMQpbLMz8pX0CDKdOXDHjyj61nxSSWxPdUjeoxI17lQTpSPRtqRHFn5Fgj2e+9BVwhhWGDQN8kUVSJHZtQiI0= ; (Secure) auth=1 zone=. from=198.41.0.4 nm= rtag= ss=0
+. 86400 86327 IN DNSKEY 257 3 8 AwEAAaz/tAm8yTn4Mfeh5eyI96WSVexTBAvkMgJzkKTOiW1vkIbzxeF3+/4RgWOq7HrxRixHlFlExOLAJr5emLvN7SWXgnLh4+B5xQlNVz8Og8kvArMtNROxVQuCaSnIDdD5LKyWbRd2n9WGe2R8PzgCmr3EgVLrjyBxWezF0jLHwVN8efS3rCj/EWgvIWgb9tarpVUDK/b58Da+sqqls3eNbuv7pr+eoZG+SrDK6nWeL3c6H5Apxz7LjVc1uTIdsIXxuOLYA4/ilBmSVIzuDWfdRUfhHdY6+cn8HFRm+2hM8AnXGXws9555KrUB5qihylGa8subX2Nn6UwNR1AkUTV74bU= ; (Secure) auth=1 zone=. from=198.41.0.4 nm= rtag= ss=0
+. 86400 86327 IN RRSIG DNSKEY 8 0 172800 20240621000000 20240531000000 20326 . axzhlvsjvXefBmyQv7pZSJsEJ5BW2poXC95oU5D4itI8yejNJmehpcjBlVodnaqSQ0nNRVnvhVqC0eGzwOc/A+DCIS8tt+w93GIHhMAIkydLuxGofblreMIdD2/WNP4QQgCf4H7ANDNURozqDaFg9rq+GEHBBucHL4jt1t3dxcDcOyKaEo+2Z28NjsucYdqAENVm0Gcy9AJ4bd/a5N7DuxzdIn9F1jdOxDUbjx4Cp6+701fbnScjW7Ebd2MXcESaTwMWr021uF16p3evTJHupQi8dGfpQdbn9Jlja2LhHD83oYT5bRArjw/ayMnDYcV4KPZjidzagV9sL7R/gbU+QA== ;
+. 86400 86327 IN NS a.root-servers.net. ; (Secure) auth=1 zone=. from=199.7.91.13 nm= rtag= ss=0
+. 86400 86327 IN NS b.root-servers.net. ; (Secure) auth=1 zone=. from=199.7.91.13 nm= rtag= ss=0
+. 86400 86327 IN NS c.root-servers.net. ; (Secure) auth=1 zone=. from=199.7.91.13 nm= rtag= ss=0
+. 86400 86327 IN NS d.root-servers.net. ; (Secure) auth=1 zone=. from=199.7.91.13 nm= rtag= ss=0
+. 86400 86327 IN NS e.root-servers.net. ; (Secure) auth=1 zone=. from=199.7.91.13 nm= rtag= ss=0
+. 86400 86327 IN NS f.root-servers.net. ; (Secure) auth=1 zone=. from=199.7.91.13 nm= rtag= ss=0
+. 86400 86327 IN NS g.root-servers.net. ; (Secure) auth=1 zone=. from=199.7.91.13 nm= rtag= ss=0
+. 86400 86327 IN NS h.root-servers.net. ; (Secure) auth=1 zone=. from=199.7.91.13 nm= rtag= ss=0
+. 86400 86327 IN NS i.root-servers.net. ; (Secure) auth=1 zone=. from=199.7.91.13 nm= rtag= ss=0
+. 86400 86327 IN NS j.root-servers.net. ; (Secure) auth=1 zone=. from=199.7.91.13 nm= rtag= ss=0
+. 86400 86327 IN NS k.root-servers.net. ; (Secure) auth=1 zone=. from=199.7.91.13 nm= rtag= ss=0
+. 86400 86327 IN NS l.root-servers.net. ; (Secure) auth=1 zone=. from=199.7.91.13 nm= rtag= ss=0
+. 86400 86327 IN NS m.root-servers.net. ; (Secure) auth=1 zone=. from=199.7.91.13 nm= rtag= ss=0
+. 86400 86327 IN RRSIG NS 8 0 518400 20240612200000 20240530190000 5613 . MLLMyNzB9ewafzcrMpEV/AI7GEV6GmRfdFVR+p3I4qbuyS60lwvmqdQZD04ZXWyQHEt8Hbz3vQIV7Sm0B7F6iAHBogOYm2CuWGaHGtRSAX9taUTyD1wDtADm+en+a0maXifScE9nWrX7wMweOZTHqE0Ek/QvsOcjnQZJyPM5EqSaxHxBQeNz82VfQOjCMMH7gsBv4+tb0powC/7FkJIrT7mD7/hsaD5bbAAbUvuwORmF7QUHKIxqoZdR/V/MHQJmGeMZFKC4TUJChgtXpnMU1PhSEmvNc6KcPI8YRJ34bDjGxgOF/LKZY0NHHg7FrIl37nEW5NxFlNOzR+Tq2DKpFw== ;
+. 86400 86357 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2024053100 1800 900 604800 86400 ; (Secure) auth=1 zone=. from=193.0.14.129 nm= rtag= ss=0
+. 86400 86357 IN RRSIG SOA 8 0 86400 20240613050000 20240531040000 5613 . NMwxpWJFnOQ8CZgYmDSV14Dzj2Zppfr/OuMeRMi6bf2IDuRzkiBVoeJHa1dvbCQPjsEaD4jGchC7GeGbxSupJuvUgsr3CMCybqyIpO1tt+J/nIsS3IHrg7m7/OnwKul6IMtg36FmqAmv48LyMWwcyfD0O70XnVlrrnvztkryCcHwM66bgzwNymJfrK7U7xES8vYpdG+vF0ruNuJp6lrUHiE6cZ+ER2faQQ6OLIQWaq79LSdQLD1cAjhQisy+Iv/QbRHhunjwsXyigp3F+mHuRtEsZW5OIbdFWRvGRC4RjgkWan/0DefzNj28j2C1OFpQ0VtK1Zate9LqsgE0aE9HmQ== ;
 ; record cache shard 202; size 0

I'm using the following configuration:

root@rdns2:/tmp# cat /etc/powerdns/recursor.yml
# THIS IS A PROOF OF CONCEPT! STRUCTURE, TYPES AND NAMES ARE SUBJECT TO CHANGE
# Start of converted recursor.yml based on /etc/powerdns/recursor.conf
dnssec:
  log_bogus: true
  validation: validate
incoming:
  allow_from:
  - 127.0.0.0/8
  - 10.0.0.0/8
  - 100.64.0.0/10
  - 169.254.0.0/16
  - 192.168.0.0/16
  - 172.16.0.0/12
  - ::1/128
  - fc00::/7
  - fe80::/10
  - 2001:67c:1400:1221::/64
  listen:
  - 0.0.0.0
  - '::'
  allow_no_rd: true
logging:
  quiet: true
outgoing:
  network_timeout: 3000
  source_address:
  - 0.0.0.0
  - '::'
recordcache:
  max_negative_ttl: 300
  refresh_on_ttl_perc: 10
  serve_stale_extensions: 200
recursor:
  config_dir: /etc/powerdns
  extended_resolution_errors: true
  hint_file: /usr/share/dns/root.hints
  include_dir: /etc/powerdns/recursor.d
  lua_config_file: /etc/powerdns/recursor.lua
  max_total_msec: 14000
  setgid: pdns
  setuid: pdns
# Validation result: OK
# End of converted /etc/powerdns/recursor.conf
#
# Found 0 .conf files in /etc/powerdns/recursor.d

root@rdns2:/tmp# cat /etc/powerdns/recursor.lua
-- Debian default Lua configuration file for PowerDNS Recursor

-- Load DNSSEC root keys from dns-root-data package.
-- Note: If you provide your own Lua configuration file, consider
-- running rootkeys.lua too.
dofile("/usr/share/pdns-recursor/lua-config/rootkeys.lua")

May 31 '24 09:05 ZaphodB

There's indeed a short while after restarting that the cache has no authoritative data on the root, so it will answer with an NODATA (NOERROR and no answer records) in that period on nord queries. The task that refreshes the root data kicks in a few seconds after startup.

If the PC has the empty answer to the nord query in cache, it will continue replying with that answer (on nord queries, as they match, regular rd queries do not suffer from this, as they explicitly get auth root data if needed).

May 31 '24 09:05 omoerbeek

pdns pdns copied to clipboard

recursor: return root-hints for IN NS . queries without rd bit for the sake of dig +trace

Short description

Usecase

Description

pdns
pdns copied to clipboard