PowerDNS
SetEDNSOption does not work as expected with dnssec enabled
- Program: dnsdist
- Issue type: Bug report
Short description
When using SetEDNSOptionAction questions with dnssec (dig +dnssec) the DNS answer is empty and the status is BADVERS
Environment
- Operating system: CentOS 7
- Software version: 1.7.1 (Lua 5.1.4 [LuaJIT 2.0.4])
- Software source: PowerDNS repository
Sample configuration
setLocal("127.0.0.20:53")
newServer({address="208.67.222.222", name="resolver1.opendns.com", order=1, pool="prodpool", reconnectOnUp=true, maxCheckFailures=3, rise=3})
newServer({address="208.67.222.222", name="resolver1.opendns.com", order=1, pool="testpool", reconnectOnUp=true, maxCheckFailures=3, rise=3})
ZonesToBeRoutedToTestPool = newSuffixMatchNode()
ZonesToBeRoutedToTestPool:add(newDNSName("test.com."))
addAction(SuffixMatchNodeRule(ZonesToBeRoutedToTestPool), SetEDNSOptionAction(20292,"ODNS\x01\x00\x00\x08\x00\x79\xbe\xef\x00\x10\xc0\xa8\x01\x37"))
addAction(SuffixMatchNodeRule(ZonesToBeRoutedToTestPool), PoolAction("testpool"))
addAction(AllRule(), PoolAction("prodpool"))
Steps to reproduce
1.) Query test.com without dnssec
dig @127.0.0.20 test.com A +nodnssec
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 test.com A +nodnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 9320
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;test.com. IN A
;; ANSWER SECTION:
test.com. 3469 IN A 67.225.146.248
;; Query time: 5 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Tue Jun 28 09:17:54 CEST 2022
;; MSG SIZE rcvd: 53
✅ Expected A-record is returned
2.) Query test.com with dnssec
dig @127.0.0.20 test.com A +dnssec
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 test.com A +dnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: BADVERS, id: 21798
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;test.com. IN A
;; Query time: 5 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Tue Jun 28 09:18:33 CEST 2022
;; MSG SIZE rcvd: 37
❌ No data is returned and status is BADVERS
Verification
1.) Query some other domain without dnssec
dig @127.0.0.20 dnsdist.org A +nodnssec
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 dnsdist.org A +nodnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30651
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;dnsdist.org. IN A
;; ANSWER SECTION:
dnsdist.org. 86400 IN A 188.166.104.92
;; Query time: 73 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Tue Jun 28 09:18:53 CEST 2022
;; MSG SIZE rcvd: 56
2.) Query the same domain with dnssec
dig @127.0.0.20 dnsdist.org A +dnssec
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 dnsdist.org A +dnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54221
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;dnsdist.org. IN A
;; ANSWER SECTION:
dnsdist.org. 86385 IN A 188.166.104.92
;; Query time: 5 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Tue Jun 28 09:19:08 CEST 2022
;; MSG SIZE rcvd: 56
✅ Expected A-record is returned with and without dnssec
Expected behaviour
Both queries regardless of the dnssec option should work when SetEDNSOption is used
Actual behaviour
As seen above the question is not answered correctly.
Also, some of your digs went to the wrong IP - 127.0.0.20. Please always provide entirely unedited information so we are certain what we are debugging.
Also, some of your digs went to the wrong IP -
127.0.0.20. Please always provide entirely unedited information so we are certain what we are debugging.
I have updated the issue with current outputs. I will take a tcpdump of it as well.
Can you tcpdump traffic between your dnsdist and OpenDNS?
Please find the tcpdump (tcpdump -i any -nn -w - -U 'host 127.0.0.20 or host 208.67.222.222' | tee dump.pcap | tcpdump -nn -r -.) attached.
dump.zip
@rgacogne: Since you have already added a defect-label: Are you aware of the issue and the reason for it? Is there a simple fix we could implement ourselves till 1.8.0 gets released?
Since you have already added a defect-label: Are you aware of the issue and the reason for it? Is there a simple fix we could implement ourselves till 1.8.0 gets released?
I'm not, no. I have added the label because based on your description it really looks like a bug, but I won't know for sure until I have reproduced it which will likely have to wait until tomorrow. I did put 1.8.0 as milestone because we will fix it in the dev branch first, but a fix for such a bug will be backported to 1.7.x, to be clear.
It was easier than I expected so I just opened a pull request (#11729) that fixes this issue in my test. If by any chance you could test it (which involves recompiling, sorry), that would be great :)
Wow! Thanks for looking into it and fixing it faster than I could even look for an update of the issue 😊 We will recompile and test the fix asap.
Not directly related to this issue, but maybe you have an idea first spot so I mention it: When we were troubleshooting the issue we had pdns-rec as backend. We enabled tracing but didn't see a single log line regarding the query. Only in tcpdump we were able to spot it. So maybe there is an issue in pdns-rec as well, when such "malformed" packets arrive?
Sorry, we had some trouble setting up a appropriate build environment.
I can confirm your commit fixes the issue:
dig @127.0.0.20 test.com A +dnssec
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 test.com A +dnssec
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20791
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;test.com. IN A
;; ANSWER SECTION:
test.com. 2123 IN A 67.225.146.248
;; Query time: 5 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Tue Jun 28 17:24:54 CEST 2022
;; MSG SIZE rcvd: 53
✅ Expected A-record is returned
Thanks for looking into it this quickly!
Thanks a lot for testing and reporting back! I'll add the missing unit and regression tests to the pull request quickly, then after a review it will be merged into the development branch, and I'll then backport it to the 1.7.x branch. I don't have a release date for the next 1.7.x release yet, so in the meantime you can either switch to the packages that will be automatically built once the backport PR has been merged, or use the version you compiled which should be mostly identical.
Thanks for the "next steps"-explanation! As we added your commit to rel/dnsdist-1.7.x branch I think we will stay on our "custom build" until the next 1.7.x gets released.
I am not sure if this is even supposed to work but I found that SetEDNSOptionAction() seems to be only working directly used in dnsdist.conf.
If I call it from within a LUA-function it seems to just do nothing...?
Should I open another issue or is this expected?
Sample configuration
setLocal("127.0.0.20:53")
newServer({address="208.67.222.222", name="resolver1.opendns.com", order=1, pool="pool", reconnectOnUp=true, maxCheckFailures=3, rise=3})
function AddEDNS(dq)
SetEDNSOptionAction(20292,"test")
return DNSAction.Pool, "pool"
end
ModifiedInMain = newSuffixMatchNode()
ModifiedInLua = newSuffixMatchNode()
ModifiedInMain:add(newDNSName("foo.tld."))
ModifiedInLua:add(newDNSName("bar.tld."))
addAction(SuffixMatchNodeRule(ModifiedInMain), SetEDNSOptionAction(20292,"test"))
addAction(SuffixMatchNodeRule(ModifiedInLua), LuaAction(AddEDNS))
addAction(AllRule(), PoolAction("pool"))
Query foo.tld
dig @127.0.0.20 foo.tld A
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 foo.tld A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 23924
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;foo.tld. IN A
;; AUTHORITY SECTION:
. 3600 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2022070100 1800 900 604800 86400
;; Query time: 4 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Fri Jul 01 17:01:43 CEST 2022
;; MSG SIZE rcvd: 111
✅ EDNS option gets added
17:01:43.898442 IP (tos 0x0, ttl 64, id 7314, offset 0, flags [none], proto UDP (17), length 64)
127.0.0.1.43751 > 127.0.0.20.53: [bad udp cksum 0xfe52 -> 0x02f7!] 23924+ [1au] A? foo.tld. ar: . OPT UDPsize=4096 (36)
0x0000: 0000 0304 0006 0000 0000 0000 0000 0800 ................
0x0010: 4500 0040 1c92 0000 4011 6006 7f00 0001 E..@....@.`.....
0x0020: 7f00 0014 aae7 0035 002c fe52 5d74 0120 .......5.,.R]t..
0x0030: 0001 0000 0000 0001 0366 6f6f 0374 6c64 .........foo.tld
0x0040: 0000 0100 0100 0029 1000 0000 0000 0000 .......)........
17:01:43.898643 IP (tos 0x0, ttl 64, id 35131, offset 0, flags [DF], proto UDP (17), length 72)
10.195.129.37.39946 > 208.67.222.222.53: [bad udp cksum 0x3b50 -> 0xfa18!] 256+ [1au] A? foo.tld. ar: . OPT UDPsize=4096 (44)
0x0000: 0004 0001 0006 001a 4a16 032f 0000 0800 ........J../....
0x0010: 4500 0048 893b 4000 4011 765f 0ac3 8125 E..H.;@[email protected]_...%
0x0020: d043 dede 9c0a 0035 0034 3b50 0100 0120 .C.....5.4;P....
0x0030: 0001 0000 0000 0001 0366 6f6f 0374 6c64 .........foo.tld
0x0040: 0000 0100 0100 0029 1000 0000 0000 0008 .......)........
0x0050: 4f44 0004 7465 7374 OD..test
Query bar.tld
dig @127.0.0.20 bar.tld A
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 <<>> @127.0.0.20 bar.tld A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 58934
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;bar.tld. IN A
;; AUTHORITY SECTION:
. 3600 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2022070100 1800 900 604800 86400
;; Query time: 5 msec
;; SERVER: 127.0.0.20#53(127.0.0.20)
;; WHEN: Fri Jul 01 17:03:19 CEST 2022
;; MSG SIZE rcvd: 111
❌ No EDNS option is added
17:03:19.065238 IP (tos 0x0, ttl 64, id 51582, offset 0, flags [none], proto UDP (17), length 64)
127.0.0.1.37000 > 127.0.0.20.53: [bad udp cksum 0xfe52 -> 0xa294!] 58934+ [1au] A? bar.tld. ar: . OPT UDPsize=4096 (36)
0x0000: 0000 0304 0006 0000 0000 0000 0000 0800 ................
0x0010: 4500 0040 c97e 0000 4011 b319 7f00 0001 E..@.~..@.......
0x0020: 7f00 0014 9088 0035 002c fe52 e636 0120 .......5.,.R.6..
0x0030: 0001 0000 0000 0001 0362 6172 0374 6c64 .........bar.tld
0x0040: 0000 0100 0100 0029 1000 0000 0000 0000 .......)........
17:03:19.065354 IP (tos 0x0, ttl 64, id 19566, offset 0, flags [DF], proto UDP (17), length 64)
10.195.129.37.39946 > 208.67.222.222.53: [bad udp cksum 0x3b48 -> 0x3d54!] 768+ [1au] A? bar.tld. ar: . OPT UDPsize=4096 (36)
0x0000: 0004 0001 0006 001a 4a16 032f 0000 0800 ........J../....
0x0010: 4500 0040 4c6e 4000 4011 b334 0ac3 8125 E..@Ln@[email protected]...%
0x0020: d043 dede 9c0a 0035 002c 3b48 0300 0120 .C.....5.,;H....
0x0030: 0001 0000 0000 0001 0362 6172 0374 6c64 .........bar.tld
0x0040: 0000 0100 0100 0029 1000 0000 0000 0000 .......)........
SetEDNSOptionAction is an action, it can only be used in conjunction with addAction, not in custom Lua actions. If you want to add custom EDNS options via a Lua action you would need a different binding, which doesn't seem to exist today. I would suggest opening a feature request, or even better a pull request if you feel up to writing that code.