unbound
unbound copied to clipboard
NLnetLabs
Garbage CNAME when using python module
Describe the bug I have unbound 1.13.1 compiled for python support. I wrote a python module for storing DNS entries in reverse so that a proxy can use them later. The parsing happens under the MODULE_EVENT_MODDONE section, when qstate.return_msg is set.
I have an algorithm for parsing A, AAAA and CNAME entries. For the most part, they work perfectly. But I have found one particular domain that was causing an exception to be thrown:
ocsp.pki.goog
If you do a dig on this domain, you will get exactly two records:
- CNAME: ocsp.pki.goog => pki-goog.l.google.com
- A: pki-goog.l.google.com => 172.217.1.227
To debug further, I am just dumping all of the contents of the data that is passed in. What I am seeing is that the CNAME and A are parsed OK, but for some reason there is another answer parsed in between that looks like random garbage. I can see part of what kind of looks like a record wedged in there, but it is totally out of place.
I will leave my code in a comment below, with a ton of debug prints and the redis writes commented out. I will also leave the data being dumped.
To reproduce Steps to reproduce the behavior:
- Compile unbound 1.13.1 with python support (./configure --with-pyunbound --with-libevent --with-pythonmodule --prefix=/opt)
- Use the following unbound.conf:
# Example configuration file for log.py
server:
verbosity: 1
interface: 0.0.0.0
do-daemonize: no
access-control: 0.0.0.0/0 allow
chroot: ""
username: ""
directory: ""
logfile: ""
pidfile: "unbound.pid"
module-config: "validator python iterator"
# Python config section
python:
# Script file to load
python-script: "/opt/etc/unbound/reverse.py"
- Use my reverse.py (will include in a comment below)
- Run unbound
- Run "dig ocsp.pki.goog" against unbound
- Check unbound output logs
Expected behavior Unbound should only process the two records mentioned in the description. The records should be valid and parseable.
System:
- Unbound version: 1.13.1
- OS: Debian Linux
unbound -Voutput:
Version 1.13.1
Configure line: --with-pyunbound --with-libevent --with-pythonmodule --prefix=/opt
Linked libs: libevent 2.1.11-stable (it uses epoll), OpenSSL 1.1.1g 21 Apr 2020
Linked modules: dns64 python respip validator iterator
BSD licensed, see LICENSE in source package for details.
Report bugs to [email protected] or https://github.com/NLnetLabs/unbound/issues
Additional information If it helps, here is a link to my dockerfile. You will have to replace reverse.py with the one I am supplying here.
https://github.com/justinschw/unbound-reversecache
reverse.py
import os
import redis
import ipaddress
MIN_TTL = 90
'''
Python module to store a reverse DNS lookup cache
'''
#redisHost = 'redis' if not ('REDIS_HOST' in os.environ.keys()) else os.environ['REDIS_HOST']
#redisPort = 6379 if not ('REDIS_PORT' in os.environ.keys()) else os.environ['REDIS_PORT']
#reverseCache = redis.Redis(host=redisHost, port=redisPort, db=0)
def bytes_to_int(bytes):
result = 0
for b in bytes:
result = result * 256 + int(b)
return result
def parse_cname(bytes):
i = 0
parts = []
while(i < len(bytes)):
lpart = bytes_to_int(bytes[i:i+1])
log_info(str(parts))
parts.append(bytes[i+1:i+1+lpart].decode('utf-8'))
i = i + lpart + 1
while '' in parts:
parts.remove('')
return '.'.join(parts)
def logDnsMsg(qstate):
"""Logs response"""
r = qstate.return_msg.rep
q = qstate.return_msg.qinfo
if (r):
for i in range(0, r.rrset_count):
rr = r.rrsets[i]
rk = rr.rk
d = rr.entry.data
for j in range(0, d.count+d.rrsig_count):
data = d.rr_data[j]
ttl = d.rr_ttl[j]
length = bytes_to_int(data[:2])
if (rk.type_str == 'AAAA' and length == 16):
hostName = rk.dname_str
if hostName[-1] == '.':
hostName = hostName[:-1]
ip = data[2:]
ip6Long = u"{0:0>2x}{1:0>2x}:{2:0>2x}{3:0>2x}:{4:0>2x}{5:0>2x}:{6:0>2x}{7:0>2x}:{8:0>2x}{9:0>2x}:{10:0>2x}{11:0>2x}:{12:0>2x}{13:0>2x}:{14:0>2x}{15:0>2x}".format(*ip)
ip6Short = str(ipaddress.ip_address(ip6Long).compressed)
#reverseCache.set(ip6Short, hostName, ex=ttl)
if (rk.type_str == 'CNAME'):
hostName = rk.dname_str
log_info('BEGIN CNAME SECTION - hostname='+hostName+'\n')
log_info('data="'+str(data[2:])+'"\n')
try:
cname = parse_cname(data[2:])
except:
log_info('Error parsing CNAME for '+hostName+'\n')
if hostName[-1] == '.':
hostName = hostName[:-1]
#reverseCache.set(cname, hostName, ex=ttl)
log_info('END CNAME SECTION - hostname='+hostName+'\n')
if (rk.type_str == 'A' and length == 4):
hostName = rk.dname_str
log_info('BEGIN A SECTION - hostname='+hostName+'\n')
if hostName[-1] == '.':
hostName = hostName[:-1]
ip = data[2:]
log_info(str(ip))
ipString = "{}.{}.{}.{}".format(*ip)
#reverseCache.set(ipString, hostName, ex=ttl)
log_info('END A SECTION - hostname='+hostName+'\n')
def init(id, cfg):
log_info("pythonmod: init called, module id is %d port: %d script: %s" % (id, cfg.port, cfg.python_script))
# TODO: Put this in a config file
return True
def deinit(id):
return True
def inform_super(id, qstate, superqstate, qdata):
return True
def operate(id, event, qstate, qdata):
if (event == MODULE_EVENT_NEW) or (event == MODULE_EVENT_PASS):
#Pass on the new event to the iterator
qstate.ext_state[id] = MODULE_WAIT_MODULE
return True
if event == MODULE_EVENT_MODDONE:
#Iterator finished, show response (if any)
if (qstate.return_msg):
logDnsMsg(qstate)
qstate.ext_state[id] = MODULE_FINISHED
return True
qstate.ext_state[id] = MODULE_ERROR
return True
This is the dump of my output. The second CNAME section is the one that is the problem. As you can see the data is a much longer string than for the normal, correct CNAME record.
[1621041792] unbound[7:0] info: BEGIN CNAME SECTION - hostname=ocsp.pki.goog.
[1621041792] unbound[7:0] info: data="b'\x08pki-goog\x01l\x06google\x03com\x00'"
[1621041792] unbound[7:0] info: []
[1621041792] unbound[7:0] info: ['pki-goog']
[1621041792] unbound[7:0] info: ['pki-goog', 'l']
[1621041792] unbound[7:0] info: ['pki-goog', 'l', 'google']
[1621041792] unbound[7:0] info: ['pki-goog', 'l', 'google', 'com']
[1621041792] unbound[7:0] info: END CNAME SECTION - hostname=ocsp.pki.goog
[1621041792] unbound[7:0] info: BEGIN CNAME SECTION - hostname=ocsp.pki.goog.
[1621041792] unbound[7:0] info: data="b"\x00\x05\x08\x03\x00\x00\x01,`\xc6-\xcc`\x9e\xa0\xcc\xd1\x8e\x03pki\x04goog\x00r|=\x93\x1a\x0b\x96\x95\x8d0\xebz\x07`V\x1c(\xd6\xf7\xddq\xfbvg\xb5\xe2\x98.@_,iHe\x98rr\x1c\xba\xbb\x8c\x89\x07\x96\x14G\x9d\xd0\xec\xdf\xb7\xd6~\x0f7m\xf80\x18\x98\xf1'\x8f\xed\x88j\x18:2\x14\x83\xb1\x8d<b~\x15#\xa5o\x1c\x9e\x1ex\x17\x10T\xd6\xb2\xd9~\x9c%E\x18\x98\x11\xed\x08Rj\xefFd(5\xf74\x89%\xe1\x85t\tl\x8a@\xffM<\xb3K\x03\x0e\xa2\xe8\x0cE""
[1621041792] unbound[7:0] info: []
[1621041792] unbound[7:0] info: ['']
[1621041792] unbound[7:0] info: ['', '\x08\x03\x00\x00\x01']
[1621041792] unbound[7:0] info: Error parsing CNAME for ocsp.pki.goog.
[1621041792] unbound[7:0] info: END CNAME SECTION - hostname=ocsp.pki.goog
[1621041792] unbound[7:0] info: BEGIN A SECTION - hostname=pki-goog.l.google.com.
[1621041792] unbound[7:0] info: b'\x8e\xfaq^'
[1621041792] unbound[7:0] info: END A SECTION - hostname=pki-goog.l.google.com
@justinschw
Not exactly related to this but is there anything specific op we need to do have the redis imported within the unbound pythonmod? We were getting this error and we do have redis installed for both python2 and python3 and scripts importing it fine.. So was not sure if I was missing a step related to the same.. We were using Ubuntu 18.04 so were working with the 1.6.7 version..
Same happens for the pyroute2 module which we try to import under the same context
root@test:/home/router# unbound-checkconf /etc/unbound/unbound.conf
Traceback (most recent call last):
File "/home/router/examples/reverse.py", line 4, in
root@test:/home/router# python -c "import redis" root@test:/home/router# python3 -c "import redis"
root@test:/home/router# python3 -m pip install redis Requirement already satisfied: redis in /usr/local/lib/python3.6/dist-packages Requirement already satisfied: packaging>=20.4 in /usr/local/lib/python3.6/dist-packages (from redis) Requirement already satisfied: importlib-metadata>=1.0; python_version < "3.8" in /usr/local/lib/python3.6/dist-packages (from redis) Requirement already satisfied: deprecated>=1.2.3 in /usr/local/lib/python3.6/dist-packages (from redis) Requirement already satisfied: async-timeout>=4.0.2 in /usr/local/lib/python3.6/dist-packages (from redis) Requirement already satisfied: typing-extensions; python_version < "3.8" in /usr/local/lib/python3.6/dist-packages (from redis) Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.6/dist-packages (from packaging>=20.4->redis) Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata>=1.0; python_version < "3.8"->redis) Requirement already satisfied: wrapt<2,>=1.10 in /usr/local/lib/python3.6/dist-packages (from deprecated>=1.2.3->redis)
root@test:/home/router# python -m pip install redis DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality. Requirement already satisfied: redis in /usr/local/lib/python2.7/dist-packages (3.5.3)
@justinschw I went through your other project and noticed that you had install python-redis package.. That solved it for me.. Thanks.. Your program has been very helpful to start off with..
ok, glad that worked for you. The CNAME issue should happen even if you aren't using redis at all though.
I have reproduced the issue.
The junk record you are seeing is a RRSIG record, not a CNAME.
Here is how Wireshark dissects the DNS answer packet:
User Datagram Protocol, Src Port: 53, Dst Port: 48505
Domain Name System (response)
Transaction ID: 0xcde4
Flags: 0x8180 Standard query response, No error
Questions: 1
Answer RRs: 3
Authority RRs: 0
Additional RRs: 1
Queries
Answers
ocsp.pki.goog: type CNAME, class IN, cname pki-goog.l.google.com
Name: ocsp.pki.goog
Type: CNAME (Canonical NAME for an alias) (5)
Class: IN (0x0001)
Time to live: 248 (4 minutes, 8 seconds)
Data length: 23
CNAME: pki-goog.l.google.com
ocsp.pki.goog: type RRSIG, class IN
Name: ocsp.pki.goog
Type: RRSIG (Resource Record Signature) (46)
Class: IN (0x0001)
Time to live: 248 (4 minutes, 8 seconds)
Data length: 156
Type Covered: CNAME (Canonical NAME for an alias) (5)
Algorithm: RSA/SHA-256 (8)
Labels: 3
Original TTL: 300 (5 minutes)
Signature Expiration: Oct 31, 2023 01:52:15.000000000 EDT
Signature Inception: Oct 9, 2023 01:52:15.000000000 EDT
Key Tag: 253
Signer's name: pki.goog
Signature: 47716c8cd29b41cbd98b5707a58c9a4010fd38243129179f6c0d871b2ea8fa00419bbfca…
pki-goog.l.google.com: type A, class IN, addr 172.217.13.163
Additional records
<Root>: type OPT
[Request In: 65]
[Time: 0.004000147 seconds]
Your code seems adapted from the log.py example: https://github.com/NLnetLabs/unbound/blob/master/pythonmod/examples/log.py#L73-L81
The example does 2 loops:
for i in range(0, r.rrset_count):
rr = r.rrsets[i]
rk = rr.rk
print i,":",rk.dname_list, rk.dname_str, "flags: %04X" % rk.flags,
print "type:",rk.type_str,"(%d)" % ntohs(rk.type), "class:",rk.rrset_class_str,"(%d)" % ntohs(rk.rrset_class)
d = rr.entry.data
for j in range(0,d.count+d.rrsig_count):
[...]
The first loop is over the 2 resource records (CNAME and A). The type (rk) gets set in the first loop.
Then we loop over d.count+d.rrsig_count. Since the CNAME record is signed, this loops twice, with the same rk value (CNAME).
I have not found a way to identify RRSIG records from their data.
I would suspect your are supposed to expect they will be at the end?
i.e., if d.count is 1 and d.rrsig_count is 1, the RRSIG is the second item, but I can't be sure.
The easy solution for your issue is simply to ignore the rrsig_count:
Your logic becomes:
for i in range(0, r.rrset_count):
rr = r.rrsets[i]
d = rr.entry.data
for j in range(0,d.count):
[..process record..]
Thank you for posting your code. Your CNAME parsing function is the only one I found that works. The data bytes we receive in the python module are not exactly in the correct DNS wire format I believe and all my attempts failed.
@cveilleux nice analysis! I can confirm that RRSIG records are stored after the regular records. For the second loop to go until d.count is correct if you don't want to go over the RRSIG records. I believe that solves this issue then?
I finally gave up trying to do my own DNS resource record parsing and switched to using the dnslib python package.
I published an advanced python logging module here if anyone needs full logging of the DNS request/responses with records:
https://gist.github.com/cveilleux/20fd7574a0a9fe796519336efc273cbe
The actual parsing is fully delegated to dnslib, which currently supports 18 types. The actual parsing code can be seen here:
https://github.com/paulc/dnslib/blob/0.9.23/dnslib/dns.py#L1085-L2258
I believe this now can be closed; it seems to be more of a misunderstanding than an issue. Feel free to ping back if you would like it reopened.