dnstwist icon indicating copy to clipboard operation
dnstwist copied to clipboard

Make WHOIS lookup fail gracefully

Open WilliamDEdwards opened this issue 1 year ago • 5 comments

When --whois is enabled, and one lookup fails, dnstwist dies with an exception such as:

Traceback (most recent call last):
  File "/usr/local/bin/dnstwist", line 8, in <module>
    sys.exit(run())
             ^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 1182, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 1540, in run
    wreply = whois.whois('.'.join(domain_tld(domain['domain'])[1:]))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 258, in whois
    return self._extract(self.query(domain, server))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 253, in query
    return self.query(query, refer.group('server'))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 237, in query
    sock.connect((server, 43))
socket.gaierror: [Errno -2] Name or service not known

As you can see, the WHOIS server 'refers' to another. In this case, it's whois.iana.org, and I don't know why it fails. But that's irrelevant to this issue; I believe such a situation should be handled gracefully, by skipping the domain silently and logging this to the debug log. After all, not all registries provide reliable WHOIS servers.

WilliamDEdwards avatar Sep 28 '24 11:09 WilliamDEdwards

Upon further inspection, (at least) .ph apparently returns an empty 'refer':

refer:

For comparison, for .com:

refer:        whois.verisign-grs.com

This -obviously- confuses dnstwist.

WilliamDEdwards avatar Sep 28 '24 11:09 WilliamDEdwards

Could you please provide domain name and DNS resolver (if publicly available) so I can reproduce this issue?

elceef avatar Sep 28 '24 18:09 elceef

Meanwhile I pushed some changes which hopefully should resolve this.

elceef avatar Sep 28 '24 19:09 elceef

Could you please provide domain name and DNS resolver (if publicly available) so I can reproduce this issue?

Did you see my follow-up comment (https://github.com/elceef/dnstwist/issues/231#issuecomment-2380613556)?

WilliamDEdwards avatar Sep 29 '24 07:09 WilliamDEdwards

Empty refer fields are skipped. Initially domain name for IANA server is resolved which seems to be the cause in your network. Commit 464bb425462d393a9e074a04e5c431985072e705 should resolve it.

elceef avatar Sep 29 '24 08:09 elceef