Make WHOIS lookup fail gracefully
When --whois is enabled, and one lookup fails, dnstwist dies with an exception such as:
Traceback (most recent call last):
File "/usr/local/bin/dnstwist", line 8, in <module>
sys.exit(run())
^^^^^
File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 1182, in wrapper
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 1540, in run
wreply = whois.whois('.'.join(domain_tld(domain['domain'])[1:]))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 258, in whois
return self._extract(self.query(domain, server))
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 253, in query
return self.query(query, refer.group('server'))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 237, in query
sock.connect((server, 43))
socket.gaierror: [Errno -2] Name or service not known
As you can see, the WHOIS server 'refers' to another. In this case, it's whois.iana.org, and I don't know why it fails. But that's irrelevant to this issue; I believe such a situation should be handled gracefully, by skipping the domain silently and logging this to the debug log. After all, not all registries provide reliable WHOIS servers.
Upon further inspection, (at least) .ph apparently returns an empty 'refer':
refer:
For comparison, for .com:
refer: whois.verisign-grs.com
This -obviously- confuses dnstwist.
Could you please provide domain name and DNS resolver (if publicly available) so I can reproduce this issue?
Meanwhile I pushed some changes which hopefully should resolve this.
Could you please provide domain name and DNS resolver (if publicly available) so I can reproduce this issue?
Did you see my follow-up comment (https://github.com/elceef/dnstwist/issues/231#issuecomment-2380613556)?
Empty refer fields are skipped. Initially domain name for IANA server is resolved which seems to be the cause in your network. Commit 464bb425462d393a9e074a04e5c431985072e705 should resolve it.