mycroft.util.network_utils:_connected_dns has hard coded DNS (fails on startup if network blocks untrusted providers)

[fermulator]

Describe the bug At home, I do not allow egress 53 to any DNS lookups unless it is from my pihole. DHCP serves the proper DNS to clients using DHCP options.

Looks like there are some startup utils/tests that try to validate stuff about the network and assume it will be allowed to talk to Google DNS (8.8.8.8 or 8.8.4.4)

$ cat /etc/resolv.confnf
# Generated by resolvconf
domain local
nameserver 127.0.0.1

$ cat  /run/dnsmasq/resolv.conf
# Generated by resolvconf
nameserver 10.0.0.60

(.venv) pi@picroft:~ $ systemctl status dnsmasq
● dnsmasq.service - dnsmasq - A lightweight DHCP and caching DNS server
   Loaded: loaded (/lib/systemd/system/dnsmasq.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2022-05-23 14:02:57 BST; 8min ago
  Process: 397 ExecStartPre=/usr/sbin/dnsmasq --test (code=exited, status=0/SUCCESS)
  Process: 408 ExecStart=/etc/init.d/dnsmasq systemd-exec (code=exited, status=0/SUCCESS)
  Process: 420 ExecStartPost=/etc/init.d/dnsmasq systemd-start-resolvconf (code=exited, status=0/SUCCESS)
 Main PID: 418 (dnsmasq)
    Tasks: 1 (limit: 2200)
   Memory: 3.0M
   CGroup: /system.slice/dnsmasq.service
           └─418 /usr/sbin/dnsmasq -x /run/dnsmasq/dnsmasq.pid -u dnsmasq -r /run/dnsmasq/resolv.conf -7 /etc/dnsmasq.d,.dpkg-dist,.dpkg-old,.dpkg-new --local-service -

(snip)
May 23 14:02:57 picroft systemd[1]: Started dnsmasq - A lightweight DHCP and caching DNS server.
May 23 14:03:09 picroft dnsmasq[418]: reading /run/dnsmasq/resolv.conf
May 23 14:03:09 picroft dnsmasq[418]: using nameserver 10.0.0.60#53

2022-05-23 14:03:48.662 | ERROR    |   755 | mycroft.util.network_utils:_connected_dns:69 | Unable to connect to primary DNS server, trying secondary...
2022-05-23 14:03:51.667 | ERROR    |   755 | mycroft.util.network_utils:_connected_dns:78 | Unable to connect to secondary DNS server.

To Reproduce Steps to reproduce the behavior:

1. Block Google DNS
2. Startup raspberry pi, ssh into it, and check the logs.
3. pi@picroft:/var/log/mycroft/skills.log

Expected behavior UTs should use system DNS (not harded)

Log files

2022-05-23 14:03:48.662 | ERROR    |   755 | mycroft.util.network_utils:_connected_dns:69 | Unable to connect to primary DNS server, trying secondary...
2022-05-23 14:03:51.667 | ERROR    |   755 | mycroft.util.network_utils:_connected_dns:78 | Unable to connect to secondary DNS server.

mycroft-core/test/unittests/util/test_network_utils.py:15:    def test_dns_and_ncsi_fail(self, mock_conf):
mycroft-core/test/unittests/util/test_network_utils.py:16:        """Check that DNS and NCSI failure results in False response"""
mycroft-core/test/unittests/util/test_network_utils.py:19:                "dns_primary": "127.0.0.1",
mycroft-core/test/unittests/util/test_network_utils.py:20:                "dns_secondary": "127.0.0.1",
mycroft-core/test/unittests/util/test_network_utils.py:28:    def test_secondary_dns_succeeds(self, mock_conf):
mycroft-core/test/unittests/util/test_network_utils.py:29:        """Check that only primary DNS failing still succeeds"""
mycroft-core/test/unittests/util/test_network_utils.py:32:                "dns_primary": "127.0.0.1",
mycroft-core/test/unittests/util/test_network_utils.py:33:                "dns_secondary": "8.8.4.4",
mycroft-core/test/unittests/util/test_network_utils.py:41:    def test_dns_success_url_fail(self, mock_conf):
mycroft-core/test/unittests/util/test_network_utils.py:45:                "dns_primary": "8.8.8.8",
mycroft-core/test/unittests/util/test_network_utils.py:46:                "dns_secondary": "8.8.4.4",

Environment (please complete the following information):

• Device type: [e.g. Raspberry Pi]
• OS: [Picroft]
• Mycroft-core version: [e.g. 21.2.2]

Additional context

[fermulator]

fwiw; the Python Resolver package should be able to lookup the system nameservers. dns.resolver.nameservers

[krisgesling]

Hey ferm, thanks for this.

How we detect an active network is a long debated topic and I've moved this the the mycroft-core repo as it affects all versions not just Picroft.

One thing I added recently was configurable values for these tests via mycroft.conf that you can see here: https://github.com/MycroftAI/mycroft-core/blob/dev/mycroft/configuration/mycroft.conf#L150

For your use case, would setting your pihole as the dns endpoint work?

I think this is still something we need to address more broadly though so want to keep this issue open.

[forslund]

The idea with the DNS check is that it's a very quick way to get a reading on if the outside world is reachable. Pointing them to the local dns wouldn't tell the whole story about current connectivity.

That said Using the ncsi capture portal test might be a truer test and maybe could replace the socket connection + google.

And we could change the default from the MS detection portal to mozillas (used MS when I wrote that since it was the one I could find decent documentation on)?

Config for mozilla's endpoint:

"ncsi_endpoint": "http://detectportal.firefox.com/canonical.html",
"ncsi_expected_text": "<meta http-equiv="refresh" content="0;url=https://support.mozilla.org/kb/captive-portal"/>"

At one point it was discussed if Mycroft should host a capture portal detect page aswell.

Since the missing DNS should trigger fallback to the ncsi method, the connection should have been detected and startup of Mycroft should proceed. Just to verify that there's no issue with the fallback, can you clarify if mycroft works as expected or halts at boot?