qubes-issues
qubes-issues copied to clipboard
QubesOS
DNS mostly fails in AppVMs under 4.2 due to change in NetworkManager behavior
Qubes OS release
4.2.0 (R4.2)
Brief summary
After installing 4.2 on my notebook I usually find DNS queries failing outside sys-net, in a manner very similar to #5648. The work-around of manually running /usr/lib/qubes/qubes-setup-dnat-to-dns is effective. I have diagnosed the cause and come up with what I think is a fix (see below).
Steps to reproduce
- Install Qubes 4.2. (I configured sys-net as a disposable VM based on debian-12-xfce which is my default template. I don't think sys-net being disposable is a factor but the template might be.)
- Boot the freshly installed OS.
- Wait for the wifi connection to come up.
- Try to reach other hosts on the LAN or the wider internet from an AppVM by their DNS names.
Expected behavior
Hosts are reachable.
Actual behavior
Mostly, but not always, no hosts are reachable.
Analysis
`qubes-nmhook` only reacts to the "up", "vpn-up" and "vpn-down" events whereas NetworkManager in debian-12-xfce inserts the DNS server _after_ "up" and before "dhcp4-change".
Having found and read #5648, I inserted trace logging into qubes-setup-dnat-to-dns and found it does get executed, even multiple times during a single start-up. It tries to contact systemd-resolved over DBUS and fails, falling back to /etc/resolv.conf where it looks for IPv4 DNS servers. Sometimes it finds one but mostly it comes up empty.
That's interesting because when I look into resolv.conf my local DNS server is reliably present. I concluded that whoever calls qubes-setup-dnat-to-dns does so before resolv.conf is fully formed. A comment in resolv.conf says it's generated by NetworkManager so I considered adding a post-connect hook to NM to call qubes-setup-dnat-to-dns when appropriate - which is how I found /etc/NetworkManager/dispatcher.d/qubes-nmhook that tries to do exactly that.
After inserting trace logging into qubes-nmhook I found it gets called with the following arguments:
2024-02-23 20:41:41: nmhook: invoked with "none" "hostname", IPv4 servers in resolv.conf: 0 2024-02-23 20:41:41: nmhook: invoked with "" "connectivity-change", IPv4 servers in resolv.conf: 0 2024-02-23 20:41:41: nmhook: invoked with "lo" "up", IPv4 servers in resolv.conf: 0 2024-02-23 20:41:47: nmhook: invoked with "wls6" "dhcp6-change", IPv4 servers in resolv.conf: 0 2024-02-23 20:41:49: nmhook: invoked with "wls6" "up", IPv4 servers in resolv.conf: 0 2024-02-23 20:41:49: nmhook: invoked with "" "connectivity-change", IPv4 servers in resolv.conf: 0 2024-02-23 20:41:50: nmhook: invoked with "wls6" "dhcp4-change", IPv4 servers in resolv.conf: 1
Whenever I install a new Qubes release I try to keep the old one in a dual-boot setup, so I traced qubes-nmhook under 4.1 as well:
2024-02-27 06:59:54: nmhook: invoked with "none" "hostname", IPv4 servers in resolv.conf: 0 2024-02-27 06:59:54: nmhook: invoked with "" "connectivity-change", IPv4 servers in resolv.conf: 0 2024-02-27 07:00:02: nmhook: invoked with "wls7" "up", IPv4 servers in resolv.conf: 1 2024-02-27 07:00:02: nmhook: invoked with "" "connectivity-change", IPv4 servers in resolv.conf: 1
My symptoms under 4.2 are due to qubes-nmhook only reacting to the "up", "vpn-up" and "vpn-down" events whereas NetworkManager in debian-12-xfce inserts the DNS server after "up" and before "dhcp4-change". When things do work in 4.2 it's because the ordering of events is different and the DNS server happens to be present on "up".
Proposed solution
I changed the following line in qubes-nmhook
if [ "$2" = "up" ] || [ "$2" = "vpn-up" ] || [ "$2" = "vpn-down" ]; then
into
if [ "$2" = "up" ] || [ "$2" = "vpn-up" ] || [ "$2" = "vpn-down" ] || [ "$2" = "dhcp4-change" ]; then
which solved the problem for me. If this is an acceptable fix I can submit it as a PR.
@codingjourney I think you made a mistake here. You merged into your own main branch instead of creating the pull request in https://github.com/QubesOS/qubes-core-agent-linux
Automated announcement from builder-github
The component core-agent-linux (including package core-agent-linux) has been pushed to the r4.2 testing repository for the Fedora template.
To test this update, please install it with the following command:
sudo dnf update --enablerepo=qubes-vm-r4.2-current-testing
Changes included in this update
Automated announcement from builder-github
The package core-agent-linux has been pushed to the r4.2 testing repository for the Debian template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing bookworm-testing (or appropriate equivalent for your template version), then use the standard update command:
sudo apt-get update && sudo apt-get dist-upgrade
Changes included in this update
Automated announcement from builder-github
The component core-agent-linux (including package core-agent-linux) has been pushed to the r4.2 testing repository for the Fedora template.
To test this update, please install it with the following command:
sudo dnf update --enablerepo=qubes-vm-r4.2-current-testing
Changes included in this update
Automated announcement from builder-github
The component core-agent-linux (including package core-agent-linux) has been pushed to the r4.2 testing repository for the Fedora template.
To test this update, please install it with the following command:
sudo dnf update --enablerepo=qubes-vm-r4.2-current-testing
Changes included in this update
Automated announcement from builder-github
The package core-agent-linux has been pushed to the r4.2 testing repository for the Debian template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing trixie-testing (or appropriate equivalent for your template version), then use the standard update command:
sudo apt-get update && sudo apt-get dist-upgrade
Changes included in this update
Automated announcement from builder-github
The component core-agent-linux (including package core-agent-linux) has been pushed to the r4.2 testing repository for the Fedora template.
To test this update, please install it with the following command:
sudo dnf update --enablerepo=qubes-vm-r4.2-current-testing
Changes included in this update
Automated announcement from builder-github
The package core-agent-linux has been pushed to the r4.2 stable repository for the Debian template.
To install this update, please use the standard update command:
sudo apt-get update && sudo apt-get dist-upgrade
Changes included in this update
Automated announcement from builder-github
The package core-agent-linux has been pushed to the r4.2 stable repository for the Debian template.
To install this update, please use the standard update command:
sudo apt-get update && sudo apt-get dist-upgrade
Changes included in this update
Automated announcement from builder-github
The component core-agent-linux (including package core-agent-linux) has been pushed to the r4.2 stable repository for the Fedora template.
To install this update, please use the standard update command:
sudo dnf update
Changes included in this update
Automated announcement from builder-github
The component core-agent-linux (including package core-agent-linux) has been pushed to the r4.2 stable repository for the Fedora template.
To install this update, please use the standard update command:
sudo dnf update
Changes included in this update
Automated announcement from builder-github
The component core-agent-linux (including package core-agent-linux) has been pushed to the r4.2 stable repository for the Fedora template.
To install this update, please use the standard update command:
sudo dnf update
Changes included in this update
Automated announcement from builder-github
The component core-agent-linux (including package core-agent-linux) has been pushed to the r4.2 stable repository for the Fedora template.
To install this update, please use the standard update command:
sudo dnf update
Changes included in this update