[BUG] DNS leaking with systemd-resolved

[khicks]

Describe the bug

DNS requests still leak through the physical interface's servers on Ubuntu 18.04 with PVPN DNS Leak Protection enabled.

When connecting to PVPN, the CLI tool will modify the systemd-resolved config in an effort to route DNS traffic to the PVPN DNS server, 10.8.8.1. However, it seems to do this with the Global config, not the tun0 interface. Here are outputs of several systemd-resolve --status runs, altered and snipped for brevity.

No VPN

Global
          DNS Domain: my.lan
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      ...
                      test

Link 3 (wlp2s0)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 10.xx.xx.x # <- My local network DNS
          DNS Domain: ~.
                      my.lan

...

ProtonVPN

Global
         DNS Servers: 10.8.8.1
          DNS Domain: my.lan
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      ...
                      test

Link 31 (tun0)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 3 (wlp2s0)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 10.xx.xx.x
          DNS Domain: ~.
                      my.lan

...

Third-party VPN

Global
          DNS Domain: my.lan
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      ...
                      test

Link 32 (tun0)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 10.yyy.yyy.1 # <- VPN provider's DNS server
          DNS Domain: ~.

Link 3 (wlp2s0)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 10.xx.xx.x
          DNS Domain: ~.
                      my.lan

...

I currently have two other VPN providers and they both handle the systemd-resolved config in about the same way. The PVPN method is different and results in my regular DNS servers appearing in DNS leak tests.

Modifying the systemd-resolved config myself after connecting to PVPN temporarily fixes the issue until I disconnect.

$ sudo systemd-resolve --interface tun0 --set-dns 10.8.8.1 --set-domain ~.
$ systemd-resolve --interface wlp2s0 --revert
# Disconnect and reconnect the physical network to restore DNS to wlp2s0 after disconnecting PVPN.

To Reproduce

Steps to reproduce the behavior:

1. $ protonvpn c -f
2. Go to dnsleaktest.com and see your real DNS servers appear in the test results.

Expected behavior

My guess is that you'd want the PVPN CLI to set the DNS server on the tun0 interface, not in the Global config.

Error Messages

Just a few debug messages showing I have DNS Leak Protection enabled.

2019-11-24 19:19:39,685 — protonvpn-cli — DEBUG — manage_dns:573 — DNS Leak Protection is enabled
2019-11-24 19:19:39,686 — protonvpn-cli — DEBUG — manage_dns:579 — resolv.conf backed up
2019-11-24 19:19:39,687 — protonvpn-cli — DEBUG — manage_dns:587 — Removed existing DNS Servers
2019-11-24 19:19:39,688 — protonvpn-cli — DEBUG — manage_dns:595 — Added ProtonVPN or custom DNS

Desktop (please complete the following information):

• OS: Ubuntu 18.04
• Python Version: 3.6.8
• ProtonVPN-CLI Version: 2.1.1

Additional context

I noticed that the CLI tool relies on directly modifying the resolv.conf file in place, so this may not be a trivial fix. The third-party VPN client does not modify the resolv.conf file, and I wish I could find exactly how it does it.

Thanks for the hard work!

[khicks]

I tried this on a fresh install and the issue isn't present there. I'm not quite sure what I'm doing differently. 😔

Update: The difference seems to be the presence of the openvpn-systemd-resolved package.

[Rafficer]

Thanks for this extensive explanation. I can also only replicate this with the openvpn-systemd-resolved package installed. I assume this gets installed by the other VPN clients?

As ProtonVPN-CLI doesn't require to have systemd installed, I'm not sure if this should be handled in the program. I'm also not sure how to properly handle this. As a default installation doesn't experience this, I don't think it's right to work around the errors introduced by another package.

[khicks]

Hey Rafficer, thanks for taking the time to look into this and verify that openvpn-systemd-resolved causes the issue.

Progress

After further research, I found that it more specifically involves the libnss-resolve (alias for libnss-systemd) package which openvpn-systemd-resolve depends on. Upon installation, it makes the following file change:

# /etc/nsswitch.conf
< hosts:          files mdns4_minimal [NOTFOUND=return] dns
---
> hosts:          files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns

Reverting the change to remove resolve [!UNAVAIL=return] and restarting the systemd-resolved service can prevent DNS leaks. Here's a rough outline of the steps I've tried.

1. Install ProtonVPN and prereqs. Do protonvpn init.
2. sudo apt install openvpn-systemd-resolve. This installs libnss-resolve/libnss-systemd too.
3. sudo protonvpn c -f
4. Go to dnsleaktest.com. DNS requests should be leaking.
5. sudo protonvpn d
6. Edit /etc/nsswitch.conf as shown above, removing resolve resolve [!UNAVAIL=return].
7. sudo systemctl restart systemd-resolved
8. sudo protonvpn c -f
9. Go back to dnsleaktest.com. DNS requests should no longer be leaking.
10. sudo protonvpn d
11. Restore /etc/nsswitch.conf file to original config.
12. sudo systemctl restart systemd-resolved

Unfortunately, I've had mixed success with this. Sometimes DNS leaks after removing the nsswitch section, sometimes it doesn't. More digging needs to be done, but I hope this is at least a start.

Arguments for systemd-resolved support

1. It is common for users of OpenVPN to have the openvpn-systemd-resolved package installed to work with their own personal OpenVPN setups. Otherwise, DNS wouldn't work at all. As this is a common configuration for Ubuntu users, I believe the PVPN CLI should support it.
2. Common configuration or not, DNS Leak Protection is an option that the PVPN CLI provides. If I took the CLI at its word, I wouldn't know that my DNS queries were leaking anyway. It should at least detect this unsupported configuration and warn the user that DNS Leak Protection may not be effective. Even better, the CLI could run its own leak test after connecting.
3. ProtonVPN is a major VPN provider and among the higher priced players. Granted, it does have the infrastructure and Windows/macOS/Android/iOS clients to match; support for other platforms is phenominal, but most Linux users can manage with a CLI only. I'd be surprised if ProtonVPN didn't have the resources to make a desktop client for the most commonly used privacy-conscous platform. I admire the amount of work and attention to detail you've put into making this CLI tool. Despite the couple of bugs here and there, it's incredible! But you shouldn't have to do this alone. IMO, an important application such as this should be backed up by at least a small team. Surely there is someone at ProtonVPN that can offer some insight into this sytemd-resolved issue.

There is a mass exodus of users looking to switch away from another major VPN provider, so now may be a good time for the folks at ProtonVPN to step on the gas for Linux support, one of the things the competitor -- to its credit -- does well. I'm new to ProtonVPN and trying a couple of other services too. ProtonVPN stands out as my favorite with the exception of this DNS leak issue.

Again, thanks so much for all of the hard work you've put into this! I hope my findings and comments are helpful to you.

[khicks]

I also toyed with adding a few lines to the template.ovpn file, but didn't get too far. Feels closer, though.

script-security 2
up /etc/openvpn/update-systemd-resolved
down /etc/openvpn/update-systemd-resolved
down-pre
dhcp-option DOMAIN-ROUTE .

Checking systemd-resolve --status, I now have DNS Domain: ~. on the physical and tun0 interfaces. Not sure how to prioritize tun0.

[mrodus]

Hey Rafficer, thanks for taking the time to look into this and verify that openvpn-systemd-resolved causes the issue.

Progress

After further research, I found that it more specifically involves the libnss-resolve (alias for libnss-systemd) package which openvpn-systemd-resolve depends on. Upon installation, it makes the following file change:

# /etc/nsswitch.conf
< hosts:          files mdns4_minimal [NOTFOUND=return] dns
---
> hosts:          files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns

Reverting the change to remove resolve [!UNAVAIL=return] and restarting the systemd-resolved service can prevent DNS leaks. Here's a rough outline of the steps I've tried.

1. Install ProtonVPN and prereqs. Do `protonvpn init`.

2. `sudo apt install openvpn-systemd-resolve`. This installs `libnss-resolve`/`libnss-systemd` too.

3. `sudo protonvpn c -f`

4. Go to dnsleaktest.com. DNS requests should be leaking.

5. `sudo protonvpn d`

6. Edit `/etc/nsswitch.conf` as shown above, removing resolve `resolve [!UNAVAIL=return]`.

7. `sudo systemctl restart systemd-resolved`

8. `sudo protonvpn c -f`

9. Go back to dnsleaktest.com. DNS requests should no longer be leaking.

10. `sudo protonvpn d`

11. Restore `/etc/nsswitch.conf` file to original config.

12. `sudo systemctl restart systemd-resolved`

Unfortunately, I've had mixed success with this. Sometimes DNS leaks after removing the nsswitch section, sometimes it doesn't. More digging needs to be done, but I hope this is at least a start.

Arguments for systemd-resolved support

1. It is [common](https://askubuntu.com/questions/1032476/ubuntu-18-04-no-dns-resolution-when-connected-to-openvpn) for users of OpenVPN to have the `openvpn-systemd-resolved` package installed to work with their own personal OpenVPN setups. Otherwise, DNS wouldn't work at all. As this is a common configuration for Ubuntu users, I believe the PVPN CLI should support it.

2. Common configuration or not, DNS Leak Protection is an option that the PVPN CLI provides. If I took the CLI at its word, I wouldn't know that my DNS queries were leaking anyway. It should at least detect this unsupported configuration and warn the user that DNS Leak Protection may not be effective. Even better, the CLI could run its own leak test after connecting.

3. ProtonVPN is a major VPN provider and among the higher priced players. Granted, it does have the infrastructure and Windows/macOS/Android/iOS clients to match; support for other platforms is phenominal, but most Linux users can manage with a CLI only. I'd be surprised if ProtonVPN didn't have the resources to make a desktop client for the most commonly used privacy-conscous platform. I admire the amount of work and attention to detail you've put into making this CLI tool. Despite the couple of bugs here and there, it's incredible! But you shouldn't have to do this alone. IMO, an important application such as this should be backed up by at least a small team. Surely there is someone at ProtonVPN that can offer some insight into this sytemd-resolved issue.

There is a mass exodus of users looking to switch away from another major VPN provider, so now may be a good time for the folks at ProtonVPN to step on the gas for Linux support, one of the things the competitor -- to its credit -- does well. I'm new to ProtonVPN and trying a couple of other services too. ProtonVPN stands out as my favorite with the exception of this DNS leak issue.

Again, thanks so much for all of the hard work you've put into this! I hope my findings and comments are helpful to you.

I am also facing this issue and this workaround seems to have fixed it for me too. Since this ticket is almost a year old, is there any update on this?

[calexandru2018]

The Linux clients are currently being rewriten, so no new commits will be made to fix this.

[chris-stones]

The Linux clients are currently being rewriten, so no new commits will be made to fix this.

The new linux beta-client does not support headless servers.
Untill it does, we still need to use this client.

Unless Im mistaken, is it not possible to use NetworkManager on a headless server setup?

It wasnt untill i started working on a different bug, that I realised my DNS queries were going to my ISP DNS, via enp0s10.

The client said i had "DNS Leak Protection".. I took it at its word. I am paying for a VPN with DNS leak protection. systemd-resolved users are not getting this, and this bug has been known about for quite some time.

Im really sorry to say, that I shall be looking for a new VPN provider.