gluetun
gluetun copied to clipboard
qdm12
Bug: Kubernetes services cannot be resolved due to DNS overrides and routing conflicts
TLDR: Kubernetes services cannot be resolved anymore because the DNS configuration is being overwritten
-
Is this urgent?
- [ ] Yes
- [x] No
-
What VPN service provider are you using?
- [ ] PIA
- [ ] Mullvad
- [ ] Windscribe
- [x] Surfshark
- [ ] Cyberghost
-
What's the version of the program?
Running version latest built on 2020-07-09T11:57:17Z (commit dc1c7ea)
-
What are you using to run the container?
- [ ] Docker run
- [ ] Docker Compose
- [x] Kubernetes
- [ ] Docker stack
- [ ] Docker swarm
- [ ] Podman
- [ ] Other:
-
Extra information
Logs:
DNS over TLS settings:
|--DNS over TLS provider:
|--cloudflare
|--Caching: disabled
|--Block malicious: disabled
|--Block surveillance: disabled
|--Block ads: disabled
...
2020-07-09T13:22:42.163Z INFO firewall configurator: accepting any input traffic on port 8888
2020-07-09T13:22:42.163Z INFO http server: listening on 0.0.0.0:8000
2020-07-09T13:22:42.163Z INFO dns configurator: using DNS address 1.1.1.1 internally
2020-07-09T13:22:42.163Z INFO dns configurator: using DNS address 1.1.1.1 system wide
2020-07-09T13:22:42.163Z INFO openvpn configurator: writing auth file /etc/openvpn/auth.conf
2020-07-09T13:22:42.164Z INFO openvpn configurator: starting openvpn
2020-07-09T13:22:42.166Z INFO openvpn: OpenVPN 2.4.9 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Apr 20 2020
2020-07-09T13:22:42.166Z INFO openvpn: library versions: OpenSSL 1.1.1g 21 Apr 2020, LZO 2.10
2020-07-09T13:22:42.167Z INFO tinyproxy configurator: starting tinyproxy server
2020-07-09T13:22:42.168Z INFO openvpn: WARNING: you are using user/group/chroot/setcon without persist-tun -- this may cause restarts to fail
2020-07-09T13:22:42.170Z INFO openvpn: TCP/UDP: Preserving recently used remote address: [AF_INET]81.19.209.124:1194
2020-07-09T13:22:42.170Z INFO openvpn: UDP link local: (not bound)
2020-07-09T13:22:42.170Z INFO openvpn: UDP link remote: [AF_INET]81.19.209.124:1194
2020-07-09T13:22:42.170Z INFO openvpn: NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
2020-07-09T13:22:42.182Z INFO openvpn: WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1633', remote='link-mtu 1581'
2020-07-09T13:22:42.182Z INFO openvpn: WARNING: 'cipher' is used inconsistently, local='cipher AES-256-CBC', remote='cipher AES-256-GCM'
2020-07-09T13:22:42.182Z INFO openvpn: WARNING: 'auth' is used inconsistently, local='auth SHA512', remote='auth [null-digest]'
2020-07-09T13:22:42.182Z INFO openvpn: [nl-ams-v024.prod.surfshark.com] Peer Connection Initiated with [AF_INET]81.19.209.124:1194
Configuration file:
apiVersion: v1
kind: Pod
metadata:
name: vpn-test
namespace: default
spec:
containers:
- name: shell
image: ubuntu
command: ['bash']
stdin: true
tty: true
- name: proxy
env:
- name: USER
value: ...
- name: PASSWORD
value: ...
- name: VPNSP
value: 'surfshark'
- name: FIREWALL
value: 'off'
- name: EXTRA_SUBNETS
value: '10.192.0.0/9'
- name: SHADOWSOCKS
value: 'on'
- name: TINYPROXY
value: 'on'
- name: DOT
value: 'on'
- name: DOT_CACHING
value: 'off'
- name: BLOCK_MALICIOUS
value: 'off'
- name: DNS_UPDATE_PERIOD
value: '0'
image: qmcgaw/private-internet-access
imagePullPolicy: Always
ports:
- containerPort: 8888
- containerPort: 8388
- containerPort: 8388
protocol: UDP
- containerPort: 8000
securityContext:
privileged: true
capabilities:
add:
- NET_ADMIN
Host OS: DigitalOcean Kubernetes cluster
I believe that svc.cluster.local should be added to the search parameter in /etc/resolv.conf and that unbound needs to use the internal k8s dns server to resolve those local domainnames.
Running in a normal pod:
root@shell:/# nslookup kube-dns.kube-system
Server: 10.245.0.10
Address: 10.245.0.10#53
Name: kube-dns.kube-system.svc.cluster.local
Address: 10.245.0.10
Running with the VPN sidecar:
root@shell:/# nslookup kube-dns.kube-system
;; Got SERVFAIL reply from 127.0.0.1, trying next server
;; Got SERVFAIL reply from 127.0.0.1, trying next server
Server: 127.0.0.1
Address: 127.0.0.1#53
** server can't find kube-dns.kube-system.default.svc.cluster.local: SERVFAIL
Running from the VPN sidecar:
/ # nslookup kube-dns.kube-system
Server: 127.0.0.1
Address: 127.0.0.1:53
** server can't find kube-dns.kube-system: SERVFAIL
** server can't find kube-dns.kube-system: SERVFAIL
PS: ping is broken in alpine it seems. nslookup works but ping says "bad address" It might be related to ndots: https://github.com/docker/for-linux/issues/755 I've seen the reverse as well: https://unix.stackexchange.com/questions/124283/busybox-ping-ip-works-but-hostname-nslookup-fails-with-bad-address
After looking around a bit I think the only "good" solution is to leave the /etc/resolv.conf alone in kubernetes and fix it by modifying the spec.dnsConfig settings.
I don't think that's possible right now though. It's either writing the local dns server when unbound is enabled or 1.1.1.1 if not. Setting DNS_PLAINTEXT_ADDRESS='' doesn't work either.
Lucky us, specifying multiple nameservers in /etc/resolv.conf works by order. It will try the first one and then the second one etc. on failure.
I'll therefore add whatever DNS before the already existing nameserver instead of overwriting it. That should do it. I'll do that tonight and we can then test.
Let me know if it works for you! There is also a rather big firewall refactor incoming tonight/tomorrow, hopefully I won't break anything 😄 I'll make a release in the coming days as well when it's proven enough stable.
It is. I think it's a routing issue due to the vpn actually
# nslookup kubernetes.default.svc.cluster.local 10.245.0.10
;; connection timed out; no servers could be reached
It's not making sense to me though... I'm trying to get a traceroute app installed so I can test
Well.. I got a little further, the firewall was blocking me. After that (and changing the order in /etc/resolv.conf) it works
In either case... if the kubernetes resolver is not at the top it doesn't work.
While changing the order is an option, I think the better solution would be to specify the k8s dns server as the upstream for svc.cluster.local
For now, either would work though :)
Actually I have to re-change the logic for writing to /etc/resolv.conf because dns blocking won't work with the way it is. I'll probably add an env variable for keeping the other nameservers.
- What order did you use in the end in /etc/resolv.conf?
- Do you mean to add i.e.
nameserver svc.cluster.localto /etc/resolv.conf? That could be an env variable as well.
- The k8s dns server needs to be first, otherwise it doesn’t seem to work
- I mean modifying the unbound configuration to use the k8s dns server for the k8s addresses. So effectively creating a stub zone only for
svc.cluster.local(or.localentirely)
I think it should look something like this:
stub-zone:
name: "svc.cluster.local"
stub-addr: 10.245.0.10
Would there be a way to set it somehow directly in /etc/resolv.conf instead? Can the container know the stub address automatically (10.245.0.10) maybe?
I also just pushed a lot of changes, hopefully nothing should break but let me know 😉 I also put back resolv.conf the way it was before as mentioned above.
Having the k8s dns first at least makes it work. But I'm not sure if it only resolves k8s addresses that way or if it resolves all hostnames through the k8s dns server.
If it does use k8s for all dns that might mean your dns requests leak outside of the vpn connection. With location based dns servers that could be problematic
As for knowing the stub addres... I'm not sure unfortunately. I'm not that well versed with k8s dns yet. I think it's supposed to be passed through the --cluster-dns=... parameter but I'm not sure how/where to extract that: https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#introduction
@qdm12 for the time being, would it be possible to put the custom dns server after the k8s dns server in /etc/resolv.conf? That way it at least works for both :)
Try with DNS_KEEP_NAMESERVER=on it should only append the nameserver to the bottom of /etc/resolv.conf. Although the latest image might still be building.
Oops... tested from the wrong server. Guess it's still building. I'll retry again in a few minutes
Yes, sorry the build has been failing... It's still building now (here) or wait for green mark instead of that orange dot next to the commit referenced above.
For some reason I'm still having issues... it's definitely a routing thing but manually trying to fix it does very little.
The default routes when the VPN is connected:
/ # route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.8.8.1 128.0.0.0 UG 0 0 0 tun0
0.0.0.0 10.244.1.32 0.0.0.0 UG 0 0 0 eth0
10.8.8.0 0.0.0.0 255.255.255.0 U 0 0 0 tun0
10.244.1.32 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
89.187.174.231 10.244.1.32 255.255.255.255 UGH 0 0 0 eth0
128.0.0.0 10.8.8.1 128.0.0.0 UG 0 0 0 tun0
Which enables internet connectivity without any issues. Pinging the k8s gateway works too. But pinging any device (including the dns server) doesn't :/
/ # traceroute -nv 10.245.0.10
traceroute to 10.245.0.10 (10.245.0.10), 30 hops max, 46 byte packets
1 10.8.8.1 54 bytes to (null) 1581.624 ms 1.266 ms 1.251 ms
2 89.187.174.254 54 bytes to (null) 1.307 ms 1.366 ms 1.335 ms
So I figured, let's add a route for that:
/ # route add -net 10.245.0.0/16 gw 10.244.1.32
/ # traceroute -nv 10.245.0.10
traceroute to 10.245.0.10 (10.245.0.10), 30 hops max, 46 byte packets
1 * * *
2 10.244.1.32 54 bytes to (null) 0.006 ms 0.003 ms 0.002 ms
3 161.35.144.254 36 bytes to (null) 2.216 ms 161.35.144.253 36 bytes to (null) 2.410 ms 161.35.144.254 36 bytes to (null) 2.704 ms
Oddly enough... still no dice. I'm probably forgetting about something here or it's because the ranges are overlapping but I'm not getting a connection to k8s yet.
I found the solution! I forgot to add the device when adding the route. If I manually add the route it works now:
ip route add 10.192.0.0/10 via 10.244.1.32 dev eth0
Would it be possible to add a custom script to run after the connection has been set up?
A few questions:
- Do you need the nameserver thing or does this routing solves it by itself?
- Is
10.244.1.32the default gateway? (i.e. for me i get in the logsrouting: default route found: interface tun0, gateway 10.3.10.5, subnet 0.0.0.0/0) - by the way the 0.0.0.0/0 is wrong, I need to fix it.
- Both are needed, otherwise it can't resolve local addresses anymore.
- Yes, it's the default route but I think it's due to overlapping ip ranges because the vpn takes precedence.
- Ok let's keep the DNS as it is now with the env variable
- For now, the program does
ip route add <some-subnet> via <defaultgateway> dev <defaultinterface>for each subnet specified inEXTRA_SUBNETS(as well as some iptables accepting).eth0should be your default interface and10.3.10.5your default gateway, so maybe specifyingEXTRA_SUBNETS=10.192.0.0/10would work?
On a side note, maybe that local subnet being detected as 0.0.0.0/0 was messing everything up, it's fixed in the commit mentioned above, it was probably introduced when switching to Alpine 3.12. Try again with that version and see what local subnet it finds. If 10.192.0.0/10 is actually the local subnet the container is running it, it should automatically work (with that fix).
Also there was a bug the last 1-3 days where the EXTRA_SUBNETS would just be ignored, which is fixed in #190 so that may had affected your testing.
I'm not seeing any difference unfortunately. The routing table looks identical, am I using it incorrectly?
=========================================
================ Gluetun ================
=========================================
==== A mix of OpenVPN, DNS over TLS, ====
======= Shadowsocks and Tinyproxy =======
========= all glued up with Go ==========
=========================================
=========== For tunneling to ============
======== your favorite VPN server =======
=========================================
=== Made with ❤️ by github.com/qdm12 ====
=========================================
Running version latest built on 2020-07-12T19:23:29Z (commit 0731b1c)
🔧 Need help? https://github.com/qdm12/private-internet-access-docker/issues/new
💻 Email? [email protected]
☕ Slack? Join from the Slack button on Github
💸 Help me? https://github.com/sponsors/qdm12
2020-07-12T20:56:54.052Z INFO OpenVPN version: 2.4.9
2020-07-12T20:56:54.054Z INFO Unbound version: 1.10.1
2020-07-12T20:56:54.057Z INFO IPtables version: v1.8.4
2020-07-12T20:56:54.078Z INFO TinyProxy version: 1.10.0
2020-07-12T20:56:54.080Z INFO ShadowSocks version: 3.3.4
2020-07-12T20:56:54.081Z INFO Settings summary below:
OpenVPN settings:
|--User: [redacted]
|--Password: [redacted]
|--Verbosity level: 1
|--Run as root: no
|--Surfshark settings:
|--Network protocol: udp
|--Region: netherlands amsterdam
System settings:
|--User ID: 1000
|--Group ID: 1000
|--Timezone:
|--IP Status filepath: /ip
DNS over TLS settings:
|--DNS over TLS provider:
|--cloudflare
|--Caching: disabled
|--Block malicious: disabled
|--Block surveillance: disabled
|--Block ads: disabled
|--Allowed hostnames:
|--
|--Private addresses:
|--127.0.0.1/8
|--10.0.0.0/8
|--172.16.0.0/12
|--192.168.0.0/16
|--169.254.0.0/16
|--::1/128
|--fc00::/7
|--fe80::/10
|--::ffff:0:0/96
|--Verbosity level: 1/5
|--Verbosity details level: 0/4
|--Validation log level: 0/2
|--IPv6 resolution: disabled
|--Update: deactivated
|--Keep nameserver (disabled blocking): yes
Firewall settings:
|--Allowed subnets: 10.192.0.0/10
TinyProxy settings:
Port: 8888
|--Authentication: disabled
|--Log level: info
ShadowSocks settings: disabled
2020-07-12T20:56:54.081Z INFO openvpn configurator: checking for device /dev/net/tun
2020-07-12T20:56:54.082Z INFO firewall: enabling...
2020-07-12T20:56:54.082Z INFO Launching standard output merger
2020-07-12T20:56:54.082Z INFO routing: default route found: interface eth0, gateway 10.244.1.32
2020-07-12T20:56:54.082Z INFO routing: local subnet found: 10.244.1.32/32
2020-07-12T20:56:54.093Z INFO firewall: enabled successfully
2020-07-12T20:56:54.094Z INFO tinyproxy configurator: generating tinyproxy configuration file
2020-07-12T20:56:54.094Z INFO http server: listening on 0.0.0.0:8000
2020-07-12T20:56:54.094Z INFO firewall: setting VPN connections through firewall...
2020-07-12T20:56:54.094Z INFO dns over tls: falling back on plaintext DNS at address 1.1.1.1
2020-07-12T20:56:54.094Z INFO dns configurator: using DNS address 1.1.1.1 internally
2020-07-12T20:56:54.095Z INFO dns configurator: using DNS address 1.1.1.1 system wide
2020-07-12T20:56:54.095Z INFO routing: default route found: interface eth0, gateway 10.244.1.32
2020-07-12T20:56:54.123Z INFO openvpn configurator: starting openvpn
2020-07-12T20:56:54.123Z INFO firewall: setting allowed port 8888 through firewall...
2020-07-12T20:56:54.125Z INFO openvpn: OpenVPN 2.4.9 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Apr 20 2020
2020-07-12T20:56:54.125Z INFO openvpn: library versions: OpenSSL 1.1.1g 21 Apr 2020, LZO 2.10
2020-07-12T20:56:54.126Z INFO openvpn: WARNING: you are using user/group/chroot/setcon without persist-tun -- this may cause restarts to fail
2020-07-12T20:56:54.129Z INFO openvpn: TCP/UDP: Preserving recently used remote address: [AF_INET]185.59.222.92:1194
2020-07-12T20:56:54.129Z INFO openvpn: UDP link local: (not bound)
2020-07-12T20:56:54.129Z INFO openvpn: UDP link remote: [AF_INET]185.59.222.92:1194
2020-07-12T20:56:54.129Z INFO openvpn: NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
2020-07-12T20:56:54.131Z INFO tinyproxy configurator: starting tinyproxy server
2020-07-12T20:56:54.145Z INFO openvpn: WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1633', remote='link-mtu 1581'
2020-07-12T20:56:54.145Z INFO openvpn: WARNING: 'cipher' is used inconsistently, local='cipher AES-256-CBC', remote='cipher AES-256-GCM'
2020-07-12T20:56:54.145Z INFO openvpn: WARNING: 'auth' is used inconsistently, local='auth SHA512', remote='auth [null-digest]'
2020-07-12T20:56:54.145Z INFO openvpn: [nl-ams-v031.prod.surfshark.com] Peer Connection Initiated with [AF_INET]185.59.222.92:1194
2020-07-12T20:56:54.157Z INFO tinyproxy: INFO Initializing tinyproxy ...
2020-07-12T20:56:54.158Z INFO tinyproxy: INFO Reloading config file
2020-07-12T20:56:54.158Z INFO tinyproxy: INFO Disabling transmission of the "Via" header.
2020-07-12T20:56:54.158Z INFO tinyproxy: INFO listen_sock called with addr = '(NULL)'
2020-07-12T20:56:54.158Z INFO tinyproxy: INFO trying to listen on host[0.0.0.0], family[2], socktype[1], proto[6]
2020-07-12T20:56:54.158Z INFO tinyproxy: INFO listening on fd [4]
2020-07-12T20:56:54.158Z INFO tinyproxy: INFO trying to listen on host[::], family[10], socktype[1], proto[6]
2020-07-12T20:56:54.158Z INFO tinyproxy: INFO listening on fd [5]
2020-07-12T20:56:54.159Z INFO tinyproxy: INFO Now running as group "1000".
2020-07-12T20:56:54.159Z INFO tinyproxy: INFO Now running as user "1000".
2020-07-12T20:56:54.159Z INFO tinyproxy: INFO Creating child number 1 of 10 ...
2020-07-12T20:56:54.159Z INFO tinyproxy: INFO Creating child number 2 of 10 ...
2020-07-12T20:56:54.159Z INFO tinyproxy: INFO Creating child number 3 of 10 ...
2020-07-12T20:56:54.159Z INFO tinyproxy: INFO Creating child number 4 of 10 ...
2020-07-12T20:56:54.159Z INFO tinyproxy: INFO Creating child number 5 of 10 ...
2020-07-12T20:56:54.159Z INFO tinyproxy: INFO Creating child number 6 of 10 ...
2020-07-12T20:56:54.160Z INFO tinyproxy: INFO Creating child number 7 of 10 ...
2020-07-12T20:56:54.160Z INFO tinyproxy: INFO Creating child number 8 of 10 ...
2020-07-12T20:56:54.160Z INFO tinyproxy: INFO Creating child number 9 of 10 ...
2020-07-12T20:56:54.160Z INFO tinyproxy: INFO Creating child number 10 of 10 ...
2020-07-12T20:56:54.160Z INFO tinyproxy: INFO Finished creating all children.
2020-07-12T20:56:54.161Z INFO tinyproxy: INFO Setting the various signals.
2020-07-12T20:56:54.161Z INFO tinyproxy: INFO Starting main loop. Accepting connections.
2020-07-12T20:56:55.185Z INFO openvpn: Options error: Unrecognized option or missing or extra parameter(s) in [PUSH-OPTIONS]:7: block-outside-dns (2.4.9)
2020-07-12T20:56:55.186Z INFO openvpn: TUN/TAP device tun0 opened
2020-07-12T20:56:55.186Z INFO openvpn: /sbin/ip link set dev tun0 up mtu 1500
2020-07-12T20:56:55.187Z INFO openvpn: /sbin/ip addr add dev tun0 10.8.8.19/24 broadcast 10.8.8.255
2020-07-12T20:56:55.190Z INFO openvpn: UID set to nonrootuser
2020-07-12T20:56:55.190Z INFO openvpn: Initialization Sequence Completed
2020-07-12T20:56:55.190Z INFO dns configurator: downloading root hints from https://raw.githubusercontent.com/qdm12/files/master/named.root.updated
2020-07-12T20:56:55.190Z INFO routing: default route found: interface eth0, gateway 10.244.1.32
2020-07-12T20:56:55.190Z INFO Gateway VPN IP address: 185.59.222.92
2020-07-12T20:56:55.458Z INFO ip getter: Public IP address is 185.59.222.93
2020-07-12T20:56:55.530Z INFO dns configurator: downloading root key from https://raw.githubusercontent.com/qdm12/files/master/root.key.updated
2020-07-12T20:56:55.707Z INFO dns configurator: generating Unbound configuration
2020-07-12T20:56:55.707Z INFO dns configurator: 0 hostnames blocked overall
2020-07-12T20:56:55.707Z INFO dns configurator: 9 IP addresses blocked overall
2020-07-12T20:56:55.708Z INFO dns configurator: starting unbound
2020-07-12T20:56:55.708Z INFO dns configurator: using DNS address 127.0.0.1 internally
2020-07-12T20:56:55.708Z INFO dns configurator: using DNS address 127.0.0.1 system wide
2020-07-12T20:56:55.723Z INFO unbound: notice: init module 0: validator
2020-07-12T20:56:55.724Z INFO unbound: notice: init module 1: iterator
2020-07-12T20:56:55.757Z INFO unbound: info: start of service (unbound 1.10.1).
2020-07-12T20:56:56.031Z INFO unbound: info: generate keytag query _ta-4a5c-4f66. NULL IN
Little question... is the route being added after connecting to the VPN or before? I think it can only work if it's being added after being connected every time. The route could also be added to the ovpn file.
- It says your local subnet is your default gateway
routing: local subnet found: 10.244.1.32/32. In my standard Docker thing, it gives me the Docker network subnet i.e.172.17.0.0/16. Can you try to run a container and send the output fromip routeand the output fromcat cat /proc/net/route? Maybe my routing parsing goes wrong with your setup, I'll try to fix it. It would help me understand a bit what's going on too. - The routes for each extra subnets are added before the vpn is connected, but it's easy to change that. Although I don't believe it would matter (correct me if I'm wrong)
Yeah, here you go :)
/ # cat /proc/net/route
Iface Destination Gateway Flags RefCnt Use Metric Mask MTU Window IRTT
tun0 00000000 0108080A 0003 0 0 0 00000080 0 0 0
eth0 00000000 2001F40A 0003 0 0 0 00000000 0 0 0
tun0 0008080A 00000000 0001 0 0 0 00FFFFFF 0 0 0
eth0 2001F40A 00000000 0005 0 0 0 FFFFFFFF 0 0 0
tun0 00000080 0108080A 0003 0 0 0 00000080 0 0 0
eth0 5CDE3BB9 2001F40A 0007 0 0 0 FFFFFFFF 0 0 0
/ # ip route
0.0.0.0/1 via 10.8.8.1 dev tun0
default via 10.244.1.32 dev eth0 mtu 1450
10.8.8.0/24 dev tun0 proto kernel scope link src 10.8.8.19
10.244.1.32 dev eth0 scope link
128.0.0.0/1 via 10.8.8.1 dev tun0
185.59.222.92 via 10.244.1.32 dev eth0
As far as I have experienced so far the last route added takes precedence, unless the metric parameter is given of course. So I do think it matters to only add it after the VPN has connected
Also the fix I just mentioned about EXTRA_SUBNETS I pushed it to the wrong branch, I just pushed it to master now, sorry about that. So it should show in the latest docker image soon. (see #190). Gonna go eat, will check back those routes soon. Thanks!
EDIT: That 0.0.0.0/1 is an error, fixed a few seconds ago.
The new results:
/ # cat /proc/net/route
Iface Destination Gateway Flags RefCnt Use Metric Mask MTU Window IRTT
tun0 00000000 0108080A 0003 0 0 0 00000080 0 0 0
eth0 00000000 2001F40A 0003 0 0 0 00000000 0 0 0
tun0 0008080A 00000000 0001 0 0 0 00FFFFFF 0 0 0
eth0 2001F40A 00000000 0005 0 0 0 FFFFFFFF 0 0 0
tun0 00000080 0108080A 0003 0 0 0 00000080 0 0 0
eth0 5CDE3BB9 2001F40A 0007 0 0 0 FFFFFFFF 0 0 0
/ #
/ # %
(likegrowers-webliker) rick@lappie# kubectl exec --stdin --tty -n default surfshark-netherlands-000 -c proxy -- /bin/sh
/ # cat /proc/net/route
Iface Destination Gateway Flags RefCnt Use Metric Mask MTU Window IRTT
tun0 00000000 0108080A 0003 0 0 0 00000080 0 0 0
eth0 00000000 2001F40A 0003 0 0 0 00000000 0 0 0
tun0 0008080A 00000000 0001 0 0 0 00FFFFFF 0 0 0
eth0 0000C00A 2001F40A 0003 0 0 0 0000C0FF 0 0 0
eth0 2001F40A 00000000 0005 0 0 0 FFFFFFFF 0 0 0
tun0 00000080 0108080A 0003 0 0 0 00000080 0 0 0
eth0 C22366D4 2001F40A 0007 0 0 0 FFFFFFFF 0 0 0
/ # route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 10.8.8.1 128.0.0.0 UG 0 0 0 tun0
default 10.244.1.32 0.0.0.0 UG 0 0 0 eth0
10.8.8.0 * 255.255.255.0 U 0 0 0 tun0
10.192.0.0 10.244.1.32 255.192.0.0 UG 0 0 0 eth0
10.244.1.32 * 255.255.255.255 UH 0 0 0 eth0
128.0.0.0 10.8.8.1 128.0.0.0 UG 0 0 0 tun0
212.102.35.194 10.244.1.32 255.255.255.255 UGH 0 0 0 eth0
/ # ip route
0.0.0.0/1 via 10.8.8.1 dev tun0
default via 10.244.1.32 dev eth0 mtu 1450
10.8.8.0/24 dev tun0 proto kernel scope link src 10.8.8.29
10.192.0.0/10 via 10.244.1.32 dev eth0
10.244.1.32 dev eth0 scope link
128.0.0.0/1 via 10.8.8.1 dev tun0
212.102.35.194 via 10.244.1.32 dev eth0
And the new logs:
=========================================
================ Gluetun ================
=========================================
==== A mix of OpenVPN, DNS over TLS, ====
======= Shadowsocks and Tinyproxy =======
========= all glued up with Go ==========
=========================================
=========== For tunneling to ============
======== your favorite VPN server =======
=========================================
=== Made with ❤️ by github.com/qdm12 ====
=========================================
Running version latest built on 2020-07-12T21:22:24Z (commit 4cd6b33)
🔧 Need help? https://github.com/qdm12/private-internet-access-docker/issues/new
💻 Email? [email protected]
☕ Slack? Join from the Slack button on Github
💸 Help me? https://github.com/sponsors/qdm12
2020-07-12T21:36:06.482Z INFO Unbound version: 1.10.1
2020-07-12T21:36:06.483Z INFO IPtables version: v1.8.4
2020-07-12T21:36:06.514Z INFO TinyProxy version: 1.10.0
2020-07-12T21:36:06.526Z INFO ShadowSocks version: 3.3.4
2020-07-12T21:36:06.536Z INFO OpenVPN version: 2.4.9
2020-07-12T21:36:06.536Z INFO Settings summary below:
OpenVPN settings:
|--User: [redacted]
|--Password: [redacted]
|--Verbosity level: 1
|--Run as root: no
|--Surfshark settings:
|--Network protocol: udp
|--Region: netherlands amsterdam
System settings:
|--User ID: 1000
|--Group ID: 1000
|--Timezone:
|--IP Status filepath: /ip
DNS over TLS settings:
|--DNS over TLS provider:
|--cloudflare
|--Caching: disabled
|--Block malicious: disabled
|--Block surveillance: disabled
|--Block ads: disabled
|--Allowed hostnames:
|--
|--Private addresses:
|--127.0.0.1/8
|--10.0.0.0/8
|--172.16.0.0/12
|--192.168.0.0/16
|--169.254.0.0/16
|--::1/128
|--fc00::/7
|--fe80::/10
|--::ffff:0:0/96
|--Verbosity level: 1/5
|--Verbosity details level: 0/4
|--Validation log level: 0/2
|--IPv6 resolution: disabled
|--Update: deactivated
|--Keep nameserver (disabled blocking): yes
Firewall settings:
|--Allowed subnets: 10.192.0.0/10
TinyProxy settings:
Port: 8888
|--Authentication: disabled
|--Log level: info
ShadowSocks settings: disabled
2020-07-12T21:36:06.537Z INFO openvpn configurator: checking for device /dev/net/tun
2020-07-12T21:36:06.537Z INFO firewall: enabling...
2020-07-12T21:36:06.537Z INFO Launching standard output merger
2020-07-12T21:36:06.537Z INFO routing: default route found: interface eth0, gateway 10.244.1.32
2020-07-12T21:36:06.537Z INFO routing: local subnet found: 10.244.1.32/32
2020-07-12T21:36:06.576Z INFO firewall: enabled successfully
2020-07-12T21:36:06.576Z INFO firewall: setting allowed subnets through firewall...
2020-07-12T21:36:06.576Z INFO routing: default route found: interface eth0, gateway 10.244.1.32
2020-07-12T21:36:06.578Z INFO routing: adding 10.192.0.0/10 as route via 10.244.1.32 eth0
2020-07-12T21:36:06.580Z INFO dns over tls: falling back on plaintext DNS at address 1.1.1.1
2020-07-12T21:36:06.580Z INFO dns configurator: using DNS address 1.1.1.1 internally
2020-07-12T21:36:06.580Z INFO dns configurator: using DNS address 1.1.1.1 system wide
2020-07-12T21:36:06.581Z INFO tinyproxy configurator: generating tinyproxy configuration file
2020-07-12T21:36:06.581Z INFO firewall: setting allowed port 8888 through firewall...
2020-07-12T21:36:06.581Z INFO http server: listening on 0.0.0.0:8000
2020-07-12T21:36:06.583Z INFO tinyproxy configurator: starting tinyproxy server
2020-07-12T21:36:06.583Z INFO firewall: setting VPN connections through firewall...
2020-07-12T21:36:06.584Z INFO routing: default route found: interface eth0, gateway 10.244.1.32
2020-07-12T21:36:06.610Z INFO tinyproxy: INFO Initializing tinyproxy ...
2020-07-12T21:36:06.611Z INFO tinyproxy: INFO Reloading config file
2020-07-12T21:36:06.611Z INFO tinyproxy: INFO Disabling transmission of the "Via" header.
2020-07-12T21:36:06.611Z INFO tinyproxy: INFO listen_sock called with addr = '(NULL)'
2020-07-12T21:36:06.611Z INFO tinyproxy: INFO trying to listen on host[0.0.0.0], family[2], socktype[1], proto[6]
2020-07-12T21:36:06.611Z INFO tinyproxy: INFO listening on fd [4]
2020-07-12T21:36:06.612Z INFO tinyproxy: INFO trying to listen on host[::], family[10], socktype[1], proto[6]
2020-07-12T21:36:06.612Z INFO tinyproxy: INFO listening on fd [5]
2020-07-12T21:36:06.612Z INFO tinyproxy: INFO Now running as group "1000".
2020-07-12T21:36:06.612Z INFO tinyproxy: INFO Now running as user "1000".
2020-07-12T21:36:06.613Z INFO tinyproxy: INFO Creating child number 1 of 10 ...
2020-07-12T21:36:06.614Z INFO tinyproxy: INFO Creating child number 2 of 10 ...
2020-07-12T21:36:06.614Z INFO tinyproxy: INFO Creating child number 3 of 10 ...
2020-07-12T21:36:06.615Z INFO tinyproxy: INFO Creating child number 4 of 10 ...
2020-07-12T21:36:06.615Z INFO tinyproxy: INFO Creating child number 5 of 10 ...
2020-07-12T21:36:06.616Z INFO tinyproxy: INFO Creating child number 6 of 10 ...
2020-07-12T21:36:06.616Z INFO tinyproxy: INFO Creating child number 7 of 10 ...
2020-07-12T21:36:06.616Z INFO tinyproxy: INFO Creating child number 8 of 10 ...
2020-07-12T21:36:06.617Z INFO tinyproxy: INFO Creating child number 9 of 10 ...
2020-07-12T21:36:06.617Z INFO tinyproxy: INFO Creating child number 10 of 10 ...
2020-07-12T21:36:06.617Z INFO tinyproxy: INFO Finished creating all children.
2020-07-12T21:36:06.618Z INFO tinyproxy: INFO Setting the various signals.
2020-07-12T21:36:06.618Z INFO tinyproxy: INFO Starting main loop. Accepting connections.
2020-07-12T21:36:06.624Z INFO openvpn configurator: starting openvpn
2020-07-12T21:36:06.626Z INFO openvpn: OpenVPN 2.4.9 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Apr 20 2020
2020-07-12T21:36:06.626Z INFO openvpn: library versions: OpenSSL 1.1.1g 21 Apr 2020, LZO 2.10
2020-07-12T21:36:06.627Z INFO openvpn: WARNING: you are using user/group/chroot/setcon without persist-tun -- this may cause restarts to fail
2020-07-12T21:36:06.629Z INFO openvpn: TCP/UDP: Preserving recently used remote address: [AF_INET]212.102.35.194:1194
2020-07-12T21:36:06.629Z INFO openvpn: UDP link local: (not bound)
2020-07-12T21:36:06.629Z INFO openvpn: UDP link remote: [AF_INET]212.102.35.194:1194
2020-07-12T21:36:06.629Z INFO openvpn: NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
2020-07-12T21:36:06.640Z INFO openvpn: WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1633', remote='link-mtu 1581'
2020-07-12T21:36:06.640Z INFO openvpn: WARNING: 'cipher' is used inconsistently, local='cipher AES-256-CBC', remote='cipher AES-256-GCM'
2020-07-12T21:36:06.640Z INFO openvpn: WARNING: 'auth' is used inconsistently, local='auth SHA512', remote='auth [null-digest]'
2020-07-12T21:36:06.640Z INFO openvpn: [nl-ams-v063.prod.surfshark.com] Peer Connection Initiated with [AF_INET]212.102.35.194:1194
2020-07-12T21:36:07.775Z INFO openvpn: Options error: Unrecognized option or missing or extra parameter(s) in [PUSH-OPTIONS]:7: block-outside-dns (2.4.9)
2020-07-12T21:36:07.776Z INFO openvpn: TUN/TAP device tun0 opened
2020-07-12T21:36:07.776Z INFO openvpn: /sbin/ip link set dev tun0 up mtu 1500
2020-07-12T21:36:07.777Z INFO openvpn: /sbin/ip addr add dev tun0 10.8.8.29/24 broadcast 10.8.8.255
2020-07-12T21:36:07.780Z INFO openvpn: UID set to nonrootuser
2020-07-12T21:36:07.780Z INFO openvpn: Initialization Sequence Completed
2020-07-12T21:36:07.780Z INFO dns configurator: downloading root hints from https://raw.githubusercontent.com/qdm12/files/master/named.root.updated
2020-07-12T21:36:07.780Z INFO routing: default route found: interface eth0, gateway 10.244.1.32
2020-07-12T21:36:07.780Z INFO Gateway VPN IP address: 212.102.35.194
2020-07-12T21:36:07.917Z INFO dns configurator: downloading root key from https://raw.githubusercontent.com/qdm12/files/master/root.key.updated
2020-07-12T21:36:07.921Z INFO dns configurator: generating Unbound configuration
2020-07-12T21:36:07.922Z INFO dns configurator: 0 hostnames blocked overall
2020-07-12T21:36:07.922Z INFO dns configurator: 9 IP addresses blocked overall
2020-07-12T21:36:07.922Z INFO dns configurator: starting unbound
2020-07-12T21:36:07.922Z INFO dns configurator: using DNS address 127.0.0.1 internally
2020-07-12T21:36:07.923Z INFO dns configurator: using DNS address 127.0.0.1 system wide
2020-07-12T21:36:07.937Z INFO unbound: notice: init module 0: validator
2020-07-12T21:36:07.938Z INFO unbound: notice: init module 1: iterator
2020-07-12T21:36:07.980Z INFO unbound: info: start of service (unbound 1.10.1).
2020-07-12T21:36:08.043Z INFO ip getter: Public IP address is 212.102.35.195
2020-07-12T21:36:08.241Z INFO unbound: info: generate keytag query _ta-4a5c-4f66. NULL IN
Oddly enough, manually removing and adding the route doesn't help either now. I can't ping anything besides the k8s gateway
I should note that when using the machine as a separate k8s pod with tinyproxy as a service it does work now. It's only as a sidecar that it doesn't work
There is actually an issue still (#194) which might be related. Sorry about that, I'll fix that one first and would probably ask you to test again then once it's fixed. Have a good week!
Thanks for all the help!
For now I'll just run the VPN as separate deployments and I'll try the sidecar option again in the future :)
Alright, found/pushed the fix! You could try now (or anytime later, no pressure). The routes were setup correctly but the firewall would be blocking everything. Not sure it's very relevant to your situation, but perhaps.
I had the firewall disabled to make sure I didn't have any issues with it. But enabling it immediately disallows pinging the k8s IP addresses.
The routing table looks a bit odd to me:
/ # ip route show table local
broadcast 10.8.8.0 dev tun0 proto kernel scope link src 10.8.8.3
local 10.8.8.3 dev tun0 proto kernel scope host src 10.8.8.3
broadcast 10.8.8.255 dev tun0 proto kernel scope link src 10.8.8.3
local 10.244.2.29 dev eth0 proto kernel scope host src 10.244.2.29
broadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1
local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1
local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1
/ # ip route show table main
0.0.0.0/1 via 10.8.8.1 dev tun0
default via 10.244.2.110 dev eth0 mtu 1450
10.8.8.0/24 dev tun0 proto kernel scope link src 10.8.8.3
10.128.0.0/9 via 10.244.2.110 dev eth0
10.244.2.110 dev eth0 scope link
89.46.223.214 via 10.244.2.110 dev eth0
128.0.0.0/1 via 10.8.8.1 dev tun0
But... it's working! For some reason I can't ping the k8s dns server anymore but that's not working from other machines either... no clue how/why but it looks like k8s changed something. But it does work! :D
I do think something is wrong with the firewall system though as it's blocking everything when enabled. The current firewall rules:
/ # iptables -vnL
Chain INPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
76 6480 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
4574 2170K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 ACCEPT all -- * * 10.244.2.110 10.244.2.110
0 0 ACCEPT all -- eth0 * 10.128.0.0/9 10.244.2.110
0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8888
0 0 ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:8888
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy DROP 8 packets, 772 bytes)
pkts bytes target prot opt in out source destination
76 6480 ACCEPT all -- * lo 0.0.0.0/0 0.0.0.0/0
4667 527K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
226 14244 ACCEPT all -- * tun0 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- * * 10.244.2.110 10.244.2.110
0 0 ACCEPT all -- * eth0 10.244.2.110 10.128.0.0/9
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 81.19.208.56 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 81.19.208.66 udp dpt:1194
1 114 ACCEPT udp -- * eth0 0.0.0.0/0 81.19.208.68 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 81.19.209.20 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 81.19.209.57 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 81.19.209.113 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 81.19.209.120 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 81.19.209.124 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 89.46.223.72 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 89.46.223.78 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 89.46.223.104 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 89.46.223.212 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 89.46.223.214 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 89.46.223.217 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 89.46.223.222 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 89.46.223.229 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 89.187.174.229 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 89.187.174.231 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 185.59.222.92 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 185.59.222.94 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 185.59.222.166 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 185.59.222.168 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 212.102.35.194 udp dpt:1194
0 0 ACCEPT udp -- * eth0 0.0.0.0/0 212.102.35.196 udp dpt:1194
/ # ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 100
link/none
inet 10.8.8.4/24 brd 10.8.8.255 scope global tun0
valid_lft forever preferred_lft forever
240: eth0@if241: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether fa:16:d3:cb:99:04 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.244.2.122/32 scope global eth0
valid_lft forever preferred_lft forever
The firewall rules look wrong. It's filtering on 10.244.2.110 as a source IP which is the gateway so that won't work. But with the firewall disabled everything works beyond that. Thanks for all the help!
Hello, sorry I lost track a bit. Let's find how to make it work with the firewall on. Keeping it off disables the kill switch and might be dangerous i.e. vpn provider misbehaving and trying to reach your system through the tunnel.
Just repeating myself, but to be sure:
- what's the
local subnetshown in the logs of gluetun? - what's the k8s ip addresses you would need to ping?
And to be sure, starting the container with the firewall on just fails straight away right? Thanks!
Took a while for me to get back to this... it's been quite a busy period
Anyhow, here's the log:
=========================================
================ Gluetun ================
=========================================
==== A mix of OpenVPN, DNS over TLS, ====
======= Shadowsocks and Tinyproxy =======
========= all glued up with Go ==========
=========================================
=========== For tunneling to ============
======== your favorite VPN server =======
=========================================
=== Made with ?? by github.com/qdm12 ====
=========================================
Running version latest built on 2020-09-05T16:57:43Z (commit 797fa33)
?? Persistent server IP addresses at /gluetun/servers.json, please BIND MOUNT
?? Need help? https://github.com/qdm12/gluetun/issues/new
?? Email? [email protected]
? Slack? Join from the Slack button on Github
?? Help me? https://github.com/sponsors/qdm12
2020-09-10T14:37:44.416Z INFO TinyProxy version: 1.10.0
2020-09-10T14:37:44.427Z INFO OpenVPN version: 2.4.9
2020-09-10T14:37:44.431Z INFO Unbound version: 1.10.1
2020-09-10T14:37:44.454Z INFO IPtables version: v1.8.4
2020-09-10T14:37:44.454Z INFO Settings summary below:
OpenVPN settings:
|--User: [redacted]
|--Password: [redacted]
|--Verbosity level: 1
|--Run as root: no
|--Surfshark settings:
|--Network protocol: udp
|--Region: us los angeles
System settings:
|--User ID: 1000
|--Group ID: 1000
|--Timezone:
|--IP Status filepath: /tmp/gluetun/ip
DNS over TLS settings:
|--DNS over TLS provider:
|--cloudflare
|--Caching: disabled
|--Block malicious: disabled
|--Block surveillance: disabled
|--Block ads: disabled
|--Allowed hostnames:
|--
|--Private addresses:
|--127.0.0.1/8
|--10.0.0.0/8
|--172.16.0.0/12
|--192.168.0.0/16
|--169.254.0.0/16
|--::1/128
|--fc00::/7
|--fe80::/10
|--::ffff:0:0/96
|--Verbosity level: 1/5
|--Verbosity details level: 0/4
|--Validation log level: 0/2
|--IPv6 resolution: disabled
|--Update: deactivated
|--Keep nameserver (disabled blocking): yes
Firewall settings: disabled
TinyProxy settings:
Port: 8888
|--Authentication: disabled
|--Log level: info
ShadowSocks settings:
|--Password: [redacted]
|--Log: disabled
|--Port: 8388
|--Method: chacha20-ietf-poly1305
Public IP check period: 12h0m0s
Version information: enabled
2020-09-10T14:37:44.455Z INFO storage: Merging by most recent 6355 hardcoded servers and 0 servers read from /gluetun/servers.json
2020-09-10T14:37:44.507Z INFO routing: default route found: interface eth0, gateway 10.244.4.152
2020-09-10T14:37:44.507Z INFO routing: local subnet found: 10.244.4.152/32
2020-09-10T14:37:44.507Z INFO openvpn configurator: checking for device /dev/net/tun
2020-09-10T14:37:44.507Z INFO firewall: firewall disabled, only updating allowed subnets internal list and updating routes
2020-09-10T14:37:44.507Z INFO routing: adding 10.0.0.0/8 as route via 10.244.4.152 eth0
2020-09-10T14:37:44.511Z INFO Launching standard output merger
2020-09-10T14:37:44.511Z INFO dns over tls: falling back on plaintext DNS at address 1.1.1.1
2020-09-10T14:37:44.511Z INFO dns configurator: using DNS address 1.1.1.1 internally
2020-09-10T14:37:44.511Z INFO dns configurator: using DNS address 1.1.1.1 system wide
2020-09-10T14:37:44.511Z INFO tinyproxy configurator: generating tinyproxy configuration file
2020-09-10T14:37:44.512Z INFO firewall: firewall disabled, only updating allowed ports internal state
2020-09-10T14:37:44.512Z INFO tinyproxy configurator: starting tinyproxy server
2020-09-10T14:37:44.529Z INFO firewall: firewall disabled, only updating allowed ports internal state
2020-09-10T14:37:44.530Z INFO firewall: firewall disabled, only updating VPN connections internal list
2020-09-10T14:37:44.530Z INFO openvpn configurator: starting openvpn
2020-09-10T14:37:44.531Z INFO http server: listening on 0.0.0.0:8000
2020-09-10T14:37:44.533Z INFO openvpn: OpenVPN 2.4.9 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Apr 20 2020
2020-09-10T14:37:44.534Z INFO openvpn: library versions: OpenSSL 1.1.1g 21 Apr 2020, LZO 2.10
2020-09-10T14:37:44.541Z INFO openvpn: TCP/UDP: Preserving recently used remote address: [AF_INET]192.111.134.205:1194
2020-09-10T14:37:44.541Z INFO openvpn: UDP link local: (not bound)
2020-09-10T14:37:44.541Z INFO openvpn: UDP link remote: [AF_INET]192.111.134.205:1194
2020-09-10T14:37:44.543Z INFO tinyproxy: Initializing tinyproxy ...
2020-09-10T14:37:44.543Z INFO tinyproxy: Reloading config file
2020-09-10T14:37:44.544Z INFO tinyproxy: Disabling transmission of the "Via" header.
2020-09-10T14:37:44.544Z INFO tinyproxy: listen_sock called with addr = '(NULL)'
2020-09-10T14:37:44.544Z INFO tinyproxy: trying to listen on host[0.0.0.0], family[2], socktype[1], proto[6]
2020-09-10T14:37:44.544Z INFO tinyproxy: listening on fd [4]
2020-09-10T14:37:44.544Z INFO tinyproxy: trying to listen on host[::], family[10], socktype[1], proto[6]
2020-09-10T14:37:44.544Z INFO tinyproxy: listening on fd [5]
2020-09-10T14:37:44.544Z INFO tinyproxy: Now running as group "1000".
2020-09-10T14:37:44.544Z INFO tinyproxy: Now running as user "1000".
2020-09-10T14:37:44.545Z INFO tinyproxy: Creating child number 1 of 10 ...
2020-09-10T14:37:44.545Z INFO tinyproxy: Creating child number 2 of 10 ...
2020-09-10T14:37:44.545Z INFO tinyproxy: Creating child number 3 of 10 ...
2020-09-10T14:37:44.545Z INFO tinyproxy: Creating child number 4 of 10 ...
2020-09-10T14:37:44.546Z INFO tinyproxy: Creating child number 5 of 10 ...
2020-09-10T14:37:44.546Z INFO tinyproxy: Creating child number 6 of 10 ...
2020-09-10T14:37:44.546Z INFO tinyproxy: Creating child number 7 of 10 ...
2020-09-10T14:37:44.546Z INFO tinyproxy: Creating child number 8 of 10 ...
2020-09-10T14:37:44.547Z INFO tinyproxy: Creating child number 9 of 10 ...
2020-09-10T14:37:44.547Z INFO tinyproxy: Creating child number 10 of 10 ...
2020-09-10T14:37:44.547Z INFO tinyproxy: Finished creating all children.
2020-09-10T14:37:44.547Z INFO tinyproxy: Setting the various signals.
2020-09-10T14:37:44.547Z INFO tinyproxy: Starting main loop. Accepting connections.
2020-09-10T14:37:45.287Z WARN openvpn: 'link-mtu' is used inconsistently, local='link-mtu 1633', remote='link-mtu 1581'
2020-09-10T14:37:45.287Z WARN openvpn: 'cipher' is used inconsistently, local='cipher AES-256-CBC', remote='cipher AES-256-GCM'
2020-09-10T14:37:45.287Z WARN openvpn: 'auth' is used inconsistently, local='auth SHA512', remote='auth [null-digest]'
2020-09-10T14:37:45.287Z INFO openvpn: [us-lax-v026.prod.surfshark.com] Peer Connection Initiated with [AF_INET]192.111.134.205:1194
2020-09-10T14:37:46.485Z ERROR openvpn: Unrecognized option or missing or extra parameter(s) in [PUSH-OPTIONS]:7: block-outside-dns (2.4.9)
2020-09-10T14:37:46.487Z INFO openvpn: TUN/TAP device tun0 opened
2020-09-10T14:37:46.487Z INFO openvpn: /sbin/ip link set dev tun0 up mtu 1500
2020-09-10T14:37:46.489Z INFO openvpn: /sbin/ip addr add dev tun0 10.8.8.2/24 broadcast 10.8.8.255
2020-09-10T14:37:46.512Z INFO openvpn: UID set to nonrootuser
2020-09-10T14:37:46.513Z INFO openvpn: Initialization Sequence Completed
2020-09-10T14:37:46.513Z INFO routing: default route found: interface eth0, gateway 10.244.4.152
2020-09-10T14:37:46.513Z INFO Gateway VPN IP address: 192.111.134.205
2020-09-10T14:37:46.513Z INFO dns configurator: downloading root hints from https://raw.githubusercontent.com/qdm12/files/master/named.root.updated
2020-09-10T14:37:47.752Z INFO ip getter: Public IP address is 192.111.134.206
2020-09-10T14:37:47.841Z INFO dns configurator: downloading root key from https://raw.githubusercontent.com/qdm12/files/master/root.key.updated
2020-09-10T14:37:47.939Z INFO You are running 3 commits behind the most recent latest
2020-09-10T14:37:47.995Z INFO dns configurator: generating Unbound configuration
2020-09-10T14:37:47.995Z INFO dns configurator: 0 hostnames blocked overall
2020-09-10T14:37:47.995Z INFO dns configurator: 9 IP addresses blocked overall
2020-09-10T14:37:47.996Z INFO dns configurator: starting unbound
2020-09-10T14:37:47.996Z INFO dns configurator: using DNS address 127.0.0.1 internally
2020-09-10T14:37:47.996Z INFO dns configurator: using DNS address 127.0.0.1 system wide
2020-09-10T14:37:48.014Z INFO unbound: init module 0: validator
2020-09-10T14:37:48.015Z INFO unbound: init module 1: iterator
2020-09-10T14:37:48.050Z INFO unbound: start of service (unbound 1.10.1).
2020-09-10T14:37:48.895Z INFO unbound: generate keytag query _ta-4a5c-4f66. NULL IN
As for IPs that I need to reach, here's a list of examples:
- 10.244.2.253
- 10.244.3.129
- 10.244.0.140
I think there are also some addresses in the 10.245.0.0/16 range but I can't find any right now
Nice to see this in the works! I initially had this issue on Kubernetes as well. I ended up using dnsConfig options with DOT disabled. I pointed to an in-cluster pi-hole with DoT enabled to solve the need here.
Let me know if you could use another set of eyes for testing.
This is the same as #281 I believe. I'm working on a sort of UDP / DNS proxy to redirect DNS requests to Unbound (DNS over TLS) or the native DNS (from the Docker bridge or K8s) depending on the request. If it has no dot it's sent to the native DNS and otherwise sent to Unbound, such that Unbound block lists can't go through using the native Docker DNS.
Not sure that will solve it for K8s as this one might use dots in its private addresses, but I'll see what I can do. Doing a simple IP check on the result to check it's private could do the trick. Anyway, that'll take several days to finish, but I'll keep you updated. Thanks for your patience!