udp-proxy-2020
udp-proxy-2020 copied to clipboard
synfinatic
Need help for configuration Clients -> Router -> (wireguard) -> Remote RoonCore
Hi, I had running udp-proxy-2020 for ages with the following setup: RoonCore (remote server in a datacenter) -> Wireguard -> Client udp-proxy-2020 installed on the remote server with the following parameters: --interface eth0,wg0 --port 9003 --no-listen
Now, I've changed my configuration because when I'm at home I don't want to have to switch on wireguard on every single client, so with a OPNSense router I achieved the following configuration:
RoonCore (remote server) -> Wireguard -> Home OPNsense router -> Clients. With wireguard installed on the router.
With this setup I can access my Roon library but device audio endpoints are not discovered. Every client can ping the remote server without any issue.
So I tried to configure udp-proxy-2020 on the router too with the following parameters: --interface re0,wg1 --fixed-ip [email protected] --port 9003 where re0 is the lan interface, without any success. I can' t still see any audio endpoint.
Here the pcap files and the udp-proxy-2020 log on the router udp-proxy-2020.log pcap.zip
I'm not that good at networking so I might be missing something very stupid.
May I ask you for some help?
Thanks,
Simone
Are you running udp-proxy-2020 on both the remote server and home OPNSense router? That is a requirement for a site-to-site VPN with the Roon Discovery protocol.
Edit: I should say I've never personally tried this and so I can't say certain that it will work. My understanding of the Roon Discovery protocol is not perfect, but I believe it to be possible.
Are you running
udp-proxy-2020on both the remote server and home OPNSense router? That is a requirement for a site-to-site VPN with the Roon Discovery protocol.
Yes I am.
On the remote server with the following parameters: --interface eth0,wg0 --port 9003 --no-listen On my home router with the following parameters: --interface re0,wg1 --fixed-ip [email protected] --port 9003 (where 10.200.200.1 is IP address of the remote server and re0 is the local LAN interface)
can you capture traffic on the remote end as well?
Also please provide the output of udp-proxy-2020 -l from both systems.
Sure.
Interfaces on remote server:
Interface: eth0 - IP: 192.168.0.50/24 Broadaddr: 192.168.0.255 - IP: fe80::6829:bcff:fe26:7dcf/64
Interface: wg0 - IP: 10.200.200.1/24 PointToPoint: 10.200.200.1
Interface: lo - IP: 127.0.0.1/8 - IP: ::1/128
Interfaces on the router:
Interface: re0 - IP: 172.16.0.1/22 Broadaddr: 172.16.3.255
Interface: pppoe0 - IP: xxx.xxx.xxx.xxx/32 PointToPoint: xxx.xxx.xxx.xxx
Interface: wg1 - IP: 10.200.200.12/24
Interface: lo0 - IP: ::1/128 - IP: fe80::1/64 - IP: 127.0.0.1/8
pcap files captured on the remote server:
Hmmm... I think I see the problem. Gonna need to figure out how I can reproduce this locally and figure out a fix.
so I don't have OPNsense, but I do have PFSense which is similar-ish AFAIK. I'd expect the Wireguard implementation at least to be the same...
but it's not. Your Wireguard interfaces are point-to-point. And I believe that is the cause of the errors in the logs:
level=warning msg="Unable to send 362 bytes from re0 out wg1: send: Address family not supported by protocol family"
You're probably going to see a different set of flags than I do for my Wireguard interface:
$ ifconfig tun_wg0
tun_wg0: flags=81c1<UP,RUNNING,NOARP,PROMISC,MULTICAST> metric 0 mtu 1500
description: WG0
options=80000<LINKSTATE>
inet 172.16.12.1 netmask 0xffffff00
groups: wg
nd6 options=101<PERFORMNUD,NO_DAD>
Note that the PROMISC flag is because udp-proxy-2020 is running. I don't really have a way of testing your config to really know what is going on, but if I were you I'd see if I could figure out how to not run it in PointToPoint mode.
Mmm could you please be more clear about "Your Wireguard interfaces are point-to-point."? My overall wireguard network topology if more like a mesh network (I guess?), all traffic pass through 10.200.200.1, every client can see each other connected client. The router (10.200.200.12) is just one of the multiple clients.
In the specific 10.200.200.1 (Server) - 10.200.200.12 (router) "subsection" it acts as a site-to-multisite, where are the natted clients behind the router can talk to the 10.200.200.1 server.
In fact, the generated wg1.conf file from OPNSense is as follows:
[Interface]
PrivateKey = xxx
Address = 10.200.200.12/24
[Peer]
PublicKey = xxx
Endpoint = aaa.bbb.ccc.ddd:51194
AllowedIPs = 10.200.200.0/24
PersistentKeepalive = 30
With udp-proxy-2020 enabled my ifconfig is the same as yours. If by point-to-point you mean the interface flag it's not the case.
root@OPNsense:/var/log # ifconfig wg1
wg1: flags=81c1<UP,RUNNING,NOARP,PROMISC,MULTICAST> metric 0 mtu 1420
description: Hetzner (opt1)
options=80000<LINKSTATE>
inet 10.200.200.12 netmask 0xffffff00
groups: wg wireguard
nd6 options=109<PERFORMNUD,IFDISABLED,NO_DAD>
When I stop the udp-proxy-2020 server the interface loses the PROMISC flag like it's expected to.
the interface on the remote server:
Interface: wg0
- IP: 10.200.200.1/24 PointToPoint: 10.200.200.1
the interface on the remote server:
Interface: wg0 - IP: 10.200.200.1/24 PointToPoint: 10.200.200.1
Mmm I'm missing something..
The udp-proxy-2020 on my router, which has those logged error messages (Address family not supported by protocol family) and whose interface (wg1) does not have the pointopoint flag is not able route the broadcast packets on the tunnel because the remote interface (on the remote server with linux) has the wg0 interface with the pointopoint flag?
As far as I know, in linux, wireguard always creates the wg0 interface with the POINTOPOINT flag. Please correct me if I'm wrong.
So the log is saying:
level=warning msg="Unable to send 362 bytes from re0 out wg1: send: Address family not supported by protocol family"
But the pcap actually shows all the packets from re0 being sent on wg1 on your OPNSense box. Just not seeing any replies from your Linux server on the remote end over the VPN tunnel. The Address family not supported by protocol family message is coming from your *BSD kernel so not sure what is going on really.
If you use the --pcap flag on the remote/Linux box it would be interesting to see if the packets are arriving or not.
Also, on the remote/linux box can you run netstat -an | grep :9003 and report that?
Sure:
root@simone-roon:~# netstat -an | grep :9003
udp 0 0 0.0.0.0:9003 0.0.0.0:*
udp 0 0 0.0.0.0:9003 0.0.0.0:*
udp 0 0 0.0.0.0:9003 0.0.0.0:*
udp 0 0 0.0.0.0:9003 0.0.0.0:*
udp 0 0 0.0.0.0:9003 0.0.0.0:*
udp 0 0 0.0.0.0:9003 0.0.0.0:*
udp 0 0 0.0.0.0:9003 0.0.0.0:*
udp 0 0 0.0.0.0:9003 0.0.0.0:*
PCAP files from the remote linux server:
Traffic for 192.168.0.0/24 is egressing eth0 on your Linux box. Either you need to update your Wireguard AllowedIPs list or manually add a route to use wg0?
It looks fine to me. Eth0 (the 192.168.0.0 subnet) is the WAN interface. The whole wireguard-roon setup on the linux server is actually in an LXC container where the host OS port forward the wireguard 51194 udp traffic into the LXC container so that's reachable through the public static IP of the server. The 192.168.x.x is just an internal lan between all the LXC containers that are on the server. (Which acts as a wan for each container)
The traffic TO the linux server wireguard lxc THROUGH the vpn tunnel is then forwarded to the wan interface so that if a wireguard client has setup as allowed IP 0.0.0.0/0 it will still have internet connection through the tunnel.
This is the wg0.conf on the lxc container on the remote linux server (only the interface section, then there are all the peers one of which is my opnsense router 10.200.200.12)
[Interface]
Address = 10.200.200.1/24
SaveConfig = true
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o>
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD >
ListenPort = 51194
(Postup and postdown iptables rules are not complete since I'm from mobile and I cannot copy the whole line from the wg0.conf file)
oh, sorry... saw the private IP space and didn't bother to verify they match what is across the VPN. Doesn't seem like Roon is seeing the packets arriving on a software tunnel interface like wg0 and hence isn't replying.
So first, try restarting Roon on the Linux box. Maybe Roon needs to start after the tunnel is up??
If that doesn't work, on the linux side, can you add --fixed-ip [email protected] or try --fixed-ip [email protected]. Basically tell udp-proxy-2020 to deliver to a different interface.
Honestly, the more I think about this there is a problem I'm not sure how to resolve without you setting a 2nd LXC container and moving Roon to that (away from udp-proxy-2020 and wireguard).
Basically the way udp-proxy-2020 sends packets makes them invisible to the system which sends them. Only different hosts see the packets. There are ways of course to send packets to yourself, but then I can't spoof the correct source IP address of the Roon client. And without that, Roon will learn the wrong IP address and it won't be able to stream music to the device.
Honestly, the more I think about this there is a problem I'm not sure how to resolve without you setting a 2nd LXC container and moving Roon to that (away from udp-proxy-2020 and wireguard).
Basically the way
udp-proxy-2020sends packets makes them invisible to the system which sends them. Only different hosts see the packets. There are ways of course to send packets to yourself, but then I can't spoof the correct source IP address of the Roon client. And without that, Roon will learn the wrong IP address and it won't be able to stream music to the device.
This seems more than reasonable. Those two LXC will communicate throught the internal LAN (192.168.x.x). I'll keep you updated
FYI, I think I just figured how to inject UDP frames locally and forge the source IP... only it's not cross platform and has to be different on Linux vs. *BSD. So yeah, I think I can make this work, only it's a bit of work. No promises right now when I'll find time, but kinda motivated to make this work so maybe "soon"... at least relative to the heat death of the universe. :D
So I tried as you said:
LXC Container Wireguard+udp-proxy-2020
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.51 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::d049:1ff:feec:414f prefixlen 64 scopeid 0x20<link>
ether ce:83:d6:36:49:a3 txqueuelen 1000 (Ethernet)
RX packets 55984 bytes 75500165 (72.0 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 86611 bytes 79260743 (75.5 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
wg0: flags=209<UP,POINTOPOINT,RUNNING,NOARP> mtu 1420
inet 10.200.200.1 netmask 255.255.255.0 destination 10.200.200.1
unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 1000 (UNSPEC)
RX packets 29878 bytes 6236664 (5.9 MiB)
RX errors 26 dropped 0 overruns 0 frame 26
TX packets 57426 bytes 71058028 (67.7 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
udp-proxy-2020 config
ARGS=--interface eth0,wg0 --port 9003 --level debug --logfile /var/log/udp-proxy-2020.log
LXC Container with Roon
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.50 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::c83e:6bff:fee4:aa03 prefixlen 64 scopeid 0x20<link>
ether ce:83:d6:36:49:a3 txqueuelen 1000 (Ethernet)
RX packets 3262 bytes 2118418 (2.0 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 4550 bytes 1667271 (1.5 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 11520 bytes 829855 (810.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 11520 bytes 829855 (810.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Now let's leave alone for a moment the connection with my opnsense router, I tried connecting my smartphone with 4G network + Wireguard (which has always been working) -> Cannot even see the Roon server anymore
LXC Wireguard + udp-proxy-2020 log (partial) - Smartphone wireguard IP 10.200.200.2 :
level=debug msg="wg0: Learned client IP: 10.200.200.2"
level=debug msg="wg0: received packet and fowarding onto other interfaces"
level=debug msg="eth0: sending out because we're not wg0"
level=debug msg="wg0: Learned client IP: 10.200.200.2"
level=debug msg="wg0: received packet and fowarding onto other interfaces"
level=debug msg="eth0: sending out because we're not wg0"
level=debug msg="wg0: Learned client IP: 10.200.200.2"
level=debug msg="wg0: received packet and fowarding onto other interfaces"
level=debug msg="eth0: sending out because we're not wg0"
level=debug msg="wg0: Learned client IP: 10.200.200.2"
level=debug msg="wg0: received packet and fowarding onto other interfaces"
level=debug msg="eth0: sending out because we're not wg0"
level=debug msg="wg0: Learned client IP: 10.200.200.2"
level=debug msg="wg0: received packet and fowarding onto other interfaces"
level=debug msg="eth0: sending out because we're not wg0"
level=debug msg="wg0: Learned client IP: 10.200.200.2"
level=debug msg="wg0: received packet and fowarding onto other interfaces"
level=debug msg="eth0: sending out because we're not wg0"
level=debug msg="processing packet from wg0 on eth0"
level=debug msg="eth0 => 192.168.0.255: packet len: 140"
level=debug msg="processing packet from wg0 on eth0"
level=debug msg="eth0 => 192.168.0.255: packet len: 140"
level=debug msg="processing packet from wg0 on eth0"
level=debug msg="eth0 => 192.168.0.255: packet len: 140"
level=debug msg="processing packet from wg0 on eth0"
level=debug msg="eth0 => 192.168.0.255: packet len: 140"
level=debug msg="processing packet from wg0 on eth0"
level=debug msg="eth0 => 192.168.0.255: packet len: 140"
level=debug msg="processing packet from wg0 on eth0"
level=debug msg="eth0 => 192.168.0.255: packet len: 140"
Multicast packets tested with iperf between 192.168.0.50 and 192.168.0.51 (the two LXC containers) working without any problem.
Am I missing something very stupid?
I'd have to look at the pcaps. Also, it would be very helpful to get a pcap and output of ifconfig from the roon container.
tcpdump -i eth0 -s0 -w roon.pcap udp port 9003
edited to add -i eth0 to listen on a specific interface
Well I figured out a way to get packets delivered locally if you install Roon on the same server as udp-proxy-2020: udp-proxy-2020-0.1.0-linux-amd64.zip
you'll need to use the new --deliver-local flag.
That said, I'm not sure why Roon didn't see packets on the tunnel interface naturally? Roon should be listening on all interfaces (0.0.0.0:9003):
nestat -an | grep :9003
Hey here I am,
Pcap from the two LXC setup
Then I switch in a single LXC setup (like before) and used your new binary, without success:
root@simone-roon:/var/log# udp-proxy-2020 --interface eth0,wg0 --port 9003 --no-listen --level warn --deliver-local
FATAL can't serialize Eth header: (layers.Ethernet) {
BaseLayer: (layers.BaseLayer) {
Contents: ([]uint8) <nil>,
Payload: ([]uint8) <nil>
},
SrcMAC: (net.HardwareAddr) ,
DstMAC: (net.HardwareAddr) (len=6 cap=6) ff:ff:ff:ff:ff:ff,
EthernetType: (layers.EthernetType) IPv4,
Length: (uint16) 0
}
So the pcap-linux-server.zip seems to contain the pcaps generated by udp-proxy-2020 but not the pcap generated by tcpdump on the Roon server. But what I can see from the pcaps you provided, the Roon discovery packets are being broadcast on eth0 and in theory the Roon server should see them... but without the other pcap file, I don't know if the issue is that they're not being delivered (perhaps an issue with LXC??) or if the replies are being lost.
The Fatal error is interesting... can you run --level debug and try agin? I haven't seen that in my testing, but also hard to test every possible combination.
So the pcap-linux-server.zip seems to contain the pcaps generated by udp-proxy-2020 but not the pcap generated by tcpdump on the Roon server
You're right, my bad. Please have some patience, to test this I need to revert back to the 2 lxc configuration.
The Fatal error is interesting... can you run --level debug and try agin? I haven't seen that in my testing, but also hard to test every possible combination.
My bad X2. I didn't update the service file to use the new binary. Now Roon is finally reachable, although players connected to my router are not visible. If I directly connect a single device with Wireguard (ex. smartphone in 4g + wireguard) then it is seen as an output device. This is the pcap on the SINGLE lxc with roon+wireguard+udp-proxy-2020 with added the --deliver-local flag
No worries... this is complicated. You've been doing better than most honestly :)
So all the pcaps in the latest zip file (single LCX) are empty. Probably need to restart the Roon client(s) to re-initiate discovery?
As for the --level debug I'm not really sure why that Fatal error happened, but if it's not happening anymore then we can ignore it.
Anyways, there seems to be two different potential solutions:
- A single LXC and use the
--deliver-localflag - Two LXCs and move the Roon Core process to the other container. Don't use
--deliver-localflag.
I think we're both getting a little confused with this, so I'd suggest doing whichever option easiest for you. I don't really care and in theory both should work.
You're right it's better to focus on one single setup to better debug the problem and avoid confusion. I'll stick with the single LXC with wireguard+roonserver+udp-proxy-2020 on it
I've tried multiple times (reconnect clients, reboot the lxc and so on) but the pcap files with the new version remain empty.
This is the log file with the debug argument. As you can see udp-proxy-2020 learns the existence of my router.
The binary was started with the following arguments:
--interface eth0,wg0 --port 9003 --no-listen --deliver-local --logfile /var/log/udp-proxy-2020.log --level debug --pcap --pcap-path /tmp
This is the actual ifconfig of the container:
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.50 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::10ac:c7ff:fef3:85b2 prefixlen 64 scopeid 0x20<link>
ether ce:83:d6:36:49:a3 txqueuelen 1000 (Ethernet)
RX packets 3496 bytes 3835677 (3.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 3846 bytes 1559961 (1.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 2936 bytes 501534 (489.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2936 bytes 501534 (489.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
wg0: flags=209<UP,POINTOPOINT,RUNNING,NOARP> mtu 1420
inet 10.200.200.1 netmask 255.255.255.0 destination 10.200.200.1
unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 1000 (UNSPEC)
RX packets 1784 bytes 206368 (201.5 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1812 bytes 1183160 (1.1 MiB)
TX errors 189 dropped 0 overruns 0 carrier 0 collisions 0
As always I can reach the server with every client but: 1- If the client connect through the wireguard network of the router, no audio devices are detected 2- If the client is connected directly through wireguard (example: laptopt connected to the 4g hotspot of my smartphone and wireguard on the laptop) all the audio devices are discovered.
I think this binary should fix the pcap generation. udp-proxy-2020-0.1.0-linux-amd64.zip
That said, when you're doing the test from the home router, it would be useful to grab the pcaps from that host as well.
Here I am, sorry for the late reply.
The binary you linked still produces empty pcap files. Nonetherless here you are the log (debug level) from both the router and the LXC with included the pcap files from the router instance