unifios-utilities
unifios-utilities copied to clipboard
unifi-utilities
RTNETLINK answers: Operation not supported when trying to create network for podman
Describe the bug A clear and concise description of what the bug is.
To Reproduce Steps to reproduce the behavior: just running the procedure to either create a pihole (https://github.com/unifi-utilities/unifios-utilities/tree/main/run-pihole) or adguard (https://github.com/unifi-utilities/unifios-utilities/tree/main/AdguardHome) on the UDR 2.5.11 and Controller 7.2.91. here is what I get when I try to run
root@BRA:/mnt/data/on_boot.d# ./10-dns.sh RTNETLINK answers: Operation not supported Cannot find device "br5.mac" Cannot find device "br5.mac" Cannot find device "br5.mac" Cannot find device "br5.mac" logger: unknown priority name: ERROR Container pihole not found, make sure you set the proper name, you can ignore this error if it is your first time setting it up
when I run netstat -r I can't find the vlan br5.mac, but I can the vlan br5 (this was defined in the console)
and as a consequence when I ran podman I'm getting this error related to network
podman run -d --network dns --restart always
--name adguardhome
-v "/mnt/data/AdguardHome-Confdir/:/opt/adguardhome/conf/"
-v "/mnt/data/AdguardHome-Workdir/:/opt/adguardhome/work/"
--dns=127.0.0.1 --dns=1.1.1.1
--hostname adguardhome
adguard/adguardhome:latest
ERRO[0002] error loading cached network config: network "dns" not found in CNI cache
WARN[0002] falling back to loading from existing plugins on disk
Error: error configuring network namespace for container a2f419f051758819ae06db309d7d2a48cfee73626115fa24d45b25a7f9b80282: error adding pod adguardhome_adguardhome to CNI network "dns": failed to create macvlan: operation not supported
also when I try to see the networks, here is what I get
oot@BRA:/mnt/data/on_boot.d# podman network ls NETWORK ID NAME VERSION PLUGINS dd75a9d6fb30 dns 0.4.0 macvlan dd75a9d6fb30 dns 0.4.0 macvlan
thank you so much for all the help!
also when i try to run the command manually, this is what i get;
root@BRA:/mnt/data/on_boot.d# ip link add "br5.mac" link "br5" type macvlan mode bridge RTNETLINK answers: Operation not supported
I got this error before I setup the VLAN 5 network in the settings of the Network app.
You should proceed to login, go to the Network Application>Settings>Networks and create a new network VLAN 5. After saving and applying, try rerunning that command again and it should work as expected.
You can try to remove those networks, podman network rm dns then rerun the script 10-dns.sh to create it.
Hi @JoeOIVOV unfortunately it didn't solve the problem. I'm still getting the error:
root@BRA:/mnt/data/on_boot.d# ./10-dns.sh RTNETLINK answers: Operation not supported Cannot find device "br5.mac" Cannot find device "br5.mac" Cannot find device "br5.mac" Cannot find device "br5.mac" logger: unknown priority name: ERROR Container pihole not found, make sure you set the proper name, you can ignore this error if it is your first time setting it up
any toughts? thanks a lot!
then when I try to create the container for AdGuard, this is what i get:
RRO[0003] error loading cached network config: network "dns" not found in CNI cache WARN[0003] falling back to loading from existing plugins on disk Error: error configuring network namespace for container 7d1da22a9d08aa7bb4448c6a4e558b2d2c674717888924c3c32aafa0c1187695: error adding pod adguardhome_adguardhome to CNI network "dns": failed to create macvlan: operation not supported
This error translates to:
The script isn't finding the conflist file in /mnt/data/podman/cni - for example: 20-dns.conflist, that specifies this 'dns' network- the settings that you set in the Unifi WebGUI, Network application in the previous step.
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// Note: IP and VLAN settings for you pihole network, 20-dns-conflist and 10-dns.sh MUST all match each other.
Example settings for pihole network: Network Name: Pihole Host address: 10.0.5.1 Netmask: 24 VLAN ID: 5 Network Type: Standard Multicast DNS: Enable DHCP: None Ipv6 Interface Type: None
YOU WILL NEED TO CHANGE 20-dns.conflist Change the line: "mac": "add 3 fake hex portions, replacing x's here 00:1c:b4:xx:xx:xx", to create a legitimate mac address that matches some vendor space(first 6 digits ). It needs to be unique on your network. The example gives one option. Locally administered mac addresses do not work.
If you are using a different IP address than the example: Change these lines to match your settings: "address": "10.0.5.3/24", "gateway": "10.0.5.1"
If you are using a different VLAN than the example: Change this line to match your VLAN number: "master": "br5", ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
Once you have that file in that location, rerun that script and you shouldn't get that error.
This error doesn't matter: logger: unknown priority name: ERROR Container pihole not found, make sure you set the proper name, you can ignore this error if it is your first time setting it up
thanks a lot @JoeOIVOV . the issue is that I'm still getting the one bellow:
RTNETLINK answers: Operation not supported Cannot find device "br5.mac" Cannot find device "br5.mac" Cannot find device "br5.mac" Cannot find device "br5.mac"
any ideas? thanks a lot for your help and patience!
If you get this error please confirm that in that 20-dns.conflist that you specified 'br5' an the next step confirm in the 10-dns.sh script, that you've also specified VLAN=5 there an then lastly in the webgui. All 3 of these networks need to match for that error to go away. Please confirm that.
The instructions under the 'Run Pihole' directory work perfectly. Try that first, then replicate that experience for AdGaurd.
hi @JoeOIVOV thanks a lot for all your support! unfortunately I'm still getting errors. I believe I did the same procedure on my UDM PRO SE and it worked perfect! but since I'm trying to do the same on the UDR (that supposed to run the same UnifiOS) I can't get the same results.
I will add here the files just in case if you see what I'm doing wrong. this is my nwtwork info
Ip that I want to assign: 192.168.2.2 GW: 192.168.2.1 VLAN:5
10-dns.sh.txt 20-dns.conflist.txt dns.conflist.txt [05-install-cni-plugins.sh.txt](https://github.com/unifi-utilities/un pihole container command.txt ifios-utilities/files/9330337/05-install-cni-plugins.sh.txt)
thank you so much one more time!
by the way this is the output when I try to create the container:
root@Dream-Router:/mnt/data/on_boot.d# podman run -d --network dns --restart always --name pihole -e TZ="America/Chicago" --cap-add=NET_ADMIN -v "/mnt/data/etc-pihole/:/etc/pihole/" -v "/mnt/data/pihole/etc-dnsmasq.d/:/etc/dnsmasq.d/" --dns=127.0.0.1 --dns=1.1.1.1 --dns=8.8.8.8 --hostname pi.hole -e VIRTUAL_HOST="pi.hole" -e PROXY_LOCATION="pi.hole" -e FTLCONF_REPLY_ADDR4="192.168.2.2" -e IPv6="False" \pihole/pihole:latest ERRO[0007] error loading cached network config: network "dns" not found in CNI cache WARN[0007] falling back to loading from existing plugins on disk Error: error configuring network namespace for container 5af66c5eb92c11319df0c3fc8c4168d202624061d19eaa05ab91f1fcbdb03c16: error adding pod pihole_pihole to CNI network "dns": failed to create macvlan: operation not supported
and this is what I have as networks inside podman:
root@Dream-Router:/mnt/data/on_boot.d# podman network ls NETWORK ID NAME VERSION PLUGINS 2f259bab93aa podman 0.4.0 bridge,portmap,firewall,tuning dd75a9d6fb30 dns 0.4.0 macvlan
and this is the result of netstat:
root@Dream-Router:/mnt/data/on_boot.d# netstat -r Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 10.3.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth4 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br0 192.168.2.0 0.0.0.0 255.255.255.0 U 0 0 0 br5
I'm experiencing a similar issue on my UXG-Pro (running v1.13.4). I had a NextDNS container working fine, but got a little too ambitious and ended up needing to perform a hard reset, and now attempts to run 10-dns.sh result in the following output:
# /mnt/data/on_boot.d/10-dns.sh
Cannot find device "br3"
Cannot find device "br3"
Cannot find device "br3.mac"
Cannot find device "br3.mac"
Cannot find device "br3.mac"
Cannot find device "br3.mac"
ERRO[0000] error loading cached network config: network "dns" not found in CNI cache
WARN[0000] falling back to loading from existing plugins on disk
ERRO[0000] Error tearing down partially created network namespace for container eedb5f1c08c48bfabc2825148567f7d9ce77229f7c23f09fa48960172218ff80: error removing pod nextdns_nextdns from CNI network "dns": Link not found
Error: unable to start container "eedb5f1c08c48bfabc2825148567f7d9ce77229f7c23f09fa48960172218ff80": error configuring network namespace for container eedb5f1c08c48bfabc2825148567f7d9ce77229f7c23f09fa48960172218ff80: error adding pod nextdns_nextdns to CNI network "dns": Link not found
Contents of /mnt/data/podman/cni/20-dns.conflist:
{
"cniVersion": "0.4.0",
"name": "dns",
"plugins": [
{
"type": "macvlan",
"mode": "bridge",
"master": "br3",
"mac": "00:19:78:9d:2b:c0",
"ipam": {
"type": "static",
"addresses": [
{
"address": "10.0.3.3/24",
"gateway": "10.0.3.1"
}
],
"routes": [
{"dst": "0.0.0.0/0"}
]
}
}
]
}
Contents of /mnt/data/on_boot.d/10-dns.sh:
#!/bin/sh
## configuration variables:
VLAN=3
IPV4_IP="10.0.3.3"
# This is the IP address of the container. You may want to set it to match
# your own network structure such as 192.168.5.3 or similar.
IPV4_GW="10.0.3.1/24"
# As above, this should match the gateway of the VLAN for the container
# network as above which is usually the .1/24 range of the IPV4_IP
# if you want IPv6 support, generate a ULA, select an IP for the dns server
# and an appropriate gateway address on the same /64 network. Make sure that
# the 20-dns.conflist is updated appropriately. It will need the IP and GW
# added along with a ::/0 route. Also make sure that additional --dns options
# are passed to podman with your IPv6 DNS IPs when deploying the container for
# the first time. You will also need to configure your VLAN to have a static
# IPv6 block.
# IPv6 Also works with Prefix Delegation from your provider. The gateway is the
# IP of br(VLAN) and you can pick any ip address within that subnet that dhcpv6
# isn't serving
IPV6_IP=""
IPV6_GW=""
# set this to the interface(s) on which you want DNS TCP/UDP port 53 traffic
# re-routed through the DNS container. separate interfaces with spaces.
# e.g. "br0" or "br0 br1" etc.
FORCED_INTFC=""
# container name; e.g. nextdns, pihole, adguardhome, etc.
CONTAINER=nextdns
if ! test -f /opt/cni/bin/macvlan; then
echo "Error: CNI plugins not found. You can install it with the following command:" >&2
echo " curl -fsSLo /mnt/data/on_boot.d/05-install-cni-plugins.sh https://raw.githubusercontent.com/unifi-utilities/unifios-utilities/main/cni-plugins/05-install-cni-plugins.sh && /bin/sh /mnt/data/on_boot.d/05-install-cni-plugins.sh" >&2
exit 1
fi
# set VLAN bridge promiscuous
ip link set "br${VLAN}" promisc on
# create macvlan bridge and add IPv4 IP
ip link add "br${VLAN}.mac" link "br${VLAN}" type macvlan mode bridge
ip addr add "${IPV4_GW}" dev "br${VLAN}.mac" noprefixroute
# (optional) add IPv6 IP to VLAN bridge macvlan bridge
if [ -n "${IPV6_GW}" ]; then
ip -6 addr add "${IPV6_GW}" dev "br${VLAN}.mac" noprefixroute
fi
# set macvlan bridge promiscuous and bring it up
ip link set "br${VLAN}.mac" promisc on
ip link set "br${VLAN}.mac" up
# add IPv4 route to DNS container
ip route add "${IPV4_IP}/32" dev "br${VLAN}.mac"
# (optional) add IPv6 route to DNS container
if [ -n "${IPV6_IP}" ]; then
ip -6 route add "${IPV6_IP}/128" dev "br${VLAN}.mac"
fi
# Make DNSMasq listen to the container network for split horizon or conditional forwarding
if ! grep -qxF "interface=br${VLAN}.mac" /run/dnsmasq.conf.d/custom.conf; then
echo "interface=br${VLAN}.mac" >> /run/dnsmasq.conf.d/custom.conf
kill -9 "$(cat /run/dnsmasq.pid)"
fi
if podman container exists "${CONTAINER}"; then
podman start "${CONTAINER}"
else
logger -s -t podman-dns -p "ERROR Container ${CONTAINER} not found, make sure you set the proper name, you can ignore this error if it is your first time setting it up"
fi
# (optional) IPv4 force DNS (TCP/UDP 53) through DNS container
for intfc in ${FORCED_INTFC}; do
if [ -d "/sys/class/net/${intfc}" ]; then
for proto in udp tcp; do
prerouting_rule="PREROUTING -i ${intfc} -p ${proto} ! -s ${IPV4_IP} ! -d ${IPV4_IP} --dport 53 -j LOG --log-prefix [DNAT-${intfc}-${proto}]"
iptables -t nat -C ${prerouting_rule} 2>/dev/null || iptables -t nat -A ${prerouting_rule}
prerouting_rule="PREROUTING -i ${intfc} -p ${proto} ! -s ${IPV4_IP} ! -d ${IPV4_IP} --dport 53 -j DNAT --to ${IPV4_IP}"
iptables -t nat -C ${prerouting_rule} 2>/dev/null || iptables -t nat -A ${prerouting_rule}
# (optional) IPv6 force DNS (TCP/UDP 53) through DNS container
if [ -n "${IPV6_IP}" ]; then
prerouting_rule="PREROUTING -i ${intfc} -p ${proto} ! -s ${IPV6_IP} ! -d ${IPV6_IP} --dport 53 -j LOG --log-prefix [DNAT-${intfc}-${proto}]"
ip6tables -t nat -C ${prerouting_rule} 2>/dev/null || ip6tables -t nat -A ${prerouting_rule}
prerouting_rule="PREROUTING -i ${intfc} -p ${proto} ! -s ${IPV6_IP} ! -d ${IPV6_IP} --dport 53 -j DNAT --to ${IPV6_IP}"
ip6tables -t nat -C ${prerouting_rule} 2>/dev/null || ip6tables -t nat -A ${prerouting_rule}
fi
done
fi
done
Which matches the output from podman network inspect dns:
Output of ifconfig after a reboot:
br0 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C3
inet addr:192.168.1.1 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::7054:e7ff:fec4:a41/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:297248 errors:0 dropped:61 overruns:0 frame:0
TX packets:135445 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:514694552 (490.8 MiB) TX bytes:18486803 (17.6 MiB)
eth0 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C2
inet addr:[PUBLIC IP] Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::f692:bfff:fe8d:17c2/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:322215 errors:0 dropped:0 overruns:0 frame:0
TX packets:184222 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:412410326 (393.3 MiB) TX bytes:64220654 (61.2 MiB)
eth3 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C5
inet6 addr: fe80::f692:bfff:fe8d:17c5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:834751 errors:0 dropped:0 overruns:0 frame:1
TX packets:970538 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:10000
RX bytes:594859559 (567.3 MiB) TX bytes:950665416 (906.6 MiB)
eth3.2 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C5
inet addr:192.168.2.1 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::f692:bfff:fe8d:17c5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:71225 errors:0 dropped:0 overruns:0 frame:0
TX packets:28759 errors:0 dropped:19 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:53982184 (51.4 MiB) TX bytes:14849225 (14.1 MiB)
eth3.3 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C5
inet addr:10.0.3.1 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::f692:bfff:fe8d:17c5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:68 errors:0 dropped:0 overruns:0 frame:0
TX packets:3675 errors:0 dropped:22 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3808 (3.7 KiB) TX bytes:787082 (768.6 KiB)
eth3.4 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C5
inet addr:192.168.4.1 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::f692:bfff:fe8d:17c5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:185617 errors:0 dropped:0 overruns:0 frame:0
TX packets:441438 errors:0 dropped:6 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:13373473 (12.7 MiB) TX bytes:792392557 (755.6 MiB)
eth3.6 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C5
inet addr:192.168.6.1 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::f692:bfff:fe8d:17c5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:51607 errors:0 dropped:0 overruns:0 frame:0
TX packets:70541 errors:0 dropped:19 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3568996 (3.4 MiB) TX bytes:120992678 (115.3 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:626299 errors:0 dropped:0 overruns:0 frame:0
TX packets:626299 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:565330904 (539.1 MiB) TX bytes:565330904 (539.1 MiB)
Contents of /mnt/data/on_boot.d/:
Output from attempting to create and run the nextdns container:
Contents of /etc/cni/net.d:
Output of podman info
# podman info
host:
arch: arm64
buildahVersion: 1.22.3
cgroupControllers:
- cpu
- io
- memory
- pids
cgroupManager: cgroupfs
cgroupVersion: v2
conmon:
package: Unknown
path: /usr/libexec/podman/conmon
version: 'conmon version 2.0.29, commit: '
cpus: 4
distribution:
distribution: ubios
version: v1.13.4.4571-97e0133
eventLogger: file
hostname: UXG-Pro
idMappings:
gidmap: null
uidmap: null
kernel: 4.19.152-al-linux-v10.2.0-v1.13.4.4571-97e0133
linkmode: dynamic
memFree: 1259573248
memTotal: 2096644096
ociRuntime:
name: runc
package: Unknown
path: /usr/bin/runc
version: |-
runc version 1.0.2
commit: v1.0.2-0-g52b36a2d-dirty
spec: 1.0.2-dev
go: go1.16.7
libseccomp: 2.3.1
os: linux
remoteSocket:
path: /run/podman/podman.sock
security:
apparmorEnabled: false
capabilities: CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_MKNOD,CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: false
seccompEnabled: true
seccompProfilePath: /etc/containers/seccomp.json
selinuxEnabled: false
serviceIsRemote: false
slirp4netns:
executable: ""
package: ""
version: ""
swapFree: 3668598784
swapTotal: 3669123072
uptime: 40m 24.73s
registries:
search:
- docker.io
- quay.io
- registry.fedoraproject.org
store:
configFile: /etc/containers/storage.conf
containerStore:
number: 2
paused: 0
running: 1
stopped: 1
graphDriverName: overlay
graphOptions:
overlay.imagestore: /var/lib/containers/storage
overlay.mountopt: nodev
overlay.skip_mount_home: "false"
graphRoot: /mnt/data/podman/storage
graphStatus:
Backing Filesystem: extfs
Native Overlay Diff: "true"
Supports d_type: "true"
Using metacopy: "false"
imageStore:
number: 4
runRoot: /var/run/containers/storage
volumePath: /mnt/data/podman/storage/volumes
version:
APIVersion: 3.3.0
Built: 1629960926
BuiltTime: Thu Aug 26 02:55:26 2021
GitCommit: 98f252a3a1a8f1ee00f9f96c6ba00500954b5093-dirty
GoVersion: go1.16.4
OsArch: linux/arm64
Version: 3.3.0
Let me know if there's anything else I can provide to help, or if you think my issue is different.
Everything looks correct...
From: Justin Rusbatch @.> Sent: Wednesday, August 31, 2022 2:59:19 PM To: unifi-utilities/unifios-utilities @.> Cc: JoeOIVOV @.>; Mention @.> Subject: Re: [unifi-utilities/unifios-utilities] RTNETLINK answers: Operation not supported when trying to create network for podman (Issue #399)
I'm also experiencing this issue on my UXG-Pro. I had a NextDNS container working fine, but got a little too ambitious and ended up needing to perform a hard reset.
Contents of /mnt/data/podman/cni/20-dns.conflist:
{ "cniVersion": "0.4.0", "name": "dns", "plugins": [ { "type": "macvlan", "mode": "bridge", "master": "br3", "mac": "00:19:78:9d:2b:c0", "ipam": { "type": "static", "addresses": [ { "address": "10.0.3.3/24", "gateway": "10.0.3.1" } ], "routes": [ {"dst": "0.0.0.0/0"} ] } } ] }
Contents of /mnt/data/on_boot.d/10-dns.sh:
#!/bin/sh
configuration variables:
VLAN=3 IPV4_IP="10.0.3.3"
This is the IP address of the container. You may want to set it to match
your own network structure such as 192.168.5.3 or similar.
IPV4_GW="10.0.3.1/24"
As above, this should match the gateway of the VLAN for the container
network as above which is usually the .1/24 range of the IPV4_IP
if you want IPv6 support, generate a ULA, select an IP for the dns server
and an appropriate gateway address on the same /64 network. Make sure that
the 20-dns.conflist is updated appropriately. It will need the IP and GW
added along with a ::/0 route. Also make sure that additional --dns options
are passed to podman with your IPv6 DNS IPs when deploying the container for
the first time. You will also need to configure your VLAN to have a static
IPv6 block.
IPv6 Also works with Prefix Delegation from your provider. The gateway is the
IP of br(VLAN) and you can pick any ip address within that subnet that dhcpv6
isn't serving
IPV6_IP="" IPV6_GW=""
set this to the interface(s) on which you want DNS TCP/UDP port 53 traffic
re-routed through the DNS container. separate interfaces with spaces.
e.g. "br0" or "br0 br1" etc.
FORCED_INTFC=""
container name; e.g. nextdns, pihole, adguardhome, etc.
CONTAINER=nextdns
if ! test -f /opt/cni/bin/macvlan; then echo "Error: CNI plugins not found. You can install it with the following command:" >&2 echo " curl -fsSLo /mnt/data/on_boot.d/05-install-cni-plugins.sh https://raw.githubusercontent.com/unifi-utilities/unifios-utilities/main/cni-plugins/05-install-cni-plugins.sh && /bin/sh /mnt/data/on_boot.d/05-install-cni-plugins.sh" >&2 exit 1 fi
set VLAN bridge promiscuous
ip link set "br${VLAN}" promisc on
create macvlan bridge and add IPv4 IP
ip link add "br${VLAN}.mac" link "br${VLAN}" type macvlan mode bridge ip addr add "${IPV4_GW}" dev "br${VLAN}.mac" noprefixroute
(optional) add IPv6 IP to VLAN bridge macvlan bridge
if [ -n "${IPV6_GW}" ]; then ip -6 addr add "${IPV6_GW}" dev "br${VLAN}.mac" noprefixroute fi
set macvlan bridge promiscuous and bring it up
ip link set "br${VLAN}.mac" promisc on ip link set "br${VLAN}.mac" up
add IPv4 route to DNS container
ip route add "${IPV4_IP}/32" dev "br${VLAN}.mac"
(optional) add IPv6 route to DNS container
if [ -n "${IPV6_IP}" ]; then ip -6 route add "${IPV6_IP}/128" dev "br${VLAN}.mac" fi
Make DNSMasq listen to the container network for split horizon or conditional forwarding
if ! grep -qxF "interface=br${VLAN}.mac" /run/dnsmasq.conf.d/custom.conf; then echo "interface=br${VLAN}.mac" >> /run/dnsmasq.conf.d/custom.conf kill -9 "$(cat /run/dnsmasq.pid)" fi
if podman container exists "${CONTAINER}"; then podman start "${CONTAINER}" else logger -s -t podman-dns -p "ERROR Container ${CONTAINER} not found, make sure you set the proper name, you can ignore this error if it is your first time setting it up" fi
(optional) IPv4 force DNS (TCP/UDP 53) through DNS container
for intfc in ${FORCED_INTFC}; do if [ -d "/sys/class/net/${intfc}" ]; then for proto in udp tcp; do prerouting_rule="PREROUTING -i ${intfc} -p ${proto} ! -s ${IPV4_IP} ! -d ${IPV4_IP} --dport 53 -j LOG --log-prefix [DNAT-${intfc}-${proto}]" iptables -t nat -C ${prerouting_rule} 2>/dev/null || iptables -t nat -A ${prerouting_rule} prerouting_rule="PREROUTING -i ${intfc} -p ${proto} ! -s ${IPV4_IP} ! -d ${IPV4_IP} --dport 53 -j DNAT --to ${IPV4_IP}" iptables -t nat -C ${prerouting_rule} 2>/dev/null || iptables -t nat -A ${prerouting_rule}
# (optional) IPv6 force DNS (TCP/UDP 53) through DNS container
if [ -n "${IPV6_IP}" ]; then
prerouting_rule="PREROUTING -i ${intfc} -p ${proto} ! -s ${IPV6_IP} ! -d ${IPV6_IP} --dport 53 -j LOG --log-prefix [DNAT-${intfc}-${proto}]"
ip6tables -t nat -C ${prerouting_rule} 2>/dev/null || ip6tables -t nat -A ${prerouting_rule}
prerouting_rule="PREROUTING -i ${intfc} -p ${proto} ! -s ${IPV6_IP} ! -d ${IPV6_IP} --dport 53 -j DNAT --to ${IPV6_IP}"
ip6tables -t nat -C ${prerouting_rule} 2>/dev/null || ip6tables -t nat -A ${prerouting_rule}
fi
done
fi done
Which matches the output from podman network inspect dns:
[image]https://user-images.githubusercontent.com/423549/187790777-b49205d4-37c1-4b63-844d-8f99bb9c9c0a.png
Output of ifconfig after a reboot:
br0 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C3 inet addr:192.168.1.1 Bcast:0.0.0.0 Mask:255.255.255.0 inet6 addr: fe80::7054:e7ff:fec4:a41/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:297248 errors:0 dropped:61 overruns:0 frame:0 TX packets:135445 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:514694552 (490.8 MiB) TX bytes:18486803 (17.6 MiB)
eth0 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C2 inet addr:[PUBLIC IP] Bcast:0.0.0.0 Mask:255.255.255.0 inet6 addr: fe80::f692:bfff:fe8d:17c2/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:322215 errors:0 dropped:0 overruns:0 frame:0 TX packets:184222 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:412410326 (393.3 MiB) TX bytes:64220654 (61.2 MiB)
eth3 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C5 inet6 addr: fe80::f692:bfff:fe8d:17c5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:834751 errors:0 dropped:0 overruns:0 frame:1 TX packets:970538 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:10000 RX bytes:594859559 (567.3 MiB) TX bytes:950665416 (906.6 MiB)
eth3.2 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C5 inet addr:192.168.2.1 Bcast:0.0.0.0 Mask:255.255.255.0 inet6 addr: fe80::f692:bfff:fe8d:17c5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:71225 errors:0 dropped:0 overruns:0 frame:0 TX packets:28759 errors:0 dropped:19 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:53982184 (51.4 MiB) TX bytes:14849225 (14.1 MiB)
eth3.3 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C5 inet addr:10.0.3.1 Bcast:0.0.0.0 Mask:255.255.255.0 inet6 addr: fe80::f692:bfff:fe8d:17c5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:68 errors:0 dropped:0 overruns:0 frame:0 TX packets:3675 errors:0 dropped:22 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:3808 (3.7 KiB) TX bytes:787082 (768.6 KiB)
eth3.4 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C5 inet addr:192.168.4.1 Bcast:0.0.0.0 Mask:255.255.255.0 inet6 addr: fe80::f692:bfff:fe8d:17c5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:185617 errors:0 dropped:0 overruns:0 frame:0 TX packets:441438 errors:0 dropped:6 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:13373473 (12.7 MiB) TX bytes:792392557 (755.6 MiB)
eth3.6 Link encap:Ethernet HWaddr F4:92:BF:8D:17:C5 inet addr:192.168.6.1 Bcast:0.0.0.0 Mask:255.255.255.0 inet6 addr: fe80::f692:bfff:fe8d:17c5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:51607 errors:0 dropped:0 overruns:0 frame:0 TX packets:70541 errors:0 dropped:19 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:3568996 (3.4 MiB) TX bytes:120992678 (115.3 MiB)
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:626299 errors:0 dropped:0 overruns:0 frame:0 TX packets:626299 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:565330904 (539.1 MiB) TX bytes:565330904 (539.1 MiB)
Contents of /mnt/data/on_boot.d/:
[image]https://user-images.githubusercontent.com/423549/187791538-fd631557-5a28-410e-8250-14ff867cb518.png
Output from attempting to create and run the nextdns container:
[image]https://user-images.githubusercontent.com/423549/187791945-1d4e3517-296a-4ed7-94cf-d6848da4b243.png
Contents of /etc/cni/net.d:
[image]https://user-images.githubusercontent.com/423549/187792115-31f84f24-1063-423d-bbb0-a66874b88f19.png
Output of podman info
podman info
host: arch: arm64 buildahVersion: 1.22.3 cgroupControllers:
- cpu
- io
- memory
- pids cgroupManager: cgroupfs cgroupVersion: v2 conmon: package: Unknown path: /usr/libexec/podman/conmon version: 'conmon version 2.0.29, commit: ' cpus: 4 distribution: distribution: ubios version: v1.13.4.4571-97e0133 eventLogger: file hostname: UXG-Pro idMappings: gidmap: null uidmap: null kernel: 4.19.152-al-linux-v10.2.0-v1.13.4.4571-97e0133 linkmode: dynamic memFree: 1259573248 memTotal: 2096644096 ociRuntime: name: runc package: Unknown path: /usr/bin/runc version: |- runc version 1.0.2 commit: v1.0.2-0-g52b36a2d-dirty spec: 1.0.2-dev go: go1.16.7 libseccomp: 2.3.1 os: linux remoteSocket: path: /run/podman/podman.sock security: apparmorEnabled: false capabilities: CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_MKNOD,CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT rootless: false seccompEnabled: true seccompProfilePath: /etc/containers/seccomp.json selinuxEnabled: false serviceIsRemote: false slirp4netns: executable: "" package: "" version: "" swapFree: 3668598784 swapTotal: 3669123072 uptime: 40m 24.73s registries: search:
- docker.io
- quay.io
- registry.fedoraproject.org store: configFile: /etc/containers/storage.conf containerStore: number: 2 paused: 0 running: 1 stopped: 1 graphDriverName: overlay graphOptions: overlay.imagestore: /var/lib/containers/storage overlay.mountopt: nodev overlay.skip_mount_home: "false" graphRoot: /mnt/data/podman/storage graphStatus: Backing Filesystem: extfs Native Overlay Diff: "true" Supports d_type: "true" Using metacopy: "false" imageStore: number: 4 runRoot: /var/run/containers/storage volumePath: /mnt/data/podman/storage/volumes version: APIVersion: 3.3.0 Built: 1629960926 BuiltTime: Thu Aug 26 02:55:26 2021 GitCommit: 98f252a3a1a8f1ee00f9f96c6ba00500954b5093-dirty GoVersion: go1.16.4 OsArch: linux/arm64 Version: 3.3.0
Let me know if there's anything else I can provide to help, or if you think my issue is different.
— Reply to this email directly, view it on GitHubhttps://github.com/unifi-utilities/unifios-utilities/issues/399#issuecomment-1233461568, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD3JZSC4UPSBHOSWZD65Z5DV37ITPANCNFSM55ZHIWAQ. You are receiving this because you were mentioned.Message ID: @.***>
Have you tried temporarily removing the 01-podman-update.sh script to see if that helps?
Also, what do you have in the Unifi WebGUI, Network App>Settings>Networks. I notice my ifconfig returns br5.mac which is my vlan for the dns network. I don't see yours in the above output. I'd image you should have a br3.mac in that output if its a vlan3 network created in the network app.
and this is the result of netstat:
root@Dream-Router:/mnt/data/on_boot.d# netstat -r Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 10.3.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth4 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br0 192.168.2.0 0.0.0.0 255.255.255.0 U 0 0 0 br5
seeing a similar issue on this reply, no br5.mac
netstat -r
10.0.0.0 * 255.255.255.0 U 0 0 0 br0
10.0.5.0 * 255.255.255.0 U 0 0 0 br5
10.0.5.2 * 255.255.255.255 UH 0 0 0 br5.mac
xx.xx.xx.xx * 255.255.255.248 U 0 0 0 br10
xx.xx.xx.xx * 255.255.252.0 U 0 0 0 eth8
This is what I'm using:
cat /mnt/data/podman/cni/20-piholeipv6.conflist
{
"cniVersion": "0.4.0",
"name": "dns",
"plugins": [
{
"type": "macvlan",
"mode": "bridge",
"master": "br5",
"mac": "b2:a5:dc:c7:a7:e3",
"ipam": {
"type": "static",
"addresses": [
{
"address": "10.0.5.2/24",
"gateway": "10.0.5.1"
},
{
"address": "fdca:5c13:1fb8::2/64",
"gateway": "fdca:5c13:1fb8::1"
}
],
"routes": [
{"dst": "0.0.0.0/0"},{"dst": "::/0"}
]
}
}
]
}
Have you tried temporarily removing the 01-podman-update.sh script to see if that helps?
Also, what do you have in the Unifi WebGUI, Network App>Settings>Networks. I notice my ifconfig returns br5.mac which is my vlan for the dns network. I don't see yours in the above output. I'd image you should have a br3.mac in that output if its a vlan3 network created in the network app.
I think so, but I've tried a few things at this point so I'm happy to try again. Here are the relevant log messages from the uxg-setup container after a reboot (the first line is the sigterm caused by the restart).
{"level":"info","message":"SIGTERM signal received.","timestamp":"2022-08-31T22:32:57.972Z"}
running /mnt/data/on_boot.d/05-container-common.sh
running /mnt/data/on_boot.d/05-install-cni-plugins.sh
curl: (6) Could not resolve: github.com (Could not contact DNS servers)
Pouring /mnt/data/.cache/cni-plugins/cni-plugins-linux-arm64-latest.tgz
running /mnt/data/on_boot.d/06-cni-bridge.sh
running /mnt/data/on_boot.d/10-dns.sh
Cannot find device "br3"
Cannot find device "br3"
Cannot find device "br3.mac"
Cannot find device "br3.mac"
Cannot find device "br3.mac"
Cannot find device "br3.mac"
grep: /run/dnsmasq.conf.d/custom.conf: No such file or directory
time="2022-08-31T18:33:13-04:00" level=error msg="Error adding network: Link not found"
time="2022-08-31T18:33:13-04:00" level=error msg="Error while adding pod to CNI network \"dns\": Link not found"
Error: unable to start container "nextdns": error configuring network namespace for container eedb5f1c08c48bfabc2825148567f7d9ce77229f7c23f09fa48960172218ff80: Link not found
running /mnt/data/on_boot.d/20-nextdns.sh
time="2022-08-31T18:33:13-04:00" level=error msg="Error adding network: Link not found"
time="2022-08-31T18:33:13-04:00" level=error msg="Error while adding pod to CNI network \"dns\": Link not found"
Error: unable to start container "nextdns": error configuring network namespace for container eedb5f1c08c48bfabc2825148567f7d9ce77229f7c23f09fa48960172218ff80: Link not found
{"level":"info","message":"Starting at https://localhost:443. Env=production","timestamp":"2022-08-31T22:33:17.207Z"}
(node:1) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `nodejs --trace-deprecation ...` to show where the warning was created)
#
The errors about being unable to resolve domains seem like something I should fix, but they also don't look like they were interfering with the scripts too much?
Also, what do you have in the Unifi WebGUI, Network App>Settings>Networks. I notice my ifconfig returns br5.mac which is my vlan for the dns network. I don't see yours in the above output.
This is something that I noticed was different from my first time setting this up. Before resetting it, when things were working, there was a br3 for my NextDNS VLAN. But now there isn't and I'm not sure why. I also vaguely remember having a /mnt/data/on_boot.sh script, but now I've forgotten what was in it or where I found it.
Below is a screenshot from my network config in the GUI.
And below is the configuration for the DNS network specifically:
Not liking that DNS says its WAN1, I think its suppose to be a LAN network. - maybe not possible on that device... which may not make a difference, but I see that as different, and strange its not creating the network.
What shows up when you run netstat -r
Not liking that DNS says its WAN1, I think its suppose to be a LAN network. - maybe not possible on that device... which may not make a difference, but I see that as different, and strange its not creating the network.
That gave me a scare, but after switching back to the legacy UI I see this:
What shows up when you run netstat -r
# netstat -r
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
10.0.3.0 * 255.255.255.0 U 0 0 0 eth3.3
[PUBLIC IP] * 255.255.255.0 U 0 0 0 eth0
192.168.1.0 * 255.255.255.0 U 0 0 0 br0
192.168.2.0 * 255.255.255.0 U 0 0 0 eth3.2
192.168.4.0 * 255.255.255.0 U 0 0 0 eth3.4
192.168.6.0 * 255.255.255.0 U 0 0 0 eth3.6
#
What if you delete the DNS network, recreate it and rerun that netstat -r. Your suppose to see one called br3.mac with the ip 10.0.3.3 but not seeing it.
# netstat -r
10.0.0.0 * 255.255.255.0 U 0 0 0 br0
10.0.5.0 * 255.255.255.0 U 0 0 0 br5
10.0.5.2 * 255.255.255.255 UH 0 0 0 br5.mac
[Public IP] * 255.255.255.248 U 0 0 0 br10
[Public IP] * 255.255.252.0 U 0 0 0 eth8
VLAN3 deleted:
Output of a netstat -r (after a reboot, for kicks):
# netstat -r
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
[PUBLIC IP] * 255.255.255.0 U 0 0 0 eth0
192.168.1.0 * 255.255.255.0 U 0 0 0 br0
192.168.2.0 * 255.255.255.0 U 0 0 0 eth3.2
192.168.4.0 * 255.255.255.0 U 0 0 0 eth3.4
192.168.6.0 * 255.255.255.0 U 0 0 0 eth3.6
#
eth3.3 goes away, but still no br3 or br3.mac.
After a few evenings of reading and trying random commands fed to me from all over the internet, I think I have two workarounds.
If, after creating the new VLAN in the GUI, I SSH into my UXG and run these commands (where 3 is my VLAN ID, and eth3.3 has my VLAN's gateway IP)...
brctl addbr br3
brctl addif br3 eth3.3
ip link set "br3" up
... then the 10-dns.sh script runs fine (after updating it to use the proper VLAN ID, of course).
Things also seem like they work I change this line in 20-dns.conflist to "master": "eth3.3",, and then replace all occurrences of br5 with eth3.3 in 10-dns.sh.
However, I'm a bit in over my head here. Its been a long time since my last networking class, and I don't usually need to worry about this stuff, so I've happily forgotten what I learned in the years since.
If either of these approaches are safe and acceptable, though, then it seems like it would be a good idea to add some conditional logic here in case br# doesn't exist.
Likewise, if neither of those workarounds are a good idea, please tell me ASAP so I can stop doing that.
Likewise, if neither of those workarounds are a good idea, please tell me ASAP so I can stop doing that.
I will try tomorrow! many thanks for keep looking for a solution! really appreciated!
Did a factory reset on the UDR, followed the podman install procedure. Same result. Wondering if there are other options on how to set this Adguard container other than the macvlan approach.
it didn't work for me... I do have the br5 my issue is really creating the br5.mac network