netbirdio
netbird try (and can't) set /proc/sys/net/ipv4/conf/all/src_valid_mark
with the latest netbird and on home assistant with netbird addon i've got this error:
2024-07-19T14:57:37+02:00 ERRO client/internal/routemanager/systemops/systemops_linux.go:100: Error setting up sysctl: 1 errors occurred:
* write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system
2024-07-19T14:57:37+02:00 INFO client/internal/routemanager/manager.go:135: Routing setup complete
after i google it there are same problems with wireguard: https://forums.docker.com/t/sysctl-error-setting-key-net-ipv4-conf-all-src-valid-mark-read-only-file-system/92567/8 but even though this container has NET_ADMIN and NET_RAW privileges it's still got this error, although the above many solution eg this one: https://community.home-assistant.io/t/wireguard-stopped-working/352348 suggest: Turns out I can create this error at will with this in the config for a peer: allowed_ips: - 0.0.0.0/0 but this does not fail: allowed_ips: []
Have you tried setting the option, e.g. with docker
docker run --sysctl net.ipv4.conf.all.src_valid_mark=1 [...]
?
it's a home assistant addon where the container run by the supervisor... but has a lot's of privileges:
ost_network: true
host_dbus: true
privileged:
- SYS_ADMIN
- SYS_RESOURCE
- NET_ADMIN
- NET_RAW
- BPF
Hello @lfarkas,
We're currently reviewing our open issues and would like to verify if this problem still exists in the latest NetBird version.
Could you please confirm if the issue is still there?
We may close this issue temporarily if we don't hear back from you within 2 weeks, but feel free to reopen it with updated information.
Thanks for your contribution to improving the project!
still exists v0.43.0:
2025-04-28T20:09:05+02:00 ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 1 error occurred:
* write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system
2025-04-28T20:09:05+02:00 WARN client/firewall/nftables/router_linux.go:956: Will use nftables to manipulate the filter table because iptables is not available: exec: "iptables": executable file not found in $PATH
2025-04-28T20:09:05+02:00 WARN client/firewall/nftables/router_linux.go:863: Will use nftables to manipulate the filter table because iptables is not available: exec: "iptables": executable file not found in $PATH
2025-04-28T20:09:05+02:00 ERRO client/firewall/nftables/router_linux.go:108: failed to set up data plane mark: flush: conn.Receive: netlink receive: operation not supported
netlink receive: operation not supported
2025-06-30T07:08:02Z ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 3 errors occurred:
* write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system
* write sysctl net.ipv4.conf.all.rp_filter: open /proc/sys/net/ipv4/conf/all/rp_filter: read-only file system
* write sysctl net.ipv4.conf.eth0.rp_filter: open /proc/sys/net/ipv4/conf/eth0/rp_filter: read-only file system
And later:
2025-06-30T07:08:04Z WARN client/internal/routemanager/systemops/systemops_linux.go:161: Default route is configured but sysctl operations failed, VPN traffic may not be routed correctly, consider using NB_USE_LEGACY_ROUTING=true or setting net.ipv4.conf.*.rp_filter to 2 (loose) or 0 (off)
Perhaps the docker setup doco needs to be updated here?
Workaround:
peer_name=HOSTNAME
setup_key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
docker run --restart unless-stopped --name netbird --hostname "$peer_name" --cap-add=NET_ADMIN --cap-add=SYS_ADMIN --cap-add=SYS_RESOURCE --env NB_SETUP_KEY="$setup_key" --volume netbird-client:/etc/netbird --sysctl net.ipv4.conf.all.src_valid_mark=1 --sysctl net.ipv4.conf.all.rp_filter=2 --sysctl net.ipv4.conf.eth0.rp_filter=2 --detach netbirdio/netbird:latest
Could the "triage needed" be removed based on this and the previous?
@HaleTom does it work fully with those additional container arguments?
I have never dug much into details, but the general recommendation for those errors was always to configure sysctls on the host system before running the container.
Yes - all good with those args.
For the doco:
You may need to add an analogous eth1 if you have two ethernet network interfaces, or a different interface as listed in the ERR messages.
@HaleTom related to https://github.com/netbirdio/netbird/issues/2345 & https://github.com/netbirdio/docs/pull/390
Could I ask you to try --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF (alternatively switch CAP_BPF with CAP_SYS_ADMIN if you're running kernel older than 5.8) in your setup? I found those 3 working just fine on my system with slightly less privileges than suggested.
these used and still not working:-(
privileged:
- SYS_ADMIN
- SYS_RESOURCE
- NET_ADMIN
- NET_RAW
- BPF
@lfarkas could you share what exactly isn't working with those capabilities? Could you share the logs/debug bundle?
Add-on version: v0.50.2
You are running the latest version of this add-on.
System: Home Assistant OS 16.0 (amd64 / generic-x86-64)
Home Assistant Core: 2025.7.1
Home Assistant Supervisor: 2025.07.1
-----------------------------------------------------------
Please, share the above information when looking for help
or support in, e.g., GitHub, forums or the Discord chat.
-----------------------------------------------------------
s6-rc: info: service base-addon-banner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service base-addon-log-level: starting
s6-rc: info: service fix-attrs successfully started
Log level is set to WARNING
s6-rc: info: service base-addon-log-level successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service netbird: starting
s6-rc: info: service netbird successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
2025-07-10T22:30:15+02:00 ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 1 error occurred:
* write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system
2025-07-10T22:30:15+02:00 WARN client/firewall/nftables/router_linux.go:952: Will use nftables to manipulate the filter table because iptables is not available: exec: "iptables": executable file not found in $PATH
2025-07-10T22:30:15+02:00 WARN client/firewall/nftables/router_linux.go:859: Will use nftables to manipulate the filter table because iptables is not available: exec: "iptables": executable file not found in $PATH
2025-07-10T22:30:15+02:00 ERRO client/firewall/nftables/router_linux.go:108: failed to set up data plane mark: flush: conn.Receive: netlink receive: operation not supported
netlink receive: operation not supported
2025-07-10T22:30:15+02:00 WARN client/internal/dns/server.go:440: the DNS manager of this peer doesn't support custom port. Disabling primary DNS setup. Learn more at: https://docs.netbird.io/how-to/manage-dns-in-your-network#local-resolver
2025-07-10T22:30:15+02:00 ERRO client/internal/dns/server.go:495: failed to apply DNS host manager update: unable to configure DNS for this peer using file manager without a nameserver group with all domains configured
2025-07-10T22:51:15+02:00 WARN [peer: vHwiLU2vP4BqhiKwrZ04UIskMKPxIomXz39OqxnSb1Q=] client/internal/peer/wg_watcher.go:130: WireGuard handshake timed out, closing relay connection: 2025-07-10 22:47:46.409118108 +0200 CEST
2025-07-11T01:58:56+02:00 WARN management/client/grpc.go:171: disconnected from the Management service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR
2025-07-11T05:11:11+02:00 WARN signal/client/grpc.go:162: disconnected from the Signal service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR
2025-07-11T05:31:56+02:00 ERRO [peer: gsrpCbJwc8lkmNV783rxIHpyj+zZIhy/rFj5HsfVuBY=] client/internal/peer/conn.go:574: failed to send offer: signal is not ready
2025-07-11T06:07:45+02:00 ERRO [peer: RtObgAe/KslyFa/t0a/iGwy7HohRzO8xhNNUPIR1ri8=] client/internal/peer/wg_watcher.go:53: WireGuard watcher already enabled
2025-07-11T06:18:15+02:00 ERRO [peer: f+tmDAAoOYRUT/WAoJl0PsqalR4zJvt7ljkxZboO9iE=] client/internal/peer/wg_watcher.go:53: WireGuard watcher already enabled
2025-07-11T08:11:24+02:00 WARN [peer: vHwiLU2vP4BqhiKwrZ04UIskMKPxIomXz39OqxnSb1Q=] client/internal/peer/wg_watcher.go:130: WireGuard handshake timed out, closing relay connection: 2025-07-11 08:09:42.131427796 +0200 CEST
2025-07-11T11:48:48+02:00 WARN management/client/grpc.go:171: disconnected from the Management service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR
2025-07-11T12:27:56+02:00 WARN signal/client/grpc.go:162: disconnected from the Signal service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR
2025-07-11T12:28:50+02:00 ERRO [peer: Yg/JDeFsAfMnue9KOTNm77L0AlG1g3Y6pYIm3KhUxyw=] client/internal/peer/conn.go:574: failed to send offer: rpc error: code = Unknown desc = error sending message
2025-07-11T12:28:59+02:00 WARN signal/client/grpc.go:162: disconnected from the Signal service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR
2025-07-11T12:32:35+02:00 WARN signal/client/grpc.go:162: disconnected from the Signal service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR
2025-07-11T12:32:45+02:00 WARN signal/client/grpc.go:143: disconnected from the Signal Exchange due to an error: didn't receive a registration header from the Signal server whille connecting to the streams
2025-07-11T12:33:20+02:00 ERRO [peer: Yg/JDeFsAfMnue9KOTNm77L0AlG1g3Y6pYIm3KhUxyw=] client/internal/peer/conn.go:574: failed to send offer: rpc error: code = Unknown desc = error sending message
2025-07-11T12:34:22+02:00 ERRO [peer: vHwiLU2vP4BqhiKwrZ04UIskMKPxIomXz39OqxnSb1Q=] client/internal/peer/conn.go:574: failed to send offer: rpc error: code = Unknown desc = error sending message
2025-07-11T12:34:24+02:00 ERRO [peer: Yg/JDeFsAfMnue9KOTNm77L0AlG1g3Y6pYIm3KhUxyw=] client/internal/peer/conn.go:574: failed to send offer: rpc error: code = Unknown desc = error sending message
Live
@lfarkas I'm not sure how the service is set up, but it looks like you're missing the following arguments mentioned above in the thread in case it is a container
--sysctl="net.ipv4.conf.all.src_valid_mark=1"--sysctl="net.ipv4.conf.all.rp_filter=2"--sysctl="net.ipv4.conf.eth0.rp_filter=2"- you might need to use other interface thaeth0here
I would recommend setting the above on the container host, at least on the interface-specific rule.
This seems more like some kind of Home Assistant OS issue?
PS: I just noticed we have a repo for HA integration and looks like you're the main contributor?
yes unfortunately on HAOS it's not possible to set these sysctl i can only set capabilities only these: https://developers.home-assistant.io/docs/add-ons/configuration/ Privilege for access to hardware/system. Available access: BPF, CHECKPOINT_RESTORE, DAC_READ_SEARCH, IPC_LOCK, NET_ADMIN, NET_RAW, PERFMON, SYS_ADMIN, SYS_MODULE, SYS_NICE, SYS_PTRACE, SYS_RAWIO, SYS_RESOURCE or SYS_TIME. or host_network or full_access but these 2 wouldn't like to add...
yes unfortunately on HAOS it's not possible to set these sysctl
Does HAOS allow setting those on the host-level? If it's already set by the time NetBird container starts up it won't try to modify those settings.
@HaleTom related to #2345 & netbirdio/docs#390
Could I ask you to try
--cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF(alternatively switchCAP_BPFwithCAP_SYS_ADMINif you're running kernel older than 5.8) in your setup? I found those 3 working just fine on my system with slightly less privileges than suggested.
2025-07-16T13:40:21Z ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 3 errors occurred:
* write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system
* write sysctl net.ipv4.conf.all.rp_filter: open /proc/sys/net/ipv4/conf/all/rp_filter: read-only file system
* write sysctl net.ipv4.conf.eth0.rp_filter: open /proc/sys/net/ipv4/conf/eth0/rp_filter: read-only file system
docker run --restart unless-stopped --name netbird-no-sysctl --hostname "$peer_name" --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_SYS_ADMIN --env NB_SETUP_KEY="$setup_key" --volume netbird-client:/etc/netbird --detach netbirdio/netbird:latest gives:
2025-07-16T13:45:07Z ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 3 errors occurred:
* write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system
* write sysctl net.ipv4.conf.all.rp_filter: open /proc/sys/net/ipv4/conf/all/rp_filter: read-only file system
* write sysctl net.ipv4.conf.eth0.rp_filter: open /proc/sys/net/ipv4/conf/eth0/rp_filter: read-only file system
@nazarewk I'm revisiting this and I'm afraid I gave you a bum steer with my command before. The command with eth0 doesn't work (see log entry below). Sorry - this means that the current documentation produces a docker error (also listed below).
TL;DR:
- Remove all references to
eth*from docker command line (see errors further below if this is there) - Create a docker network and apply the sysctls to the interface attached to that network
# Create network to apply com.docker.network.endpoint.sysctls:
docker network create netbird-net
docker run \
--restart unless-stopped \
--name netbird \
--hostname "$peer_name" \
--env NB_SETUP_KEY="$setup_key" \
--cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF \
--volume netbird-client:/etc/netbird \
--sysctl net.ipv4.conf.all.src_valid_mark=1 \
--sysctl net.ipv4.conf.all.rp_filter=2 \
--network=name=netbird-net,"driver-opt=com.docker.network.endpoint.sysctls=net.ipv4.conf.IFNAME.rp_filter=2"\
--detach netbirdio/netbird:latest
Above, docker will replace IFNAME with the interface that it creates.
As I (incorrectly) said to do it before:
docker run --restart unless-stopped --name netbird --hostname "$peer_name" --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF --env NB_SETUP_KEY="$setup_key" --volume netbird-client:/etc/netbird --sysctl net.ipv4.conf.all.src_valid_mark=1 --sysctl net.ipv4.conf.all.rp_filter=2 --sysctl net.ipv4.conf.eth0.rp_filter=2 --detach netbirdio/netbird:latest
This produces:
docker: Error response from daemon: interface specific sysctl setting "net.ipv4.conf.eth0.rp_filter" must be supplied using driver option 'com.docker.network.endpoint.sysctls'
Dropping eth0 from the command line prevents docker complaining immediately:
docker run --restart unless-stopped --name netbird --hostname "$peer_name" --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF --env NB_SETUP_KEY="$setup_key" --volume netbird-client:/etc/netbird --sysctl net.ipv4.conf.all.src_valid_mark=1 --sysctl net.ipv4.conf.all.rp_filter=2 --detach netbirdio/netbird:latest
But I still get:
2025-07-16T13:55:03Z ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 1 error occurred:
* write sysctl net.ipv4.conf.eth0.rp_filter: open /proc/sys/net/ipv4/conf/eth0/rp_filter: read-only file system
AFAICT from this issue, docker always mounts /proc/sys RO, so even with --cap-add=ALL, it's not possible to have the container change these values unless docker is run --privileged (which is generally considered bad form).
@HaleTom, thanks for the more extensive information. I'll try to take a look at it later on.
