flannel on loopback interface

[gfrankliu]

As discussed here, using unicast ip as secondary on loopback is now supported, but I have problem making it work with latest k3s and default flannel. The ultimate goal is to have a single node k3s cluster without external network. Since loopback is always online, I am trying to leverage it.

On a single node Debian 12, I assigned 10.10.10.10 as secondary ip to loopback interface:

ip address add 10.10.10.10/32 dev lo
ip route add default via 10.10.10.10 dev lo metric 1000

I also added below to /etc/network/interfaces so that the secondary IP and backup dummy default route can be added after system reboot:

auto lo:0
iface lo:0 inet static
  address 10.10.10.10/32
  up ip route add default via 10.10.10.10 dev $IFACE metric 1000 || true

I then installll latest k3s with default flannel cni:

curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode 644 --disable-network-policy --flannel-backend=host-gw --cluster-cidr=10.200.0.0/16 --service-cidr=10.201.0.0/16 --disable traefik --flannel-iface=lo --advertise-address=10.10.10.10 --node-ip=10.10.10.10

The flannel fails to work. sudo journalctl -u k3s.service gives the error:

Jun 29 00:12:17 debian-arm64 k3s[922]: E0629 00:12:17.448967     922 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8c019ef97d66ec0550d7bd84b411c1a3dc7168e50ca6da0c4702f6c31bd104b1\": plugin type=\"flannel\" failed (add): failed to create bridge \"cni0\": could not add \"cni0\": invalid argument"
Jun 29 00:12:17 debian-arm64 k3s[922]: E0629 00:12:17.449000     922 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8c019ef97d66ec0550d7bd84b411c1a3dc7168e50ca6da0c4702f6c31bd104b1\": plugin type=\"flannel\" failed (add): failed to create bridge \"cni0\": could not add \"cni0\": invalid argument" pod="kube-system/metrics-server-6f4c6675d5-fndcm"
Jun 29 00:12:17 debian-arm64 k3s[922]: E0629 00:12:17.449012     922 kuberuntime_manager.go:1237] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8c019ef97d66ec0550d7bd84b411c1a3dc7168e50ca6da0c4702f6c31bd104b1\": plugin type=\"flannel\" failed (add): failed to create bridge \"cni0\": could not add \"cni0\": invalid argument" pod="kube-system/metrics-server-6f4c6675d5-fndcm"

BTW, the same setup works if I use k3s with cilium cni.

[gfrankliu]

Looks like cilium cni had the similar issue last year as mentioned in this ticket but they have fixed it since. That explained why I didn't have this issue when I setup k3s with cilium and only flannel failed.

[manuelbuil]

Maybe creating a dummy interface fulfills your use case and works?

[gfrankliu]

Yes that's the current workaround k3s people are using but I thought I would open an issue here so flannel team can track and properly fix it one day.