weave
weave copied to clipboard
weaveworks
FATA: creating dummy interface: operation not supported
Install WeaveNet as pod network add-on according to doc https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network
but got this error and result to coreDNS pending
OS info
[root@k8s-master-01 init-k8s]# uname -a
Linux k8s-master-01 4.19.0-1.0.0.2.rc1 #25 SMP PREEMPT Mon Sep 16 06:58:47 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[root@k8s-master-01 init-k8s]# cat /etc/centos-release
CentOS Linux release 7.7.1908 (Core)
[root@k8s-master-01 init-k8s]# kubectl -v
Error: flag needs an argument: 'v' in -v
See 'kubectl --help' for usage.
[root@k8s-master-01 init-k8s]# kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.1", GitCommit:"d224476cd0730baca2b6e357d144171ed74192d6", GitTreeState:"clean", BuildDate:"2020-01-14T21:04:32Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-11T18:07:13Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"linux/amd64"}
interface info, xgbe1 as the k8s cluster interface
[root@k8s-master-01 init-k8s]# ifconfig xgbe0
xgbe0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 119.249.51.135 netmask 255.255.255.128 broadcast 119.249.51.255
ether b8:59:9f:01:a7:28 txqueuelen 1000 (Ethernet)
RX packets 651192 bytes 720059616 (686.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 100215 bytes 10386259 (9.9 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@k8s-master-01 init-k8s]# ifconfig xgbe1
xgbe1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.11 netmask 255.255.255.0 broadcast 192.168.0.255
ether 70:57:bf:37:20:73 txqueuelen 1000 (Ethernet)
RX packets 150388 bytes 9322340 (8.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 51398 bytes 3420122 (3.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@k8s-master-01 init-k8s]# ip addr xgbe1
Command "xgbe1" is unknown, try "ip address help".
[root@k8s-master-01 init-k8s]# ip addr show xgbe1
2: xgbe1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 70:57:bf:37:20:73 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.11/24 brd 192.168.0.255 scope global xgbe1
valid_lft forever preferred_lft forever
inet 192.168.0.100/24 scope global secondary xgbe1
valid_lft forever preferred_lft forever
Error
[root@k8s-master-01 init-k8s]# kubectl -n kube-system logs weave-net-8w88p weave
DEBU: 2020/02/24 08:13:06.442549 [kube-peers] Checking peer "a2:0c:15:6c:50:73" against list &{[]}
Peer not in list; removing persisted data
INFO: 2020/02/24 08:13:06.666938 Command line options: map[conn-limit:200 datapath:datapath db-prefix:/weavedb/weave-net docker-api: expect-npc:true host-root:/host http-addr:127.0.0.1:6784 ipalloc-init:consensus=0 ipalloc-range:10.32.0.0/12 log-level:debug metrics-addr:0.0.0.0:6782 name:a2:0c:15:6c:50:73 nickname:k8s-master-01 no-dns:true port:6783]
INFO: 2020/02/24 08:13:06.667007 weave 2.6.0
FATA: 2020/02/24 08:13:06.725482 creating dummy interface: operation not supported
pod info
[root@k8s-master-01 init-k8s]# kubectl -n kube-system get pod
NAME READY STATUS RESTARTS AGE
coredns-9d85f5447-h85x2 0/1 Pending 0 40m
etcd-k8s-master-01 1/1 Running 0 40m
kube-apiserver-k8s-master-01 1/1 Running 0 40m
kube-controller-manager-k8s-master-01 1/1 Running 0 40m
kube-proxy-s2wdg 1/1 Running 0 40m
kube-scheduler-k8s-master-01 1/1 Running 0 40m
weave-net-8w88p 1/2 CrashLoopBackOff 1 7s
Does any One help me ?
FATA: 2020/02/24 08:13:06.725482 creating dummy interface: operation not supported
@Roarain As part of weave-net pod startup steps, a dummy interface is created and deleted. For some reason on your setup it is failing to get created.
Can you please try ip link add test type dummy and see if you are able to create dummy interface manually/
Just encountered this. Running the command above gives me following
# ip link add test type dummy
Error: Unknown device type.
Something unusual about your kernel? Could you provide version details please.
@bboreham Sure.
# uname -a
Linux k3s-server-002 5.4.0-1011-kvm #11-Ubuntu SMP Mon May 11 14:03:06 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
# lsmod
Module Size Used by
ip6table_nat 12288 0
ip6_tables 24576 1 ip6table_nat
openvswitch 114688 0
nsh 12288 1 openvswitch
nf_conncount 12288 1 openvswitch
xt_NFLOG 12288 0
xt_physdev 12288 0
xt_state 12288 0
nfnetlink_log 16384 1
ip_set_hash_ip 32768 12
xt_set 12288 4
veth 20480 0
xt_nat 12288 5
xt_tcpudp 12288 15
xt_mark 12288 5
xt_comment 12288 43
iptable_mangle 12288 1
xt_conntrack 12288 8
xt_MASQUERADE 12288 2
nf_conntrack_netlink 32768 0
xfrm_user 32768 1
xfrm_algo 12288 1 xfrm_user
xt_addrtype 12288 3
iptable_filter 12288 1
iptable_nat 12288 1
nf_nat 28672 5 ip6table_nat,xt_nat,openvswitch,iptable_nat,xt_MASQUERADE
nf_conntrack 86016 8 xt_conntrack,nf_nat,xt_state,xt_nat,openvswitch,nf_conntrack_netlink,nf_conncount,xt_MASQUERADE
nf_defrag_ipv6 12288 2 nf_conntrack,openvswitch
nf_defrag_ipv4 12288 1 nf_conntrack
libcrc32c 12288 3 nf_conntrack,nf_nat,openvswitch
aufs 188416 0
br_netfilter 20480 0
overlay 77824 2
nls_iso8859_1 12288 1
nls_cp437 16384 1
vfat 16384 1
fat 57344 1 vfat
kvm_amd 69632 0
pata_acpi 12288 0
loop 28672 8
efivarfs 12288 1
ip_tables 20480 3 iptable_filter,iptable_nat,iptable_mangle
x_tables 24576 15 xt_conntrack,iptable_filter,xt_NFLOG,xt_state,xt_tcpudp,xt_addrtype,xt_physdev,xt_nat,xt_comment,xt_set,ip6_tables,ip_tables,xt_MASQUERADE,iptable_mangle,xt_mark
Also, weave reports some additional things in logs:
INFO: 2020/05/24 09:15:52.780236 weave 2.6.2
WARN: 2020/05/24 09:15:52.858448 Skipping bridge creation of "bridged_fastdp" due to: kernel does not have Open vSwitch VXLAN support: bridge not supported
FATA: 2020/05/24 09:15:52.865903 creating dummy interface: operation not supported
root@k3s-online-01:/opt/cni/bin# kubectl logs -f weave-net-gnh2z weave
INFO: 2020/05/24 09:15:52.780110 Command line options: map[conn-limit:200 datapath:datapath db-prefix:/weavedb/weave-net docker-api: expect-npc:true host-root:/host http-addr:127.0.0.1:6784 ipalloc-init:consensus=1 ipalloc-range:100.64.0.0/12 metrics-addr:0.0.0.0:6782 name:52:80:ab:ee:05:40 nickname:k3s-server-002 no-dns:true port:6783]
INFO: 2020/05/24 09:15:52.780236 weave 2.6.2
WARN: 2020/05/24 09:15:52.858448 Skipping bridge creation of "bridged_fastdp" due to: kernel does not have Open vSwitch VXLAN support: bridge not supported
FATA: 2020/05/24 09:15:52.865903 creating dummy interface: operation not supported
Servers are Scaleway's cloud instances, currently I am trying to make it work with k3s.
Potentially it might be related (I guess) to the openswitch warning above, I'll try to install it and see if anything changes.
I've potentially found the issue.
Looks like kvm kernel (https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/1833375) doesn't have dummy module and in effect
modprobe dummy
modprobe: FATAL: Module dummy not found in directory /lib/modules/5.4.0-1011-kvm
# ls -l /lib/modules/5.4.0-1011-kvm
total 512
lrwxrwxrwx 1 root root 37 May 11 13:50 build -> /usr/src/linux-headers-5.4.0-1011-kvm
drwxr-xr-x 2 root root 4096 May 11 13:50 initrd
drwxr-xr-x 12 root root 4096 May 23 16:31 kernel
-rw-r--r-- 1 root root 38761 May 23 16:31 modules.alias
-rw-r--r-- 1 root root 45049 May 23 16:31 modules.alias.bin
-rw-r--r-- 1 root root 4109 May 11 13:50 modules.builtin
-rw-r--r-- 1 root root 13558 May 23 16:31 modules.builtin.alias.bin
-rw-r--r-- 1 root root 5858 May 23 16:31 modules.builtin.bin
-rw-r--r-- 1 root root 30476 May 11 13:50 modules.builtin.modinfo
-rw-r--r-- 1 root root 31546 May 23 16:31 modules.dep
-rw-r--r-- 1 root root 47638 May 23 16:31 modules.dep.bin
-rw-r--r-- 1 root root 202 May 23 16:31 modules.devname
-rw-r--r-- 1 root root 14178 May 11 13:50 modules.order
-rw-r--r-- 1 root root 419 May 23 16:31 modules.softdep
-rw-r--r-- 1 root root 113764 May 23 16:31 modules.symbols
-rw-r--r-- 1 root root 135517 May 23 16:31 modules.symbols.bin
drwxr-xr-x 3 root root 4096 May 23 16:31 vdso
I'll try to find out if it's possible to add this module (not sure if it's possible with kvm kernels). Otherwise is the dummy interface interface essential for weave functionality?
Thanks for the update. I believe Weave Net uses the dummy device as a way to make some network settings.
Usually I would ask that you open a new issue to avoid confusion with the original one, which has slightly different symptoms. But they never followed up.
Yep, just ran into this with the kvm kernel. Switched to the generic kernel and it works fine there.
If someone wants to make a PR to give Weave Net an alternate way to achieve what it needs, that would be great.
I had the same error on my Raspberry Pi 5 with Ubuntu 23.10 installed. This fixed it: apt install linux-modules-extra-6.5.0-1012-raspi
