cannot set "host-device" interface name to "net1": validateIfName: interface name net1 already exists

[cucosion]

What happend:

Warning FailedCreatePodSandBox 20s (x16 over 35s) kubelet (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ee66bc0cfe368c3512d5d0504cf9d625d465cddf9d5d290a61bceb785f87fc59": plugin type="multus" name="multus-cni-network" failed (add): [default/teravm-testmodule-client-1-0/5f30eb60-e2e9-4545-be53-1e02095d4e92:sriovnetens1f0]: error adding container to network "sriovnetens1f0": DelegateAdd: cannot set "host-device" interface name to "net1": validateIfName: interface name net1 already exists

What you expected to happen:

pod to be created with two interfaces, the second one with the host-device CNI plugin

How to reproduce it (as minimally and precisely as possible): After i created the nad and created the pod with that nad the first time it worked. After a cluster restart, I tried one more time to deploy same pod and I got the above error

[root@kubemaster ~]# cat multus-cni/tmp/nad.yaml apiVersion: "k8s.cni.cncf.io/v1" kind: NetworkAttachmentDefinition metadata: name: sriovnetens1f0 spec: config: '{ "cniVersion": "0.3.0", "type": "host-device", "device": "ens1f0", "ipam": { "type": "static", "addresses": [ { "address": "192.168.1.23/24" } ] } }'

---

apiVersion: "k8s.cni.cncf.io/v1" kind: NetworkAttachmentDefinition metadata: name: sriovnetens1f1 spec: config: '{ "cniVersion": "0.3.0", "type": "host-device", "device": "ens1f1", "ipam": { "type": "static", "addresses": [ { "address": "192.168.1.24/24" } ] } }' [root@kubemaster ~]# cat multus-cni/tmp/ex.yaml apiVersion: v1 kind: Pod metadata: name: samplepod-left annotations: k8s.v1.cni.cncf.io/networks: sriovnetens1f0@net2 spec: containers:

• name: samplepod-left command: ["/bin/bash", "-c", "trap : TERM INT; sleep infinity & wait"] image: nicolaka/netshoot

Anything else we need to know?: Kubernetes 1.27.1 with Flannel latest version Environment:

• Multus version image path and image ID (from 'docker images') https://github.com/k8snetworkplumbingwg/multus-cni/blob/master/deployments/multus-daemonset.yml ghcr.io/k8snetworkplumbingwg/multus-cni:snapshot kube-multus: Container ID: containerd://d0d567c90ede4dfc12410e6edc8b62ada1a2474ce6bcab936dd50a0647da0140 Image: ghcr.io/k8snetworkplumbingwg/multus-cni:snapshot Image ID: ghcr.io/k8snetworkplumbingwg/multus-cni@sha256:95fc9bceac23de1b643da5f4d0fc0a8c5166fa9c1282cac4cb3cbc37b2e5924d

• Kubernetes version (use kubectl version): kubectl version WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version. Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.1", GitCommit:"4c9411232e10168d7b050c49a1b59f6df9d7ea4b", GitTreeState:"clean", BuildDate:"2023-04-14T13:21:19Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"linux/amd64"} Kustomize Version: v5.0.1 Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.1", GitCommit:"4c9411232e10168d7b050c49a1b59f6df9d7ea4b", GitTreeState:"clean", BuildDate:"2023-04-14T13:14:42Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"linux/amd64"}

• Primary CNI for Kubernetes cluster:

• OS (e.g. from /etc/os-release): [root@kubemaster ~]# cat /etc/os-release NAME="AlmaLinux" VERSION="9.1 (Lime Lynx)" ID="almalinux" ID_LIKE="rhel centos fedora" VERSION_ID="9.1" PLATFORM_ID="platform:el9" PRETTY_NAME="AlmaLinux 9.1 (Lime Lynx)" ANSI_COLOR="0;34" LOGO="fedora-logo-icon" CPE_NAME="cpe:/o:almalinux:almalinux:9::baseos" HOME_URL="https://almalinux.org/" DOCUMENTATION_URL="https://wiki.almalinux.org/" BUG_REPORT_URL="https://bugs.almalinux.org/"

ALMALINUX_MANTISBT_PROJECT="AlmaLinux-9" ALMALINUX_MANTISBT_PROJECT_VERSION="9.1" REDHAT_SUPPORT_PRODUCT="AlmaLinux" REDHAT_SUPPORT_PRODUCT_VERSION="9.1"

• File of '/etc/cni/net.d/' ls /etc/cni/net.d/ 00-multus.conf 10-flannel.conflist multus.d
• File of '/etc/cni/multus/net.d' ls /etc/cni/multus/net.d ls: cannot access '/etc/cni/multus/net.d': No such file or directory [root@kubemaster bin]# ls bandwidth bridge dhcp dummy firewall flannel host-device host-local ipvlan loopback macvlan multus portmap ptp sbr static tuning vlan vrf [root@kubemaster bin]# ./multus version meta-plugin that delegates to other CNI plugins CNI protocol versions supported: 0.1.0, 0.2.0, 0.3.0, 0.3.1, 0.4.0, 1.0.0
• NetworkAttachment info (use kubectl get net-attach-def -o yaml) kubectl get net-attach-def -o yaml apiVersion: v1 items:
• apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"k8s.cni.cncf.io/v1","kind":"NetworkAttachmentDefinition","metadata":{"annotations":{},"name":"sriovnetens1f0","namespace":"default"},"spec":{"config":"{ "cniVersion": "0.3.0", "type": "host-device", "device": "ens1f0", "ipam": { "type": "static", "addresses": [ { "address": "192.168.1.23/24" } ] } }"}} creationTimestamp: "2023-05-08T17:13:07Z" generation: 1 name: sriovnetens1f0 namespace: default resourceVersion: "2582" uid: 2b8500a2-1599-42d8-bfb8-8ffe7d7a3c31 spec: config: '{ "cniVersion": "0.3.0", "type": "host-device", "device": "ens1f0", "ipam": { "type": "static", "addresses": [ { "address": "192.168.1.23/24" } ] } }'
• apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"k8s.cni.cncf.io/v1","kind":"NetworkAttachmentDefinition","metadata":{"annotations":{},"name":"sriovnetens1f1","namespace":"default"},"spec":{"config":"{ "cniVersion": "0.3.0", "type": "host-device", "device": "ens1f1", "ipam": { "type": "static", "addresses": [ { "address": "192.168.1.24/24" } ] } }"}} creationTimestamp: "2023-05-08T17:13:07Z" generation: 1 name: sriovnetens1f1 namespace: default resourceVersion: "2583" uid: ef32ee6e-9e41-4f2f-987e-7595c8b14d82 spec: config: '{ "cniVersion": "0.3.0", "type": "host-device", "device": "ens1f1", "ipam": { "type": "static", "addresses": [ { "address": "192.168.1.24/24" } ] } }' kind: List metadata: resourceVersion: ""
• Target pod yaml info (with annotation, use kubectl get pod <podname> -o yaml) kubectl get pod samplepod-left -o yaml apiVersion: v1 kind: Pod metadata: annotations: k8s.v1.cni.cncf.io/network-status: |- [{ "name": "cbr0", "interface": "eth0", "ips": [ "10.244.0.59" ], "mac": "02:01:2e:21:da:e3", "default": true, "dns": {}, "gateway": [ "10.244.0.1" ] },{ "name": "default/sriovnetens1f0", "interface": "net2", "ips": [ "192.168.1.23" ], "mac": "24:1c:04:f3:d7:fd", "dns": {} }] k8s.v1.cni.cncf.io/networks: sriovnetens1f0@net2 kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{"k8s.v1.cni.cncf.io/networks":"sriovnetens1f0@net2"},"name":"samplepod-left","namespace":"default"},"spec":{"containers":[{"command":["/bin/bash","-c","trap : TERM INT; sleep infinity \u0026 wait"],"image":"nicolaka/netshoot","name":"samplepod-left"}]}} creationTimestamp: "2023-05-08T19:26:27Z" name: samplepod-left namespace: default resourceVersion: "19589" uid: fade95aa-513f-4f39-860c-b3e8647c3331 spec: containers:
  • command:
    • /bin/bash
    • -c
    • 'trap : TERM INT; sleep infinity & wait' image: nicolaka/netshoot imagePullPolicy: Always name: samplepod-left resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts:
    • mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: kube-api-access-468wp readOnly: true dnsPolicy: ClusterFirst enableServiceLinks: true imagePullSecrets:
  • name: docker-registry-credentials nodeName: kubemaster preemptionPolicy: PreemptLowerPriority priority: 0 restartPolicy: Always schedulerName: default-scheduler securityContext: {} serviceAccount: default serviceAccountName: default terminationGracePeriodSeconds: 30 tolerations:
  • effect: NoExecute key: node.kubernetes.io/not-ready operator: Exists tolerationSeconds: 300
  • effect: NoExecute key: node.kubernetes.io/unreachable operator: Exists tolerationSeconds: 300 volumes:
  • name: kube-api-access-468wp projected: defaultMode: 420 sources:
    • serviceAccountToken: expirationSeconds: 3607 path: token
    • configMap: items:
      • key: ca.crt path: ca.crt name: kube-root-ca.crt
    • downwardAPI: items:
      • fieldRef: apiVersion: v1 fieldPath: metadata.namespace path: namespace status: conditions:
  • lastProbeTime: null lastTransitionTime: "2023-05-08T19:26:27Z" status: "True" type: Initialized
  • lastProbeTime: null lastTransitionTime: "2023-05-08T19:26:27Z" message: 'containers with unready status: [samplepod-left]' reason: ContainersNotReady status: "False" type: Ready
  • lastProbeTime: null lastTransitionTime: "2023-05-08T19:26:27Z" message: 'containers with unready status: [samplepod-left]' reason: ContainersNotReady status: "False" type: ContainersReady
  • lastProbeTime: null lastTransitionTime: "2023-05-08T19:26:27Z" status: "True" type: PodScheduled containerStatuses:
  • image: nicolaka/netshoot imageID: "" lastState: {} name: samplepod-left ready: false restartCount: 0 started: false state: waiting: reason: ContainerCreating hostIP: 10.160.160.168 phase: Pending qosClass: BestEffort startTime: "2023-05-08T19:26:27Z"
• Other log outputs (if you use multus logging) journalctl -u kubelet -n 100 -f May 08 20:34:08 kubemaster kubelet[1970]: E0508 20:34:08.035480 1970 kuberuntime_manager.go:1122] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox "8534602124fe984c934d1335860bf3662a5302b2f457e93f98b2d32792e19d6e": plugin type="multus" name="multus-cni-network" failed (add): [default/samplepod-left/fade95aa-513f-4f39-860c-b3e8647c3331:sriovnetens1f0]: error adding container to network "sriovnetens1f0": DelegateAdd: cannot set "host-device" interface name to "net2": validateIfName: interface name net2 already exists" pod="default/samplepod-left" May 08 20:34:08 kubemaster kubelet[1970]: E0508 20:34:08.035531 1970 pod_workers.go:1281] "Error syncing pod, skipping" err="failed to "CreatePodSandbox" for "samplepod-left_default(fade95aa-513f-4f39-860c-b3e8647c3331)" with CreatePodSandboxError: "Failed to create sandbox for pod \"samplepod-left_default(fade95aa-513f-4f39-860c-b3e8647c3331)\": rpc error: code = Unknown desc = failed to setup network for sandbox \"8534602124fe984c934d1335860bf3662a5302b2f457e93f98b2d32792e19d6e\": plugin type=\"multus\" name=\"multus-cni-network\" failed (add): [default/samplepod-left/fade95aa-513f-4f39-860c-b3e8647c3331:sriovnetens1f0]: error adding container to network \"sriovnetens1f0\": DelegateAdd: cannot set \"host-device\" interface name to \"net2\": validateIfName: interface name net2 already exists"" pod="default/samplepod-left" podUID=fade95aa-513f-4f39-860c-b3e8647c3331 May 08 20:34:09 kubemaster kubelet[1970]: E0508 20:34:09.036212 1970 remote_runtime.go:176] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox "1ad2fb3478d4ed997ac19d0812c72aed408ebfd08ceb30b25abbd4191cfef4df": plugin type="multus" name="multus-cni-network" failed (add): [default/samplepod-left/fade95aa-513f-4f39-860c-b3e8647c3331:sriovnetens1f0]: error adding container to network "sriovnetens1f0": DelegateAdd: cannot set "host-device" interface name to "net2": validateIfName: interface name net2 already exists" May 08 20:34:09 kubemaster kubelet[1970]: E0508 20:34:09.036478 1970 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox "1ad2fb3478d4ed997ac19d0812c72aed408ebfd08ceb30b25abbd4191cfef4df": plugin type="multus" name="multus-cni-network" failed (add): [default/samplepod-left/fade95aa-513f-4f39-860c-b3e8647c3331:sriovnetens1f0]: error adding container to network "sriovnetens1f0": DelegateAdd: cannot set "host-device" interface name to "net2": validateIfName: interface name net2 already exists" pod="default/samplepod-left" May 08 20:34:09 kubemaster kubelet[1970]: E0508 20:34:09.036502 1970 kuberuntime_manager.go:1122] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox "1ad2fb3478d4ed997ac19d0812c72aed408ebfd08ceb30b25abbd4191cfef4df": plugin type="multus" name="multus-cni-network" failed (add): [default/samplepod-left/fade95aa-513f-4f39-860c-b3e8647c3331:sriovnetens1f0]: error adding container to network "sriovnetens1f0": DelegateAdd: cannot set "host-device" interface name to "net2": validateIfName: interface name net2 already exists" pod="default/samplepod-left" May 08 20:34:09 kubemaster kubelet[1970]: E0508 20:34:09.036563 1970 pod_workers.go:1281] "Error syncing pod, skipping" err="failed to "CreatePodSandbox" for "samplepod-left_default(fade95aa-513f-4f39-860c-b3e8647c3331)" with CreatePodSandboxError: "Failed to create sandbox for pod \"samplepod-left_default(fade95aa-513f-4f39-860c-b3e8647c3331)\": rpc error: code = Unknown desc = failed to setup network for sandbox \"1ad2fb3478d4ed997ac19d0812c72aed408ebfd08ceb30b25abbd4191cfef4df\": plugin type=\"multus\" name=\"multus-cni-network\" failed (add): [default/samplepod-left/fade95aa-513f-4f39-860c-b3e8647c3331:sriovnetens1f0]: error adding container to network \"sriovnetens1f0\": DelegateAdd: cannot set \"host-device\" interface name to \"net2\": validateIfName: interface name net2 already exists"" pod="default/samplepod-left" podUID=fade95aa-513f-4f39-860c-b3e8647c3331 ^C