multus v4 thick mode breaks NetworkAttachmentDefinition with CNI config file

[leerin-ruby]

What happend: I used multus + istio cni for app sidecar injection, while for the injection, pod reports error with "ERRORED: error configuring pod [nrd/nrd-v1-77c6dbbd48-586x5] networking: Multus: [nrd/nrd-v1-77c6dbbd48-586x5/4c1a126c-c889-472e-9e44-46193c8cfaa3]: error loading k8s delegates k8s args: TryLoadPodDelegates: error in getting k8s network for pod: GetNetworkDelegates: failed getting the delegate: GetCNIConfig: err in GetCNIConfigFromFile: no network available in the name istio-cni in cni dir /host/etc/cni/net.d"

Seems multus CNI directory where other CNI that is used by multus as net-attach-def without config was changed from "/etc/cni/multus/net.d" to "/host/etc/cni/net.d" in multus v4 thick mode.

What you expected to happen: keep the directory as "/etc/cni/multus/net.d" the same as v3 and v4 thin mode

How to reproduce it (as minimally and precisely as possible): install multus v4 thick install istio with cni enabled install a app with sidecar injection

Anything else we need to know?: from the source code, seems the "confDir" in deamonset config is change to "/host/etc/cni/net.d" in v4.0.2 and was removed in master

Environment: tanzu2.5

• Multus version image path and image ID (from 'docker images') Image: ghcr.io/k8snetworkplumbingwg/multus-cni:snapshot-thick Image ID: ghcr.io/k8snetworkplumbingwg/multus-cni@sha256:e47cac8a15cc2f5f7e25a92bcd56e508888d2d7bed25b38aae5b0bb09fb84c1c

• Kubernetes version (use kubectl version): 1.28

• Primary CNI for Kubernetes cluster: calico

• OS (e.g. from /etc/os-release): tkg25-cni-md-0-gqrxm-rtxh5 6.1.62-6.ph5

• File of '/etc/cni/net.d/' 00-multus.conf 10-calico.conflist calico-kubeconfig multus.d

• File of '/etc/cni/multus/net.d' istio-cni.conf ZZZ-istio-cni-kubeconfig

• NetworkAttachment info (use kubectl get net-attach-def -o yaml) `apiVersion: v1 items:

• apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: creationTimestamp: "2024-03-06T08:40:55Z" generation: 1 name: istio-cni namespace: default resourceVersion: "280413" uid: e41aa7e1-a5de-439e-932f-0e41b2268f25 spec: config: "" kind: List metadata: resourceVersion: "" `

• Target pod yaml info (with annotation, use kubectl get pod <podname> -o yaml)

• Other log outputs (if you use multus logging)