Detect dir for CNI bins

[brb]

A user on Fedora 28 reported that kubelet is not able to locate the weave-net CNI bin. The relevant params of the kubelet: --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/usr/libexec/cni set by kubeadm v1.10.1 reveals a different location than weave setup-cni expects (/opt/cni/bin).

It seems that the directory is a distro specific: https://github.com/projectcalico/cni-plugin/issues/342

We should check multiple dirs for installation, i.e. /usr/libexec/cni and/opt/cni/bin.

[bboreham]

Current behaviour of prog/weave-kube/launch.sh is to try /opt/cni/bin then /home/kubernetes/bin which is where ContainerOS puts it.

We could inspect the kubelet args. Or just add a third...

[brb]

Maybe there is a nice way to get the --cni-bin-dir value w/o reading /proc/$KUBELET/cmdline? Asking, as otherwise weave setup-cni would depend on kubelet being up and running.

[murali-reddy]

Accessing kubelet cmd line arguments also mean hostPID: true (though its already set).

I can not think of any elegant solution. Let me know if you are fine with adding /usr/libexec/cni as third directory that launch.sh will check.

Alternatively launch.sh can check if an environment variable CNI_CONF_DIR for e.g. exist and use it to install the CNI.

[bboreham]

I think it's ok to add a third option, and to add an environment variable. Note we are talking about the bin dir not the conf dir.

What is also a problem is we need access to each directory: we already have this in the DaemonSet:

          volumes:
[...]
            - name: cni-bin
              hostPath:
                path: /opt
            - name: cni-bin2
              hostPath:
                path: /home

if we carry on with /usr/libexec it starts to get silly.

[murali-reddy]

Sorry i meant CNI_BIN_DIR.

if we carry on with /usr/libexec it starts to get silly.

Yes. May be just add /usr as well for now? for sake of convenience, a single manifest that works.

If it going to get any messier in future, perhaps we should introduce environment variable that launch.sh considers and document on how user should expose the required host path to weave-kube pods.

[rajch]

Just got hit by this, on 05 November 2020, while setting up weave on Alpine Linux. I have never contributed to weave before, but will be happy to take this up if required.

[bboreham]

@rajch The problem was that Kubelet is looking somewhere else? Which kubelet, and where is it looking?

[rajch]

@rajch The problem was that Kubelet is looking somewhere else? Which kubelet, and where is it looking?

This is kubelet v1.19.3, on Alpine Linux edge (3.13.0). CNI binaries are in /usr/libexec/cni, and kubelet is configured to look there by default.

[rajch]

I have currently solved the problem by editing the manifest, and adding the following:

1. Mount /usr to /host/usr in the Daemonset
2. Add environment variable WEAVE_CNI_PLUGIN_DIR with the value /host/usr/libexec/cni

It would be nice if launch.sh checked for the /usr/libexec/cni directory too, and the manifest mounted it.