Sync the containerd files for Kubernetes

[afbjorklund]

Comparing with: ~~https://github.com/containerd/containerd/blob/main/docs/cri/installation.md~~

Unfortunately the upstream documentation has been deprecated, and no longer includes the configuration needed for CRI and CNI.

It is also no longer available from the kubernetes documentation, so it is up to the user to figure out /etc/crictl.yaml and /etc/cni/net.d

https://github.com/containerd/containerd/blob/main/script/setup/install-critools

https://github.com/containerd/containerd/blob/main/script/setup/install-cni

Upstream includes configuration for CRI and CNI, so make sure that is added to the nerdctl install. Currently using "flannel" for multi-node and VXLAN, but template can be edited to use "containerd-net".

The image endpoint defaults to the runtime endpoint, so use the same configuration file as upstream has. Currently we use kubernetes-cni which is a slightly older version so don't bump the cniVersion (yet).

---

Reviewer note: this is basically a no-op, unless edited

We create the default file, and then we delete it again

[afbjorklund]

The end goal here was to make the installation more similar as to what is documented upstream:

https://kubernetes.io/docs/setup/production-environment/container-runtimes/#containerd

There is a new version of this document for k8s 1.24, changing from docker.com to containerd.io

https://deploy-preview-32738--kubernetes-io-main-staging.netlify.app/docs/setup/production-environment/container-runtimes/#containerd

And it recommends using the "cri-containerd-cni" tarball:

https://containerd.io/docs/getting-started/#starting-containerd

i.e. including the /etc/crictl.yaml and /etc/cni/net.d (and plugins)

But the configuration is not included, in the deb packages...

[afbjorklund]

When not using flannel, then the pod-network-cidr should be removed too (to default to 10.88.0.0/16)

Ideally this should have some kind of boolean configuration, but I think that is overkill for a yaml file...

containerd

The default configuration in /etc/cni/net.d/10-containerd-net.conflist works, without modifications.

    systemctl enable --now kubelet

    kubeadm init --cri-socket=/run/containerd/containerd.sock --apiserver-cert-extra-sans 127.0.0.1

flannel

The default bridge CNI must be deleted, and the flannel pod will create one once the cluster is booted.

    rm -f /etc/cni/net.d/*.conf*

    systemctl enable --now kubelet

    # Initializing your control-plane node
    kubeadm init --cri-socket=/run/containerd/containerd.sock --pod-network-cidr=10.244.0.0/16 --apiserver-cert-extra-sans 127.0.0.1
    # Installing a Pod network add-on
    kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/v0.14.0/Documentation/kube-flannel.yml

[afbjorklund]

Side note: the new version of flannel will install another copy of cni-plugins (1.0.1), because the kubernetes-cni are too old (0.8.7)

It is copied to the host from the container image, overwriting any existing /opt/cni/bin/flannel.

REPOSITORY                           TAG        IMAGE ID       CREATED         SIZE
flannelcni/flannel                   v0.17.0    9247abf08677   2 weeks ago     59.8MB
flannelcni/flannel-cni-plugin        v1.0.1     ac40ce625740   8 weeks ago     8.1MB

https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml

      initContainers:
      - name: install-cni-plugin
       #image: flannelcni/flannel-cni-plugin:v1.0.1 for ppc64le and mips64le (dockerhub limitations may apply)
        image: rancher/mirrored-flannelcni-flannel-cni-plugin:v1.0.1
        command:
        - cp
        args:
        - -f
        - /flannel
        - /opt/cni/bin/flannel
        volumeMounts:
        - name: cni-plugin
          mountPath: /opt/cni/bin
      - name: install-cni
       #image: flannelcni/flannel:v0.17.0 for ppc64le and mips64le (dockerhub limitations may apply)
        image: rancher/mirrored-flannelcni-flannel:v0.17.0
        command:
        - cp
        args:
        - -f
        - /etc/kube-flannel/cni-conf.json
        - /etc/cni/net.d/10-flannel.conflist
        volumeMounts:
        - name: cni
          mountPath: /etc/cni/net.d
        - name: flannel-cfg

The default "containerd-net" network runs without any extra container images, using the installation from the deb packages.

cri-tools/kubernetes-xenial,now 1.23.0-00 amd64
kubernetes-cni/kubernetes-xenial 0.8.7-00 amd64

[afbjorklund]

The deployment for flannel is not included in the release, so installing 0.17.0 actually installs 0.16.3

https://raw.githubusercontent.com/flannel-io/flannel/v0.17.0/Documentation/kube-flannel.yml

      containers:
      - name: kube-flannel
       #image: flannelcni/flannel:v0.16.3 for ppc64le and mips64le (dockerhub limitations may apply)
        image: rancher/mirrored-flannelcni-flannel:v0.16.3

Also installing from rancher, possibly due to pull limits

[afbjorklund]

https://raw.githubusercontent.com/flannel-io/flannel/v0.17.0-1-gbf9dbdb0/Documentation/kube-flannel.yml

Includes https://github.com/flannel-io/flannel/commit/bf9dbdb0e7e7dd71311b3de1affb1e27c5bb1d5e

Will move flannel upgrade to a separate PR.

Would still like to have it "optional", though.

[afbjorklund]

I guess it can default to using flannel on both Kubernetes distributions, as a Lima feature.

Upstream containerd seems to be deprecating the bundled CNI configuration anyway...

https://github.com/containerd/containerd/blob/main/docs/getting-started.md

The cri-containerd-... archives are deprecated, do not work on old Linux distributions, and will be removed in containerd 2.0.

So in case the bridge network doesn't work anymore then, don't add another copy here.

Will do a new PR, just to remove the redundant (default) line from the CRI configuration...

@@ -62,7 +62,6 @@ provision:
     apt-get install -y cri-tools
     cat  <<EOF | sudo tee /etc/crictl.yaml
     runtime-endpoint: unix:///run/containerd/containerd.sock
-    image-endpoint: unix:///run/containerd/containerd.sock
     EOF
     # cni-plugins
     apt-get install -y kubernetes-cni

[afbjorklund]

• #1384