How to automatically add routing table for cilium CNI?

[yee379]

I have cilium setup as a secondary delegate network on multus v3.9.1 using:

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: cilium
spec:
  config: |-
    {
      "cniVersion": "0.3.1",
      "name": "cilium",
      "type": "cilium-cni",
      "enable-debug": true,
      "ipam": {
        "type": "host-local",
        "subnet": "10.0.0.0/16",
        "routes": [ { "dst": "0.0.0.0/0" } ],
        "gateway": "10.0.0.1"
      },
      "log-file":"/var/run/cilium/cilium-cni.log"
    }

and cilium installed with the following values.yaml:

cni:
  exclusive: false
ipam:
  mode: cluster-pool

I am able to spin up pods fine using

  annotations:
    k8s.v1.cni.cncf.io/networks: cilium

and pods get IPs etc fine. However, the pods routing table does appear to be configured correctly:

❯ kubectl exec -ti <pod> -- ip route
default via 169.254.1.1 dev eth0
10.0.0.100 dev net1 scope link
169.254.1.1 dev eth0 scope link

specifically, there is no route entry for the cilium subnet at 10.0.0.0/16.

if i manually add the ip route entry via a priviledged container, things seem to work fine (i can ping between net1 cilium interfaces etc); ie

❯ kubectl exec -ti <pod> -- ip route add 10.0.0.0/16 via 10.0.0.100 dev net1

what is the recommended way to have the correct routes injected into the pods? i don't particular want to have privileged (user) init-containers on the cluster.

[ccardenosa]

I take up of this.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 7 days.