[feat]: New flavor for Cilium BGP load-balancing for Services

[AshleyDumaine]

What type of PR is this? /kind feature

What this PR does / why we need it: New flavor for using the linode-CCM without creating NodeBalancers to perform Service load-balancing.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): Fixes #

Special notes for your reviewer: Requires https://github.com/linode/linode-cloud-controller-manager/pull/208 be merged and a new release version to be cut and published for the Helm Chart

TODOs:

• [ ] squashed commits
• [x] includes documentation
• [ ] adds unit tests
• [ ] adds or updates e2e tests

Testing

Since the image for the CCM is not yet available as well as the Chart, you'll need to edit the image and args on the CCM daemonset after the HelmChartRelease completes on the created cluster:

      spec:
        containers:
        - args:
          - --leader-elect-resource-lock=leases
          - --v=3
          - --secure-port=10253
          - --webhook-secure-port=0
          - --enable-route-controller=true
          - --vpc-name=cilium-bgp-lb
          - --configure-cloud-routes=true
          - --cluster-cidr=10.0.0.0/8
          - --bgp-node-selector=cilium-bgp-peering=true
          - --load-balancer-type=cilium-bgp
          image: ashleydumaine/linode-cloud-controller-manager:shared-ip
          imagePullPolicy: Always

You'll also need to do kubectl edit clusterrole ccm-linode-clusterrole to add the cilium RBAC pemissions:

- apiGroups: ["cilium.io"]
  resources: ["ciliumloadbalancerippools"]
  verbs: ["get", "watch", "list", "update", "create", "delete"]
- apiGroups: ["cilium.io"]
  resources: ["ciliumbgppeeringpolicies"]
  verbs: ["get", "watch", "list", "create"]

After the Damonset and Clusterrole are updated, deploy a LoadBalancer service (no need to set the loadBalancerClass), and check the following:

• BGP nodes exist (2 by default in the MachineDeployment)
• The BGP nodes are labeled with cilium-bgp-peering=true and node.k8s.linode.com/ip-sharing-updated=true once an LB Service is created
• if you scale up the bgp MachineDeployment (k edit md $CLUSTER_NAME-md-bgp), the new Node eventually joins and gets the cilium-bgp-peering=true and node.k8s.linode.com/ip-sharing-updated=true labels. Looking at Cloud Manager, you should see the new Linode has the shared IP for the LB Service added in the "Networking" tab.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 66.09%. Comparing base (9cd0d2d) to head (7a99734).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #317   +/-   ##
=======================================
  Coverage   66.09%   66.09%           
=======================================
  Files          40       40           
  Lines        2274     2274           
=======================================
  Hits         1503     1503           
  Misses        704      704           
  Partials       67       67           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[rahulait]

Should we add these flavor changes to kubeadm-full and kubeadm-full-vpcless flavors as well?

[AshleyDumaine]

Should we add these flavor changes to kubeadm-full and kubeadm-full-vpcless flavors as well?

maybe in a follow-up PR?