How can I declare vip of kube-vip?

[wooarchi]

I used Kubekey provider from ClusterAPI. Also, Kubernetes was deployed based on ClsterClass.

Deployment was successful. However, I think there is a problem in implementing this as GitOps.

The reason is that it is difficult to declare kube-vip address among the values ​​to be declared in Cluster.yaml.

Below are parts of cluster.yml and clusterTemplate.yml.

How can I define kube-vip environment variable address in ClusterTemplate.yml in Cluster.yml?

Do you have any good ideas?

• Cluster.yml

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: kubeops-kk-dev
  namespace: default
  labels:
    provider: kk
    tool: "true"
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
      - 10.233.64.0/18
    serviceDomain: cluster.local
    services:
      cidrBlocks:
      - 10.233.0.0/18
  topology:
    class: kubeops-kk-dev
    version: v1.25.3
    controlPlane:
      replicas: 1
    workers:
      machineDeployments:
       - class: kubeops-kk-dev
         name: kubeops-kkworker
         replicas: 2
    variables:
    - name: user
      value: ubuntu
    - name: password
      value: password
    - name: instances
      value:
      - {address: 10.10.10.11, name: "master1", roles: ["control-plane"]}
      - {address: 10.10.10.31, name: "worker1" , roles: ["worker"]}
      - {address: 10.10.10.32, name: "worker2", roles: ["worker"]}
    - name: controlPlaneLoadBalancer
      value: 10.10.10.100

• ClusterTemplate.yml

---
kind: KubeadmControlPlaneTemplate
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
metadata:
  name: kubeops-kk-control-plane
spec:
  template:
    spec:
      kubeadmConfigSpec:
        files:
        - content: |
            apiVersion: v1
            kind: Pod
            metadata:
              creationTimestamp: null
              name: kube-vip
              namespace: kube-system
            spec:
              containers:
              - args:
                - manager
                env:
                - name: address
                  value: 10.10.10.100
                - name: vip_interface
                  value: ""
                - name: vip_arp
                  value: "true"
                - name: port
                  value: "6443"
                - name: vip_cidr
                  value: "32"
                - name: cp_enable
                  value: "true"
                - name: cp_namespace
                  value: kube-system
                - name: vip_ddns
                  value: "false"
                - name: svc_enable
                  value: "true"
                - name: vip_leaderelection
                  value: "true"
                - name: vip_leaseduration
                  value: "5"
                - name: vip_renewdeadline
                  value: "3"
                - name: vip_retryperiod
                  value: "1"
                - name: lb_enable
                  value: "true"
                - name: lb_port
                  value: "6443"
                image: ghcr.io/kube-vip/kube-vip:v0.5.0
                imagePullPolicy: IfNotPresent
                name: kube-vip
                resources: {}
                securityContext:
                  capabilities:
                    add:
                    - NET_ADMIN
                    - NET_RAW
                volumeMounts:
                - mountPath: /etc/kubernetes/admin.conf
                  name: kubeconfig
              hostNetwork: true
              hostAliases:
                - hostnames:
                    - kubernetes
                  ip: 127.0.0.1
              volumes:
              - hostPath:
                  path: /etc/kubernetes/admin.conf
                  type: FileOrCreate
                name: kubeconfig
            status: {}
          owner: root:root
          path: /etc/kubernetes/manifests/kube-vip.yaml
        joinConfiguration:
          nodeRegistration:
            criSocket: unix:///var/run/containerd/containerd.sock
        initConfiguration:
          nodeRegistration:
            criSocket: unix:///var/run/containerd/containerd.sock

[redscholar]

kube-vip is used for cluster high availability. You can access the entire cluster's API-Server using this address, without worrying about which master node the API-Server is running on. In kubekey, this component is necessary. You just need to configure an available virtual IP address.

[wooarchi]

First of all thanks for your answer.

I want to define kube_vip in cluster.yml not clustertemplate.yml.

I want to define it like in the example below.

What do you think about this?

Any good ideas would be appreciated.

cluster.yml
---
...
    variables:
    - name: user
      value: ubuntu
    - name: password
      value: password
    - name: instances
       value:
       - {address: 10.10.10.11, name: "master1", roles: ["control-plane"]}
       - {address: 10.10.10.31, name: "worker1" , roles: ["worker"]}
       - {address: 10.10.10.32, name: "worker2", roles: ["worker"]}
    - name: controlPlaneLoadBalancer
       value: 10.10.10.100
    - name: kube_vip
       value: ${kube_vip} 

[redscholar]

The controlPlaneLoadBalancer.value in cluster.yaml is actually equivalent to kube_vip.address in ClusterTemplate.yml. When the environment variable CONTROL_PLANE_ENDPOINT_IP is set and the clusterctl generate cluster command is executed, this variable will automatically populate both values.

[redscholar]

We will consider canceling kube-vip in the future. kube-vip is not a necessary option in the case of domain or single-node setups.

[redscholar]

kubekey-provider-for-clusterapi

Defining cluster.yaml

Setting environment variables is a quick way to generate cluster.yaml. It also allows defining the cluster without using the generate command.

About controlPlaneEndpointType

https://github.com/kubesphere/kubekey/blob/e40c57fb9f09695ec7d57ded72f0f652415ba892/config/capkk/release/cluster-template.yaml#L30

• controlPlaneEndpointType: dns

  • kube-vip will not be deployed in the cluster.
  • An existing DNS or IP address must be specified in cluster.spec.controlPlaneEndpoint to access the workload cluster.

• controlPlaneEndpointType: kube_vip

  • kube-vip will be deployed in the cluster.
  • An external VIP (Virtual IP) must be defined to access the workload cluster.