k0sctl icon indicating copy to clipboard operation
k0sctl copied to clipboard

Multiple privateAddress (node InternalIP)

Open Sensei-CHO opened this issue 2 years ago • 2 comments

I'm trying to deploy a k0s cluster using k0sctl with dualStack ipv4/ipv6. My goal is to use cilium and make use of nodes InternalIPs.

k0sctl.yaml:

apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
  name: magi
spec:
  hosts:
  - role: controller+worker
    privateInterface: eth0
    ssh:
      address: 192.168.1.50
      user: root
      keyPath: ~/.ssh/id_rsa
    noTaints: true
  - role: worker
    privateInterface: eth0
    ssh:
      address: 192.168.1.60
      user: root
      keyPath: ~/.ssh/id_rsa
  - role: worker
    privateInterface: eth0
    ssh:
      address: 192.168.1.70
      user: root
      keyPath: ~/.ssh/id_rsa
  k0s:
    version: v1.27.1+k0s.0
    config:
      apiVersion: k0s.k0sproject.io/v1beta1
      kind: ClusterConfig
      metadata:
        name: magi
      spec:
        network:
          podCIDR: "10.244.0.0/16"
          serviceCIDR: "10.96.0.0/12"
          provider: custom
          dualStack:
            enabled: true
            IPv6podCIDR: "fd00::/108"
            IPv6serviceCIDR: "fd01::/108"

Each node have an IPv4 and and IPv6 address, the problem is that when I deploy my cluster, my nodes only have the IPv4 InternalIP.

Is there any way to set multiples InternalIP on cluster deployment?

UPDATE: I achieve to get this working by modifying manually the k0scontroller.service:

ExecStart=/usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml --enable-worker=true --no-taints=true --kubelet-extra-args=--node-ip=192.168.1.x,2a12:xxxx:xxxx:xxxx::x

The --node-ip kubelet argument can take multiple IP addresses separated with commas, but k0sctl cannot:

spec:
  hosts:
  - role: controller+worker
    privateAddress: 192.168.1.50,2a12:xxxx:xxxx:xxxx::x

FATA configuration validation failed: spec: (hosts: host #1: privateAddress: must be a valid IP address..).

Sensei-CHO avatar May 26 '23 08:05 Sensei-CHO

Requires some changes, but it's doable.

kke avatar May 31 '23 07:05 kke

This needs a bit more thought I think. currently (as fas as I remember) this k0sctl field also maps to some k0s.yaml fields. Like for example the etcd peering address. So if we have multiple addresses, which one should be used for etcd?

jnummelin avatar Jun 05 '23 12:06 jnummelin