k0sctl icon indicating copy to clipboard operation
k0sctl copied to clipboard

custom --kubelet-extra-args overwrite privateInterface setting

Open an-tex opened this issue 2 years ago • 2 comments

when defining both, a privateInterface and --kubelet-extra-args in installFlags, those two clash.

k0sctl writes two separate --kubelet-extra-args parameters in /etc/systemd/system/k0scontroller.service so it looks like this when e.g. defining an own resolvConf: ExecStart=/usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml --enable-worker=true --kubelet-extra-args=--node-ip=10.10.1.12 --no-taints=true --token-file=/etc/k0s/k0stoken --profile=workarounds --kubelet-extra-args=--resolv-conf=/etc/k0s_resolv.conf

what happens is that k0s controller only picks up the last --kubelet-extra-args parameter, hence sth like this is running where the --node-ip parameter is missing: root 3421769 4.8 0.1 846388 113888 ? Sl 17:12 0:20 /var/lib/k0s/bin/kubelet --bootstrap-kubeconfig=/var/lib/k0s/kubelet-bootstrap.conf --v=1 --cert-dir=/var/lib/k0s/kubelet/pki --node-labels=node.k0sproject.io/role=control-plane --container-runtime-endpoint=unix:///run/k0s/containerd.sock --containerd=/run/k0s/containerd.sock --runtime-cgroups=/system.slice/containerd.service --resolv-conf=/etc/k0s_resolv.conf --config=/var/lib/k0s/kubelet-config.yaml --kubeconfig=/var/lib/k0s/kubelet.conf --root-dir=/var/lib/k0s/kubelet

(in my case role: controller+worker, but i imagine the same would happen with k0s worker)

as a workaround you can manually edit the systemd file to merge both parameters like this: ExecStart=/usr/local/bin/k0s controller --config=/etc/k0s/k0s.yaml --enable-worker=true --kubelet-extra-args="--node-ip=10.10.1.12 --resolv-conf=/etc/k0s_resolv.conf" --no-taints=true --token-file=/etc/k0s/k0stoken --profile=workarounds

i see two options here:

  1. k0sctl would automagically merge the kubelet-extra-args it deducts by itself from predefined parameters like privateInterface and the ones defined by the user from the installFlag (or rather even a new separate kubeletExtraArgs flag to make the merging easier?)
  2. k0s worker & k0s controller would accept multiple --kubelet-extra-args flags and add them all as a parameter to the kubelet binary (seems like the better option in my simple eyes. in which case this issue probably rather belongs in to the k0s repository but as it occurred to me with k0sctl i thought to post it here first)

an-tex avatar Apr 12 '23 15:04 an-tex

I think it would be nice to have a dedicated option for user-specified kubelet-extra-args, so that k0sctl can internally merge them.

pschichtel avatar Jun 09 '23 01:06 pschichtel

K0sctl writing two separate extra args is definitely a bug if that happens.

kke avatar Jun 09 '23 07:06 kke