hcloud-cloud-controller-manager icon indicating copy to clipboard operation
hcloud-cloud-controller-manager copied to clipboard

Allow specifying IP family priority for dual-stack node ExternalIPs

Open rastislavs opened this issue 1 year ago • 0 comments

The order of node's ExternalIP list published in the node's status is important, as the first one is picked when the apiserver is contacting the kubelet on the node. The current order favours IPv6 address over IPv4, e.g.:

Addresses:
  Hostname:    qcjg5wgzzc-worker-8z7msv-6b4fbc8c8f-d9lgf
  ExternalIP:  2a01:4f8:c012:a19e::1
  ExternalIP:  168.119.58.63
  InternalIP:  10.0.0.2

This can be problematic in dual-stack scenarios where IPv4 is the primary address family of the cluster - i.e. the apiserver is running on IPv4 only.

In that case, an attempt to kubectl logs may end up with en error, e.g.:

dial tcp [2a01:4f8:c012:ab3b::1]:10250: connect: cannot assign requested address

Note that this probably won't be an issue once the dual-stack support of the apiserver is enhanced, but we are not yet there (https://github.com/kubernetes/enhancements/issues/2438).

I would see several possible solutions to this:

  1. (temporarily) switch the order to favour IPv4, as most of the dual-stack k8s clusters probably will be mostly relying on IPv4 control plane before apiserver actually supports dual-stack
  2. Instead / in addition to the dualstack value of HCLOUD_INSTANCES_ADDRESS_FAMILY env var, allow passing comma-separated list of address families, e.g. ipv4,ipv6 / ipv6,ipv4 and respect the order when publishing node IPs.
  3. Introduce a new env var, that would specify the primary address family / address family order for dual-stack, e.g. HCLOUD_INSTANCES_ADDRESS_FAMILY_ORDER and respect the order when publishing node IPs.

For the reference, the cloud-provider-vsphere takes a similar approach (e.g. ip-family = ipv6,ipv4 - link)

rastislavs avatar Aug 03 '22 13:08 rastislavs