hcloud-cloud-controller-manager
hcloud-cloud-controller-manager copied to clipboard
Allow specifying IP family priority for dual-stack node ExternalIPs
The order of node's ExternalIP
list published in the node's status is important, as the first one is picked when the apiserver is contacting the kubelet on the node. The current order favours IPv6 address over IPv4, e.g.:
Addresses:
Hostname: qcjg5wgzzc-worker-8z7msv-6b4fbc8c8f-d9lgf
ExternalIP: 2a01:4f8:c012:a19e::1
ExternalIP: 168.119.58.63
InternalIP: 10.0.0.2
This can be problematic in dual-stack scenarios where IPv4 is the primary address family of the cluster - i.e. the apiserver is running on IPv4 only.
In that case, an attempt to kubectl logs
may end up with en error, e.g.:
dial tcp [2a01:4f8:c012:ab3b::1]:10250: connect: cannot assign requested address
Note that this probably won't be an issue once the dual-stack support of the apiserver is enhanced, but we are not yet there (https://github.com/kubernetes/enhancements/issues/2438).
I would see several possible solutions to this:
- (temporarily) switch the order to favour IPv4, as most of the dual-stack k8s clusters probably will be mostly relying on IPv4 control plane before apiserver actually supports dual-stack
- Instead / in addition to the
dualstack
value ofHCLOUD_INSTANCES_ADDRESS_FAMILY
env var, allow passing comma-separated list of address families, e.g.ipv4,ipv6
/ipv6,ipv4
and respect the order when publishing node IPs. - Introduce a new env var, that would specify the primary address family / address family order for dual-stack, e.g.
HCLOUD_INSTANCES_ADDRESS_FAMILY_ORDER
and respect the order when publishing node IPs.
For the reference, the cloud-provider-vsphere takes a similar approach (e.g. ip-family = ipv6,ipv4
- link)