kube-router icon indicating copy to clipboard operation
kube-router copied to clipboard

Adds basic support for advertising both v4 and v6 VIPs in a dual-stack cluster

Open sanjmonkey opened this issue 4 years ago • 2 comments

A service can have multiple address from both v4 and v6 in a dual-stack cluster. Prior to this PR, only single-stack was really considered, and further, all VIP prefixes were masked with /32 which is not the intention for single-stack v6 clusters.

This PR lets you to advertise dual-stack service VIPs to upstream neighbours, and correctly masks the v6 VIPs in a single-stack v6 cluster.

In this implementation, for a node to be considered dual-stack it should have addresses in both address families in either NodeInternalIP or NodeExternalIP. The first of each of these addresses are chosen as the NH in the update to the upstream peer. You can do this by starting kubelet with --node-ip=<node_v4_address>,<node_v6_address>

Upstream neighbours dual-stack sessions can be initiated by annotating the node like this:

'kube-router.io/peer.ips=<neigh_v4_address>,<neigh_v6_address>'
'kube-router.io/bgp-local-addresses=<node_v4_address>,<node_v6_address>'
'kube-router.io/peer.asns=<upstream_asn>,<upstream_asn>'

sanjmonkey avatar Jul 02 '21 09:07 sanjmonkey

@aauren / @murali-reddy - Is this simple PR still useful to anyone, if so I'll rebase, if not feel free to close. I know its not much of a contribution in the feat that remains to support dual-stack everywhere else in KR, but likewise its not breaking and may help others get off the ground.

Note; We use this branch in production where KR does v4/v6 VIP advertisement only, and cilium for CNI / everything else.

sanjmonkey avatar Feb 22 '22 16:02 sanjmonkey

@sanjmonkey Thanks for submitting this and sorry for the late response! I think that you're right, that this doesn't hurt anything and it makes the IPv6 support a little more robust along the way.

Can you let me know how much testing have you done with these changes so far? Preferably, this patch would be tested on more than just a dev cluster and shown that it can handle the mileage.

Even if with just a dev cluster I think that we could accept this PR if you're willing to rebase and add a few things:

  • Add unit testing around the utility methods you added
  • Add unit testing around the v6 additions to the VIP and policy changes (we have fairly robust unit tests for this stuff, and it would be good to get this stuff tested so that as we make more v6 additions we can ensure we don't regress functionality)
  • Add documentation to the ipv6 documentation stating how to use this new functionality (basically the annotation bit you have there) as well as any limitations with this functionality in regards to the network_routes_controller.

I know that's a lot, so let me know if you're not up for it. However, I'm trying to keep active here again, so I should be able to give you a more timely review if you add those items.

aauren avatar Mar 12 '22 03:03 aauren

Replaced by #1386

aauren avatar Oct 31 '22 04:10 aauren