[Feature] Allow custom load balancer (or "bring your own") load balancer

[aslatter]

Is your feature request related to a problem? Please describe. We are interested in establishing private connectivity to our AKS-hosted services to third-parties in separate Azure tenants.

We would like to do this with a Private Link Service. A Private Link Service is an Azure resource we can place in front of an Azure load-balancer to make the application behind the load-balancer privately accessible.

We can do this with the LBs provisioned as a part of AKS, however these load-balancers get deleted and recreated whenever we need to re-create the AKS cluster. New AKS features often require provisioning a new AKS cluster, and we sometimes provision a fresh control-plane to resolve production issues.

Whenever we re-create the AKS cluster we would be forced to re-create the private-link-service as well, which would force the third-parties to request access to the new private-link-service. This would require a service disruption and potential network-reconfiguration on the side of the third-party.

Describe the solution you'd like We would like the ability to link a Kubernetes Service object to an existing Load Balancer back-end pool which we managed.

EKS offers a similar feature with a CRD tying the service to an LB: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.6/guide/targetgroupbinding/targetgroupbinding/

Describe alternatives you've considered

• We could introduce a load balancer in front of the kubernetes-managed load balancer, but our understanding is that we would need some VM-hosted appliance to route traffic between the LBs. We're not interesting in managing things like OS-patches and OS-upgrades ourselves, and making this appliance highly available would be complex.

• We could front our services with a separate Azure VNet and ask the third parties to peer to that, and place the private-link services in the vnet we managed. That way, we would be controlling both ends of the private-link and could manage failing over to a second AKS control-plane without service disruptions (with some DNS manipulation). This solution is more complex, and requires pushing additional DNS-management onto the third-parties - if the other party were managing the private-endpoint themselves they could take advantage of Azure Private Zone integrations.

Additional context

[aslatter]

Additional context:

Our AKS clusters are typically deployed with two Kubernetes Services of type load-balancer, each corresponding to a different public IP address with different IP-allow-lists on them, mapped to different Kubernetes ingress controllers.

So having this configuration be per Kubernetes Service (and not a global AKS-level setting) is important to us.

[pawelpabich]

Another vote for this issue or another way of decoupling Private Service Link (PSL) from the AKS load balancer.

We also upgrade AKS clusters by re-creating them and we want to do that without affecting our customers.

💭worth exploring:

1. Add an ability to attach PSL to a load balancer with an IP backend which then can be chained with an existing AKS load balancer
2. Add an ability to switch PSL between load balancers. (preferred)

[chasewilson]

Eyes on this, thank you for the input here. We'll do some investigating here and update shortly within the next week.

[chasewilson]

This isn't something we can currently do in our roadmap due to how to SLB and PLS work. Going to keep this open to keep an eye out for solutions and potential features we could do.

Please keep reactions and comments to highlight it. I've put this in our AKS Roadmap as a backlog item and the interactions do help us prioritize work and let us know of common pain points.

Thank you for keeping this up and hope we can work towards a solid solution.

[pawelpabich]

Thanks @chasewilson for an update. Out of curiosity, are any of the ideas mentioned in this issue more likely to be implemented than others?

[cmeury]

We're still interested in this.

[aslatter]

We can probably use the upcoming Private Link service Direct Connect to meet our above need.