Multiple Protocols Not Supported (TCP+UDP)

[dhawal1304]

Describe the bug If i specify multiple protocols for a AWS LB, aws load balancr controller gives error.

Steps to reproduce

apiVersion: v1 kind: Service metadata: name: vvr-lb # the name of our service namespace: infoscale-vtas annotations: service.beta.kubernetes.io/aws-load-balancer-internal: "true" service.beta.kubernetes.io/aws-load-balancer-type: "nlb" # service.beta.kubernetes.io/aws-load-balancer-target-node-labels: "app1=master" spec: externalTrafficPolicy: Cluster internalTrafficPolicy: Cluster type: LoadBalancer ports:

• name: tcpportone port: 4145 protocol: TCP targetPort: 4145
• name: udpportone port: 4145 protocol: UDP targetPort: 4145
• name: tcpporttwo port: 8199 protocol: TCP targetPort: 8199
• name: tcpportthree port: 8989 protocol: TCP targetPort: 8989 selector: app1: infomas

Expected outcome AWS LB should create a LB with mixed protcol .

Environment

• AWS Load Balancer controller version v2.4.1
• Kubernetes version v1.24.3
• Using EKS (yes/no), if so version? No

Additional Context:

Getting the below error while creating LB

[root@ip-10-248-176-137 21Arpil]# I0809 06:57:47.409828 1 controller.go:403] Ensuring load balancer for service infoscale-vtas/vvr-lb I0809 06:57:47.409891 1 aws.go:4163] EnsureLoadBalancer(kubernetes, infoscale-vtas, vvr-lb, us-east-1, , [{tcpportone TCP 4145 {0 4145 } 31684} {tcpporttwo TCP 8199 {0 8199 } 30591} {tcpportthree TCP 8989 {0 8989 } 30453} {udpportthree UDP 4145 {0 4145 } 31684}], map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{"service.beta.kubernetes.io/aws-load-balancer-internal":"true","service.beta.kubernetes.io/aws-load-balancer-type":"nlb"},"name":"vvr-lb","namespace":"infoscale-vtas"},"spec":{"ports":[{"name":"tcpportone","port":4145,"protocol":"TCP","targetPort":4145},{"name":"tcpporttwo","port":8199,"protocol":"TCP","targetPort":8199},{"name":"tcpportthree","port":8989,"protocol":"TCP","targetPort":8989},{"name":"udpportthree","port":4145,"protocol":"UDP","targetPort":4145}],"selector":{"app1":"infomas"},"type":"LoadBalancer"}} service.beta.kubernetes.io/aws-load-balancer-internal:true service.beta.kubernetes.io/aws-load-balancer-type:nlb]) E0809 06:57:47.409939 1 controller.go:310] error processing service infoscale-vtas/vvr-lb (will retry): failed to ensure load balancer: mixed protocol is not supported for LoadBalancer I0809 06:57:47.409986 1 event.go:294] "Event occurred" object="infoscale-vtas/vvr-lb" kind="Service" apiVersion="v1" type="Normal" reason="EnsuringLoadBalancer" message="Ensuring load balancer" I0809 06:57:47.410007 1 event.go:294] "Event occurred" object="infoscale-vtas/vvr-lb" kind="Service" apiVersion="v1" type="Warning" reason="SyncLoadBalancerFailed" message="Error syncing load balancer: failed to ensure load balancer: mixed protocol is not supported for LoadBalancer"

[dhawal1304]

Also , please let know if there is a workaround for this.

[kishorj]

Since you've used the annotation service.beta.kubernetes.io/aws-load-balancer-type: "nlb", the load balancer is provisioned by the kubernetes in-tree/cloud-provider controller where mixed protocol support is not available. This controller supports mixed protocols, you'd have to create a service with the following annotation instead:

    service.beta.kubernetes.io/aws-load-balancer-type: "external"

You could also set service spec.loadBalancerClass to service.k8s.aws/nlb instead of the annotation.

While you can mix TCP and UDP ports, but the TCP and UDP ports need to be distinct due to AWS NLB side limitations.

[dhawal1304]

While you can mix TCP and UDP ports, but the TCP and UDP ports need to be distinct due to AWS NLB side limitations : Does this mean that aws load balancer controller cannot provision tcp and udp on the same port.

We have a requirement of tcp and udp listening on the same port which is 4145. Is this possible by any means.

[kishorj]

LB controller currently cannot provision TCP and UDP listener using the same port for an NLB, feature request #1608 is in our roadmap.

That being said, you can see if the following steps helps your case:

• create a k8s service of type ClusterIP for your application
• Provision an NLB from the AWS console, create a Listener of type TCP_UDP for the port you want to expose and create a new target group to forward the traffic to, note down the target group ARN
• create a TargetGroupBinding resource that binds the ClusterIP service from above to your NLB target group

[dhawal1304]

@kishorj Thanks for the feedback. Used "You could also set service spec.loadBalancerClass to service.k8s.aws/nlb instead of the annotation." as suggested by you, and was able to create mixed protocols(Not on the same port).

Would like your views on when we are planning to support mix protocol(TCP_UDP), on the same port ,as mentioned by you #1608

[kishorj]

@dhawal1304, the feature #1608 is in our roadmap. There is also a PR for the same. We will include it in future release. I'm closing the issue for now, if you have further concerns, feel free to reach out to us.

[dhawal1304]

Thanks @kishorj.