[EKS] [request]: Allow pods with the same set of SGs attached to use one branch network interface

[georgio-sd]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request We use security groups for pods. Currently each pod with security groups attached gets its own branch network interface. Since the number of interfaces per instance is limited, we are not able to launch as many pods as it is possible without using security groups.

For example, c5.12xlarge instance can have only 54 branch network interfaces. Normally, the c5.12xlarge instance type can have 234 pods scheduled on it, assuming they aren't associated with a security group.

The idea is to use one branch network interface for a few pods with the same set of security groups.

Which service(s) is this request for? EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? This will allow to utilize nodes more effectively and save our money.

Are you currently working around this issue? Launching more nodes.

[jose-ledesma]

I would also like to add that creating a branch ENI is a heavy operation. We have been benchmarking Security Group for Pods and, while using this feature, is not possible to create more than 1.5 pods per second, so if we could share the same branch ENI for several pods sharing the same security group we could overcome this performance limitation.

[chlunde]

This would also reduce AWS Config costs, which can surpass EC2 costs when using ENIs and many small microservices. There are updates to the VPC, ENI and subnet objects so each pod start costs 3*$0.003 in AWS Config