aws
eks with secondary network for pods
Following the documentation, we end up in the following scenario. https://aws.github.io/aws-eks-best-practices/networking/custom-networking/
The pod receives the IPs from the secondary subnet and to try to access resources outside the VPC, the main IP of the node in the main network is used, but for resources in the same VPC it tries to use the secondary IP that the pod receives. Is there a way to mask the pod IP to the main IP for internal resources in the same VPC?
pod ip: 100.127.35.58 node ip: 192.168.1.44
when trying to access an RDS resource in the same vpc the pod uses the ip 100.127.35.58
to access a resource outside the vpc a masking is done so that the pod uses the node ip 192.168.1.44
We would like this masking to also be done for internal resources in the same VPC
Is there a way to mask the pod IP to the main IP for internal resources in the same VPC?
No, there is not a way to mask the pod ip to the main ip (or node ip) of internal resources in the same VPC.
If you use the hostnetworking pod, then the ip of the pod is the same as the node's primary ip.
do we not have the possibility to create a function for this? "AWS_VPC_K8S_CNI_INTERNALSNAT=true"
Do you mean, create a new flag to have an Internal SNAT? I am not sure how that will work with the property of hostnetwork pods.
the intention is to make the pods receive the ips of the secondary subnet as expected, but to communicate internally use the ip of the main subnet that is also tied to the node
for example, the pods receive the range 100.127.xxx.xxx and intermanete they use this ip 100.127.xxx.xxx to communicate with an RDS.
the idea is to use the primary ip of the node: 10.72.xxx.xxx (this occurs for when the resource is outside the vpc)
in GCP and azure there is this way of performing this configuration
https://cloud.google.com/kubernetes-engine/docs/how-to/ip-masquerade-agent
https://github.com/Azure/ip-masq-agent-v2
@davirezendegb - can you raise this feature request containers-roadmap. It needs to be evaluated and prioritized as a product requirement for CNI.
Hi this is a blocker for us too, we want to use a seperate subnet for pods whilst maintaining routing for nodes via internalsnat
This was very difficult to debug as the documentation was lacking. This is also a feature GCP has documented well for GKE (via ipmasqagent) - its surprising to not see this in EKS.
With https://github.com/aws/containers-roadmap/issues/1342 - you could effectively also achieve this. Either implementation would be very welcome to reduce the cost of a secure configuration.
This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Any update on this?
Same scenario.
The VPC used by the EKS cluster with VPC CNI Custom Networking enabled and the RDS is the same but different subnets.
I know that the default behavior expected as per official documentation for the SNAT implementation.
