Incorrect IP range set by LB controller for internal NLB

[tanvp112]

Describe the bug

The IP range added by AWS LB Controller for the internal NLB does not cover the provisioned internal LB's IP. As a result, connection from the internal NLB to the registered target on EKS cluster is blocked.

Steps to reproduce

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    run: nginx
spec:
  containers:
  - name: nginx
    image: nginx:alpine
    ports:
    - name: http
      containerPort: 80
---
  apiVersion: v1
  kind: Service
  metadata:
    name: nginx-lb
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-type: nlb-ip
  spec:
    type: LoadBalancer
    selector:
      run: nginx
    ports:
    - name: http
      port: 8080
      targetPort: http

The internal NLB was provisioned successfully:

Target registered successfully:

Now, exec into a pod and resolve the internal NLB DNS name result to:

Noticed that the resolved internal NLB IP is NOT in the IP range added by LB controller:

10.236.32.0/19 range is 10.236.32.1 to 10.236.63.254 that does not cover the internal NLB IP (10.236.67.52), this means the internal NLB will not able to reach the registered target IP 10.236.84.139 as the security group will block it, right? As a result, curl with the internal NLB DNS has failed but when curl directly to the target IP succeeded.

Expected outcome

The IP range added by AWS LB controller needs to ensure it covers the internal NLB IP.

• AWS Load Balancer controller version

2.4.2

• Using EKS (yes/no), if so version?

1.22

[kishorj]

@tanvp112, in case of internal NLB with IP targets, the controller adds SG rules for each of of the load balancer subnets CIDRs. Since the target is healthy, the appropriate rules must be in place.

Could you verify the NLB address is assigned from one of the subnets in the NLB AZ configuration?

[tanvp112]

Yes, the NLB IP is assigned from one of the 3 subnets which has kubernetes.io/role/internal-elb tagged. Does cross-zone load balancing matters here?

[kishorj]

If cross-zone load balancing is disabled, the DNS records corresponding to the AZs without any targets get removed from the NLB. Could you verify whether cross zone load balancing is disabled in your case?