Unable to create an ALB in a different VPC than the one where EKS exists.

[bagel-dawg]

Describe the bug I am using a single ALB ingress controller to deploy two ALBs (1 internet-facing, 1 internal) via ingress annotations. I am able to deploy the internal ALB without issue, it is deployed in the same VPC as EKS.

The internet-facing ALB is deployed into subnets that are shared with the account, and thus are in a different VPC than the VPC that EKS is in.

When deploying I receive this error: InvalidConfigurationRequest: Security group 'sg-XXX' does not belong to VPC '<shared-subnet-vpc>.

When I dig into that security group, I see that the security group is deployed in the EKS VPC instead of the shared VPC that I expected.

Steps to reproduce Ingress YAML:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig":{ "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
    alb.ingress.kubernetes.io/backend-protocol: HTTPS
    alb.ingress.kubernetes.io/healthcheck-path: /ping
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/subnets: subnet-shared-vpc-1, subnet-shared-vpc-2, subnet-shared-vpc-3
    alb.ingress.kubernetes.io/target-type: ip
    kubernetes.io/ingress.class: alb
spec:
  rules:
  - http:
      paths:
      - backend:
          service:
            name: ssl-redirect
            port:
              name: use-annotation
        path: /*
        pathType: ImplementationSpecific
      - backend:
          service:
            name: traefik-external
            port:
              number: 443
        path: /*
        pathType: ImplementationSpecific

Expected outcome ALB Ingress Controller should be able to create an ALB in subnets that are apart of a VPC that EKS does not belong to. This use case I believe will become more common with Transit gateway. Our use case specifically involves a centralized ingress VPC for public-facing ALBs in a separate central networking account.

Environment

• AWS Load Balancer controller version v2.4.3
• Kubernetes version

Server Version: version.Info{Major:"1", Minor:"20+", GitVersion:"v1.20.15-eks-18ef993", GitCommit:"77b5697130c2dea4087e1009638e21cc93f5c5b6", GitTreeState:"clean", BuildDate:"2022-07-06T18:04:29Z", GoVersion:"go1.15.15", Compiler:"gc", Platform:"linux/amd64"

Additional Context: Before adding the alb.ingress.kubernetes.io/subnets annotation, the controller reported that it wasn't able to auto-discover subnets. I assume this is because the ALB controller filters out subnets that belong to non-EKS VPCs?

If I specify --aws-vpc-id <shared-vpc>, this break the internal ALB deployment. I assume because of the same issue, just in reverse.

[bagel-dawg]

I see this comment that having multiple VPCs is not supported. Is this still true in 2022?

[kishorj]

@bagel-dawg, we still do not support multiple VPC. You can configure the controller to a specific vpc than the default, but as you mentioned, it will be limited to that particular VPC only.

[kishorj]

Closing this issue in favor of feature request #2652