Incompatibility with non-EKS clusters.

[is-it-ayush]

Describe the bug I was trying to expose my services to the internet on a self-managed kubeadm cluster running on AWS EC2 in a secure way. The only way to expose cluster services to the outside internet is to provision an ALB or NLB from AWS. However, I noticed two problems in my experiments while trying to provision load balancer's with aws-load-balancer-controller.

1. ALB provisioning fails entirely on non-EKS clusters when using TLS spec in the Ingress spec. The issue here is that Certificate is issued by LetsEncrypt & managed by cert-manager while aws-load-balancer-controller expects the Certificate to be present on AWS ACM.
2. NLB TargerGroup provisioning would fail on non-EKS clusters because nodes do not contain providerID & therefore aws-load-balancer-controller cannot add instances in public subnets (which it auto-discovers via tags) to the TargetGroup on which the traffic is to be redirected from the NLB.
   • NLB get's created but suffers from TLS problem described in 1.)

Steps to reproduce

1. Setup a non-EKS cluster through any k8s distro on AWS EC2. I used kubeadm.
2. Install aws-load-balancer-controller by following the install instructions on the docs!
3. For NLB: Try to expose any service through the aws-load-balancer-controller such that it would provison an NLB in the cloud.
   • NLB get's created but fails to work when service is annotated with TLS annotations from aws-load-balancer-controller.
   • TargetGroup is empty.
4. For ALB: Try to expose an Ingress that redirects a service to a path. Ensure your service has loadBalancerClass set to anything other than service.k8s.aws/nlb such as loadBalancerClass: "none" to prevent aws-load-balancer-controller from provisioning an NLB instead of an ALB.
   • ALB isn't created in cloud when TLS spec in the Ingress spec.
   • TargetGroup is empty.

Expected outcome

• TargetGroup should not be empty. aws-load-balancer-controller should attempt to auto discover instances and annotate them with providerID if it's required in non-EKS cluster. If the providerID is already present, then it should do nothing. This should be achievable by using the Private IP for the node.
• aws-load-balancer-controller should use the TLS certificate from cert-manager or upload the certificate from cert-manager to AWS ACM if it really wants to depend on AWS ACM.

Without above, securing internet exposed services running on AWS cloud is impossible on non-EKS clusters and therefore prevents the usage of ALB & NLB for production making non-EKS clusters impossible to automate within AWS cloud.

Environment

• AWS Load Balancer controller version: v2.7.2
• Kubernetes version: v1.30
• Using EKS (yes/no), if so version? no

Additional Context:

1. I've been through cert-manager/issues/333 and it seems like the issue was resolved for certificates issued by AWS ACM aka private certificates with aws-privateca-issuer. My use case is different as I'm seeking a way to upload the certificates issued by LetsEncrypt to AWS ACM and then tag my Ingress/Service resource with the Certificate ARN returned from the ImportCertificate API call.
2. I've been through aws-load-balancer-controller/issues/3178 and aws-load-balancer-controller attemtps to auto discover certificate on ACM based on hostname in TLS. If 1. is resolved, this should be resolved to.
3. I've been through aws-controllers-k8s/community/issues/482 and it seems the discussion for uploading certificates issued by other CA's was left unfinished.