aws-load-balancer-controller
aws-load-balancer-controller copied to clipboard
kubernetes-sigs
Use an existing ALB
As a user of Terraform (or, substitute CloudFormation), I would like to use an existing ALB with the ingress controller so that I can keep my infrastructure automation centralized rather than in several different places. This also externalizes the various current and future associations between ALB and other parts of the infrastructure that may already be defined in TF/CFN (certs, Route53, WAF, CloudFront, other config).
@countergram Thanks, we've heard this request and similar a few times now.
Seems a feature that many would like is an ability to explicitly call our a named ALB via annotation (or eventually configmap).
Hey, I might have solved your problem in this PR: https://github.com/kubernetes-sigs/aws-alb-ingress-controller/pull/830 . Testing and feedback is welcome :)
Relevant: https://github.com/kubernetes-sigs/aws-alb-ingress-controller/issues/914
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale
I was able to use one ALB for multiple Ingress Resources in Version 1.0.1. I did create a new cluster and installed Version 1.1.2, which is creating a new ALB for each Ingress Resource. Is there anyway that I can use same ALB in 1.1.2 ?
^ cross-posted at https://github.com/kubernetes-sigs/aws-alb-ingress-controller/issues/984#issue-475466050, https://github.com/kubernetes-sigs/aws-alb-ingress-controller/issues/724#issuecomment-517113838
Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten
Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /close
@fejta-bot: Closing this issue.
In response to this:
Rotten issues close after 30d of inactivity. Reopen the issue with
/reopen. Mark the issue as fresh with/remove-lifecycle rotten.Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /close
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@leoskyrocker: You can't reopen an issue/PR unless you authored it or you are a collaborator.
In response to this:
/reopen
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@M00nF1sh @joshrosso Any update here? I think this issue is still relevant today and yet unsolved.
Having a way for Kubernetes to attach to existing ALBs/Target Groups is incredibly valuable for lots of reasons. Really surprised there's no way to do it right now.
@gkrizek
we'll address this issue in V2.
For attach to existing TargetGroups, we'll expose an CRD called endpointBinding to allow do that.
For attach to existing ALB, we haven't decided whether to use an annotation on Ingress(like alb-arn: xxxx) or an AWS tag on the ALB(like ownership: shared, ingress: ingress-name, cluster: cluster-name). any opinions?
@M00nF1sh Great to hear! I hate to be "that guy" but is there a v2 anticipated release date?
I like the CRD idea for target groups, I think that's the right direction. I think both options are valid for the ALB, however I think tags are preferred. Because with an ARN you might have to do some wacky stuff to get the ARN into a manifest/helm chart. With tags it'd be pretty easy to define without needed explicit values from AWS. Also it would allow for an ingress to attach to multiple ALBs if one chooses.
@gkrizek There is no anticipated released date yet( i cannot promise one), but I'll keep update https://github.com/kubernetes-sigs/aws-alb-ingress-controller/projects/1 whenever i got time to work on it 🤣. BTW, there is an alpha version of V2 which works just fine: https://github.com/kubernetes-sigs/aws-alb-ingress-controller/releases/tag/v1.2.0-alpha.1 (you can reuse an ALB by apply correct tags, however, the controller will try to delete the ALB once we delete the ingress)
having a single ingress: tag forces a many:1 relationship of ingresses to ALBs which contradicts the concept of ingress grouping mentioned in other issues. It'd be great if the design could allow many ALBs to be reused by many ingresses. Perhaps a "binding" CRD similar to the endpoint/targetgroup solution mentioned above?
@M00nF1sh I figured so 😉 . Sounds good, I'll check out the alpha for now. Thanks for the help.
@rifelpet
it's actually an ingress.k8s.aws/stack: <value> annotation on ALB in V2, where the
(However, personally i favor to require an explicit annotation of ..../alb-arn:xxxx on one-of-ingresses among group to denote the reuse, since tagging on ALB requires to plan for Ingress before hand)
What do you mean by allow many ALBs to be reused by many ingresses.? Current design is one group will only have one ALB.
It's possible to extend it to be like one group with multiple ALB(like auto-split rules), but is there really a use case for this? since i assume there are app-specific dependencies like some Ingress must be hosted by a single DNS name, so it's impossible for the controller to make the split decision if rule exceeds ALB's limits, instead it's better for the user to split there ingresses into different groups.
ok, my main concern was supporting a 1-1 relation between a group and ALB and it sounds like the tags can achieve that 👍
Hi @M00nF1sh keen to start testing with the v1.2.0-alpha.1 version - can you point me to what tags are needed in order to reuse an existing ALB?
BTW, there is an alpha version of V2 which works just fine: https://github.com/kubernetes-sigs/aws-alb-ingress-controller/releases/tag/v1.2.0-alpha.1 (you can reuse an ALB by apply correct tags, however, the controller will try to delete the ALB once we delete the ingress)
/reopen This is something in our roadmap, we'll do designs around this and have a better ETA after v2.2.0 is released. in v2.2.0, we'll add support for tagging listeners and listener rules, which helps to implement this feature.
@M00nF1sh now that v2.2.0 has been released, can you provide any updates on this feature?
Hi, can anyone please clarify a bit if my understanding is correct - Currently we have to create an ALB, listeners and target groups externally, e.g. Terraform, and then within the cluster, reference the target group arn in a target group binding right? The listener rules we don't have to create externally because we would manage it using ingress right? I appreciate any info on this subject because these links I found in a similar issue aren't working: https://kubernetes-sigs.github.io/aws-load-balancer-controller/guide/targetgroupbinding/targetgroupbinding/
It is possible to create only the ALB with terraform and then create the target groups and listeners with the aws-load-balancer-controller, but it isn't yet officially supported. I believe there is planned support for this in v2.3. Basically the aws-load-balancer-controller will automatically update any ALB which has the appropriate AWS tags. So if you set the tags on your ALB in Terraform then the aws-load-balancer-controller will take ownership of updating it.
I believe the necessary tags are:
"ingress.k8s.aws/stack" = "The value specified in the alb.ingress.kubernetes.io/group.name annotation"
"ingress.k8s.aws/resource" = "LoadBalancer"
"elbv2.k8s.aws/cluster" = "The name of your k8s cluster as specified in the aws-load-balancer-controller config"
You can validate if the tags I mentioned above are correct by configuring your aws-load-balancer-controller instance to provision an ALB then inspect the AWS tags on the ALB after it has been provisioned.
@jwenz723 this is actually not working for me. If I set the tags you mentioned, my alb gets deleted, and then a new ALB is being created by the controller.
"@christophebeling I didn't observe this behaviour. @jwenz723 To successfully manage it I need to use the target group binding as well as an ingress describing the rules right?"
To fill in the details, using the tags on the ALB,created by Terraform, didn't work for me. First I created the ALB and a target group with a listener using terraform and then, after the controller installation, tried to use the targetgroupbinding without an ingress created and then the target group doesn't see the instances anymore. Using an ingress just creates another load balancer and ignores the first.
I also tried to create just the ALB without any target groups and listeners and then used an ingress. This replaces the ALB created by terraform with the one created by the controller.
Sorry for the confusing comment in the beginning.
Update: I've managed to get the ALB, created by terraform, to work with the load balancer controller. First step is to create the ALB along with a target group and a listener. Second is to deploy the controller and a targetgroup binding in which the target group arn is referenced. The only thing is I don't think any ingress is working in this case and everything should be defined in terraform, but I need to experiment further on that.