Load Balancer Annotations & AWS NLB Additional Steps

[RickFoland]

1. Maybe I missed this, but currently there is no way to add annotations to the load balancer template through a values file. These annotations are necessary for changing the load balancer type, attaching Elastic IPs, etc. It wasn’t a big add and I can submit a pull request with my changes if this wasn’t meant to be done elsewhere.

2. This may not be supported yet, but creating an NLB instead of a classic LB in AWS still requires a couple of manual steps.

• It doesn’t automatically register the nodes into the target group. Although, once the nodes manually added additional nodes added to the ASG are automatically added to the target group. I did try using annotations like service.beta.kubernetes.io/aws-load-balancer-target-node-labels without any luck.
• It does create additional security group rules for the listeners, but is still missing an inbound rule that needs to be added manually. I’m not sure what this rule is for yet, but I only got communication between the LB and nodes to work after allowing inbound traffic from my VPC CIDR on ports 30000-50000 (in the same cluster SG that the NLB created its other rules in).

I'm currently working on a project with this, so I can help with trying things out, testing, and submitting PRs with changes that work.

[Embraser01]

Hi,

For 1), you're right, we need to add a lot more templating options to our values file (#109 is almost about the same thing). Feel free to submit a PR, I'll try to get to it asap.

For 2), I'm not sure on what we could do here, as it's mainly AWS things for any LB setup. I did succeeded at running a NLB with everything working (w/ proxy protcol, direct pod traffic, ...). I'd like to add a section in the documentation on how to setup caddy ingress controller on AWS/GCP/Azure (once we actually have a website 😓, still need to work on it)

[RickFoland]

2. So I have the NLB working, but needed to add a couple of manual steps. I may have just missed something in my setup that would tell AWS to create the NLB properly. Do you have a copy of the annotations you used for that NLB setup? Did you have to do anything with the AWS LoadBalancer Controller?