ELB ACM cert annotation not working

[haghabozorgi]

We are testing traefik as a replacement for ingress-nginx controller. We want to deploy traefik behind ELB and terminate TLS at the ELB. In nginx land you would add something like the below

    external-dns.alpha.kubernetes.io/hostname: "traefik.example.com"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "$AcmCertArn"
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
    service.beta.kubernetes.io/aws-load-balancer-connection-draining-enabled: "true"
    service.beta.kubernetes.io/aws-load-balancer-connection-draining-timeout: "60"

and this would create an ELB with two listeners, 80 (HTTP) and 443 (HTTPS), and the 443 listener would have the Certificate setup. We tried this on traefik and we get the listeners but they are setup as 80 (HTTP) and 443 (HTTP) and the Certificate is not setup on the 443 listener. After trying to troubleshoot this, we tried renaming web and websecure to http and https respectively. This seems to fix the ELB setup but crashes the container. The ELB gets setup as we expect with 80 (HTTP) and 443 (HTTPS), and the 443 listener has the Certificate setup. but then the pod crashes with

time="2021-06-18T05:03:53Z" level=info msg="Configuration loaded from flags."
2021/06/18 05:03:53 traefik.go:76: command traefik error: error while building entryPoint https: error preparing server: error opening listener: listen tcp :8443: bind: address already in use
time="2021-06-18T05:04:16Z" level=info msg="Configuration loaded from flags."
2021/06/18 05:04:16 traefik.go:76: command traefik error: error while building entryPoint websecure: error preparing server: error opening listener: listen tcp :8443: bind: address already in use
time="2021-06-18T05:05:10Z" level=info msg="Configuration loaded from flags."
2021/06/18 05:05:10 traefik.go:76: command traefik error: error while building entryPoint websecure: error preparing server: error opening listener: listen tcp :8443: bind: address already in use

After some further troubleshooting we ended up setting up the ports section of values.yaml like this, where we disable exposing web and websecure and change their default ports so we can use those for http and https and expose these ports, and this seems to do what we want but feels kind of janky.

ports:
  http:
    port: 8000
    expose: true
    exposedPort: 80
    protocol: TCP
    redirectTo: https
  web:
    port: 8001
    expose: false
    exposedPort: 80
    protocol: TCP
  https:
    port: 8443
    expose: true
    exposedPort: 443
    protocol: TCP
  websecure:
    port: 8444
    expose: false
    exposedPort: 443
    protocol: TCP

I am not sure if this is related to https://github.com/traefik/traefik-helm-chart/issues/172, or if we are missing something basic on our end. But any pointers would be appreciated. Ideally we just want a 80 and 443 with redirect 80 to 443 set.

[faust64]

Hi,

Shouldn't your service.beta.kubernetes.io/aws-load-balancer-ssl-ports annotation point to websecure instead of https -- assuming default ports value?

Checking https://github.com/traefik/traefik-helm-chart/issues/172#issuecomment-653868930 , sounds like when the requested port name is not found, then the first port from the list is picked - which could explain the unexpected http termination?!

I'm not really familiar with Kubernetes AWS integration, though their docs suggests this annotation should match a valid name in your Services ports: https://aws.amazon.com/premiumsupport/knowledge-center/terminate-https-traffic-eks-acm/

Or is it something you tried already?

Setting http and https ports in your values, while reusing 8443 and 8080 would indeed result in those address already in use errors. Your values file is merged with the defaults from Chart. The resulting ports array would include both web/websecure from defaults, and http/https from your configured values. And you figured it out: re-defining web/websecure in your values, with different ports, would prevent this - although I'm not convinced setting expose to false helps: it would be used picking ports to expose in the Service object, unrelated with container configuration. ... And on that topic: if your ELB rewrites http to https, then you may not need to expose the http endpoint.

Let us know if changing that annotation helps getting it right with the default ports from Charts. You're right it's most likely the same issue as described in #172 .

[albertschwarzkopf]

Thanks @faust64 in my case this has solved the same issue!