Docker-ECS load balancer, HTTPS and routing

[MartinJ-Dev]

Greetings! This issue has been discussed a lot elsewhere. After reading all the docs and other related issues, we still aren’t sure how to do it. We are hoping to get concrete guidance for setting up our stack which can be conceptualized in the following way

services:
  # UI - single page app using Vue.js. It calls API gateway for any backend services
  frontend:
    image: ${IMG_FRONTEND}
    ports:
      - 80:80
      - 443:443

  # API gateway using Gunicorn. It handles auth and forwards requests to internal services
  api:
    image: ${IMG_API}
    ports:
      - 8000:8000

  # Internal service only accessible from API gateway and other internal services
  svc1:
    image: ${IMG_SVC1}
    expose:
      - 8000

  # Same as svc1
  svc2:
    image: ${IMG_SVC2}
    expose:
      - 8000

How do we set up the compose so that:

• All external traffics secured by HTTPS
• Requests can be correctly routed to frontend and API gateway
• Redirect 80 to 443
• Ideally everything done in compose and no need to mess with CloudFormation

---

While this can be easily achieved using Traefik but we struggle with Docker-ECS. So far we understand:

• No path routing so we can’t route UI request to example.com and API requests to api.example.com
• Port mapping has to be the same

We are particularly confused because the official doc is limited but community resource with examples contain conflictive information.

For example, the doc says

Setting SSL termination by Load Balancer

x-aws-cloudformation:
  Resources:
    WebappTCP80Listener:
      Properties:
        Certificates:
          - CertificateArn: "arn:aws:acm:certificate/123abc"
        Protocol: HTTPS

Looks simple but then the community suggests otherwise.

https://techsparx.com/software-development/docker/docker-ecs/load-balancer/https.html

Slack groups also confirms there no way to avoid going through this.

We understand this project is still in development with its limitation so we are flexible in terms of technical solutions as long as business requirements are satisfied. Any feedback is appreciated.

[MartinJ-Dev]

Related issues: https://github.com/docker/compose-cli/issues/775 https://github.com/docker/compose-cli/issues/777 https://github.com/docker/compose-cli/issues/1426 https://github.com/docker/compose-cli/issues/703 https://github.com/docker/compose-cli/pull/871

[ndeloof]

Hi.

While this can be easily achieved using Traffik but we struggle with Docker-ECS.

This is indeed the main issue to address this: we don't want to provide yet another set of AWS-specific extensions that won't ever be adopted in the compose-spec. But on the other hand, compose model doesn't cover the "routing" aspect, so this need to be defined in a portable way. We also need to offer a transparent support for local development, maybe using a simplistic approach like we do for secrets by just bind mounting plain text files: we could use a "routing container" dynamically added the the compose application.

A viable solution will require

• a high-level, declarative approach in compose model
• embed Træffik (or comparable) on compose up for local development
• map this to AWS resources and infrastructure

In the meantime, there's no simple option with ECS integration but to tweak the CloudFormation template to "fill the gap"

[MartinJ-Dev]

@ndeloof Thanks for getting back. We are happy to adopt any alternatives. Can you elaborate on your viable solution, specifically "a high-level, declarative approach" and "map to AWS"?

Since we can't do path routing, we are considering the following approaches:

Frontend and API in the same compose

• frontend maps ports 80:80 and 443:443
• API gateway maps port 8000:8000
• Users visit frontend at https://example.com
• Configure frontend to call root API at example.com:8000

Frontend and API in different stacks

• Use Docker-ECS as pure backend with the API gateway as a single entry point connected to ALB.
• Host frontend somewhere else (S3 or comparable)
• Local development will have compose up for the backend stack and run npm serve locally

Would something like these work? How would setting HTTPS up look like in any of those?

[ndeloof]

I'm only thinking "middle/long terms". There's no silver-bullet solution to this yet, but to create your own Load balancer and tweak the generated CloudFormation template for your needs.

[MartinJ-Dev]

Understood. Could you suggest a short term solution that fits in our need? We are okay to compromise but hope to get specific and actionable guideline. Would any of the two approaches in the previous comment work? If our needs aren't clear, we can always clarify.

Assuming we can follow some AWS doc to create a load balancer and docker compose convert to generate CFN, how do we connect them and tweak it?

Thanks again!

[MartinJ-Dev]

@ndeloof Any feedback? Today we try to setup HTTPS for load balancer using the example in doc

version: '3.8'

services:
  proxy:
    image: nginx
    ports:
      - "80:80"

x-aws-cloudformation:
  Resources:
    ProxyTCP80Listener:
      Properties:
        Certificates:
          - CertificateArn: "arn:aws:acm:us-west-2:xxxx:certificate/xxxx"
        Protocol: HTTPS

The certificate is a valid one from ACM. We point our domain DNS CNAME to the DNS name of load balancer created by compose xxxx.us-west-2.elb.amazonaws.com. Unfortunately it can't be reached. Please advise.

[ndeloof]

double check you access your service on port 80, which is not the default one for https: https://xxxx.us-west-2.elb.amazonaws.com:80/

[MartinJ-Dev]

https://xxxx.us-west-2.elb.amazonaws.com:80/ gives "Not secure" warning in Chrome. I assumed this is because certificate is tied to a domain name. Following that information, we updated our certificate and DNS CNAME. Now we can successfully access https://www.domain.com:80

While this is encouraging, how do we access this domain on HTTPS without :80? In production, we can't have clients visit us by attaching a port number.

[ndeloof]

we don't yet have an option to expose a distinct port for ingress traffic. You can tweak the cloudformation template so it set Listener external port to 443, or can change your compose file so that port 443 is used for service's HTTP traffic

[al-dpopowich]

Another option which I have used is adding a nginx container, which is the only container with published ports, 80 and 443. Then I use nginx for all internal routing. You can put your cert and private key in secrets and configure your nginx container to use those files mounted at /run/secrets/{certificate,private_key}. Added bonus: your SPA doesn't have to be configured to use a different host/port and just references, say, /api/... on the origin host. Another bonus: there's virtually no difference between my development environment and production environment.

[chingc]

~I've tried the ssl termination example given here, but ran into this error:~

~ValidationError: Template format error: [/Resources/WebappTCP80Listener] Every Resources object must contain a Type member.~

~I've tried Type: AWS::ElasticLoadBalancingV2::Listener and Type: AWS::ElasticLoadBalancingV2::ListenerCertificate but neither seems to be correct. What should be the type be?~

Edit: Sorry, this was user error on my part.

[cinoss]

@chingc replacing Webapp with the name of your service in Pascal case seems to fix the problem for me. Another the thing is the guide (https://docs.docker.com/cloud/ecs-integration/#setting-ssl-termination-by-load-balancer) does not cover the part where 443 port is not opened in security group.

for my case:

x-aws-cloudformation:
  Resources:
    BackendTCP80Listener:
      Properties:
        Certificates:
          - CertificateArn: "arn:aws:acm:xxxxxxx"
        Protocol: HTTPS
        Port: 443
    Default80Ingress:
      Properties:
        FromPort: 443
        ToPort: 443

[zavbala]

@chingc replacing Webapp with the name of your service in Pascal case seems to fix the problem for me. Another the thing is the guide (https://docs.docker.com/cloud/ecs-integration/#setting-ssl-termination-by-load-balancer) does not cover the part where 443 port is not opened in security group.

for my case:

x-aws-cloudformation:
  Resources:
    BackendTCP80Listener:
      Properties:
        Certificates:
          - CertificateArn: "arn:aws:acm:xxxxxxx"
        Protocol: HTTPS
        Port: 443
    Default80Ingress:
      Properties:
        FromPort: 443
        ToPort: 443

Even with these extra lines isn't works :/

[JohnPreston]

Or, just use ECS Compose-X ? Literally been doing all this for 2y, in production.

Using x-elbv2 (nightly)

Here is a walkthrough of ELBv2 (ALB) that uses cognito userpool and Azure for auth And one with ELBv2 (NLB)

This does not just take care of creating CFN templates. It will actively check that

• ELB settings are valid and auto-correct when appropriate
• Validates that the ports you define for the target groups are actually open on the services
• Supports multi-container tasks definitions to route it to the right one based on rules
• Supports to create ACM certificates, as well as finding existing ones (discovery based on tags)
• One ELB to many different services (again, routing based on rules)
• Creating DNS records pointing to LB (using x-route53)
• Supports discovery of cognito-userpool and will do the necessary to associate to listener rules.
• For ALB, deals with all the SG ingress rules.

Hope this helps, and if any, feedback most welcome

Edit: I missed that band wagon of the x-aws-cloudformation thing, but I think that ecs-compose-x is much closer to what the suggestion is in Discuss an idea of a way to implement this in Compose from this article mentioned above

Not to mention, all the other x-<service> extensions supported.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it had not recent activity during the stale period.