Set LaunchType when creating Services and Tasks

[zpatrick]

Summary

Enable Layer0 to deploy services and tasks using AWS Fargate.

Implementation

Retract Typed Environments Decision

It turns out that tying EC2/Fargate compatibility to environments is unnecessary and requires Layer0 to be too hands-on, and creates a restrictive UX.

Stateful & Stateless Layer0 Concepts

The idea is to evangelize two paths for service/task deployments. The default path will be a "stateless" deployment, which will utilize Fargate instances and which a user does not require fine-grained and manual management of instances. The second path is layer0's historical "stateful" deployment pattern, in which a user needs to manage instance sizes and scale, as well as state.

Compatibilities in Task Definitions

In order for a task definition to be Fargate-compatible, there are a few stipulations (defined at the same level as "containerDefinitions"):

    "requiresCompatibilities": [ "FARGATE" ], 
    "networkMode": "awsvpc", 
    "cpu": "256", 
    "memory": "0.5GB",

More info at https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html - in particular, note that there are specific values allowed for "cpu" and "memory".

When ECS.RegisterTaskDefinition is called during deploy create, AWS will always at least check that the task definition is EC2-compatible. In order to determine if the task definition is also Fargate-compatible, FARGATE must be specified in the requiresCompatibilities field. Once a task definition has been returned from AWS, we inspect the AWS-added compatibilities field and populate the Compatibilities field on the Deploy model with stateful and stateless values accordingly.

Service/Task Creation

Because Fargate is so new and likely to change, we want to be fairly hands-off and shouldn't build a ton of our own tooling around it. To that end, we'll introduce a new optional --stateful boolean flag on service create and task create. By default, a service created will proceed along the stateless (Fargate) deployment path, and if the supplied task definition is not Fargate-compatible, we return the resulting AWS error. Alternatively, if --stateful is supplied during service create, layer0 will pursue an EC2 deployment.

Application Load Balancer Support

fargate services only support application load balancers for exposing a service behind a load balancer. To support an ALB, the existing Layer0 Load Balancer will be extended to support a new alb load balancer type.

As the default behavior for Layer0 services going forward will be stateless (i.e., fargate), creating a Layer0 Load Balancer will be updated to create an ALB as default (rather than a classic load balancer).

Tasks

• LaunchType support (@tlake)
  • [x] Implementation (PR: #592)
    • Remove static/dynamic types from environments
    • Add EC2/Stateful and Fargate/Stateless logic in layer0 deploys
    • Add Stateful/Stateless functionality to layer0 services and tasks
  • [ ] Documentation
  • [x] Unit tests (part of #592)
  • [x] Smoke tests (Issue: #602)
  • [x] System tests (PR: #609)
• Application Load Balancer support (@sesh-kebab)
  • [x] Implementation (PR: #580)
  • [ ] Documentation
  • [x] Smoke tests (PR: #593)

[tlake]

I'm closing #551 because it's become redundant since we've started using the checklist in this epic to track all the work for the launch types feature.

[tlake]

I think there's a tiny bit of design work that still needs to be done around handling errors. For consideration, here's a table:

SVC Type	LB Type	DPL Type	Result
stateless	alb	fargate	👍
stateless	alb	ec2	✔️ ~TODO~ Layer0 error: unsupported stateless+ec2 (if left alone, AWS will return InvalidParameterException: Task definition does not support launch_type FARGATE.)
stateless	clb	fargate	✔️ ~TODO~ Layer0 error: unsupported stateless+clb
stateless	clb	ec2	✔️ ~TODO~ Layer0 error: unsupported stateless+clb (if left alone, AWS will return InvalidParameterException: Task definition does not support launch_type FARGATE.)
stateless	---	fargate	👍
stateless	---	ec2	✔️ ~TODO~ Layer0 error: unsupported stateless+ec2 (if left alone, AWS will return InvalidParameterException: Task definition does not support launch_type FARGATE.)
stateful	alb	fargate	✔️ Layer0 error: unsupported stateful+alb (already implemented by @sesh-kebab)
stateful	alb	ec2	✔️ Layer0 error: unsupported stateful+alb (already implemented by @sesh-kebab)
stateful	clb	fargate	✔️ ~TODO~ Layer0 error: unsupported clb+fargate (if left alone, AWS will return InvalidParameterException: The task definition is configured to use network mode awsvpc, which is incompatible with the load balancer [lb-name])
stateful	clb	ec2	👍
stateful	---	fargate	👍 (tentatively) This one is tricky, because AWS will let you deploy this. As far as I can tell, a task definition's Compatibilities array will never contain "FARGATE" without also containing "EC2". However, it also seems like this is a crossing-of-the-streams in terms of our evangelized deployment paths, so I'm not sure how to handle this one. For now, I'll allow it.
stateful	---	ec2	👍

@sesh-kebab's comment in PR #592 is making me reconsider just passing AWS errors in favor of returning Layer0-vocabulary errors where we deem appropriate as well as wrapping AWS errors in similar Layer0 errors.

I can wrap the AWS errors when they arise in createService(), or I can be even more proactive and try to bundle all of this sort of error handling up in the master Create() function, which is where @sesh-kebab's placed his error checking for stateful service + ALB.

EDIT: Tasks should get the same treatment, but they're also less complex due to the absence of deployment behind load balancers.

Task Type	DPL Type	Result
stateless	fargate	👍
stateless	ec2	✔️ ~TODO~ Layer0 error: unsupported stateless+ec2 (if left alone, AWS will return InvalidParameterException: Task definition does not support launch_type FARGATE.)
stateful	fargate	👍 (tentatively) This one is tricky, because AWS will let you deploy this. As far as I can tell, a task definition's Compatibilities array will never contain "FARGATE" without also containing "EC2". However, it also seems like this is a crossing-of-the-streams in terms of our evangelized deployment paths, so I'm not sure how to handle this one. For now, I'll allow it.
stateful	ec2	👍

[sesh-kebab]

@tlake agreed; especially in light of more ALB changes in the pipeline.