Question: Having a Scheduled Job with a Spot Instance

[dmathewwws]

Hey,

I would like the feature of scheduling while also saving money using spot instances for low priority scheduling tasks. (I'm ok if it job doesn't run because no spot instances are available)

I already have a scheduled job in copilot, is using AWS batch possible OR is there a spot instance variable like for Backend services?

Thanks!

[huanjani]

Hello, @dmathewwws!

Unfortunately, there isn't a way to use AWS Batch with Copilot, and there isn't a Copilot-native way to choose Fargate Spot as the capacity provider for the Scheduled Job workload type.

I explored using a YAML patch to change the Step Function task definition...something like:

- op: add
  path: /Resources/StateMachine/Properties/DefinitionString
  value: "{\n  \"Version\": \"1.0\",\n  \"Comment\": \"Run AWS Fargate task\",\n  \"StartAt\": \"Run Fargate Task\",\n  \"States\": {\n    \"Run Fargate Task\": {\n      \"Type\": \"Task\",\n      \"Resource\": \"arn:${Partition}:states:::ecs:runTask.sync\",\n      \"Parameters\": {\n        \"CapacityProviderStrategy\": \"FARGATE_SPOT\",\n        \"LaunchType\": \"FARGATE\",\n        \"PlatformVersion\": \"LATEST\",\n        \"Cluster\": \"${Cluster}\",\n        \"TaskDefinition\": \"${TaskDefinition}\",\n        \"PropagateTags\": \"TASK_DEFINITION\",\n        \"Group.$\": \"$$.Execution.Name\",\n        \"NetworkConfiguration\": {\n          \"AwsvpcConfiguration\": {\n            \"Subnets\": [\"${Subnets}\"],\n            \"AssignPublicIp\": \"${AssignPublicIp}\",\n            \"SecurityGroups\": [\"${SecurityGroups}\"]\n          }\n        }\n      },\n      \"End\": true\n    }\n  }\n}      "

...but, sadly, CapacityProviderStrategy does not appear to be a supported parameter: https://docs.aws.amazon.com/step-functions/latest/dg/connect-ecs.html.

Thanks for the suggestion! I'll label this issue appropriately.

[huanjani]

Related: https://github.com/aws/aws-cdk/issues/20013#issuecomment-1281074819

[vbarba]

👍