valheim-ecs-fargate-cdk icon indicating copy to clipboard operation
valheim-ecs-fargate-cdk copied to clipboard

updownstatus API `startstop` endpoint has no effect

Open rjaduthie opened this issue 2 years ago • 1 comments

I've added as much info as possible here, not sure of the cause of the problem, but I've put my next thing to check in the "Hypotheses" section. I might have time to look at this later, but it might be the weekend before I can sit down to work this through. Any input appreciated!

Issue

After applying the fix in PR #33, the API endpoints appear to work, but in fact the startstop endpoint is having no effect on the desire count of the ECS service.

Recreate

  1. Deploy using code from PR #33
  2. Check server status - by looking at the AWS console or using the serverstatus endpoint - at this point if the status is "not running" the response is empty from the serverstatus endpoint:
curl --request GET 'https://[YOUR_API_GATEWAY_DOMAIN]/prod/serverstatus
  1. If the response is blank try starting the task:
curl --request GET 'https://[YOUR_API_GATEWAY_DOMAIN]/prod/startstop?key=[YOUR_API_PASSWORD_FROM_DOTENV_FILE]&desiredCount=1'

... else try to stop the task:

curl --request GET 'https://[YOUR_API_GATEWAY_DOMAIN]/prod/startstop?key=[YOUR_API_PASSWORD_FROM_DOTENV_FILE]&desiredCount=0'
  1. Check the status of the task again, either in the console or by using the API serverstatus endpoint.

Checks

Lambda Logs

The Lambda backing the startstop endpoint logs that it has made the update command - e.g.:

2023-01-20T00:05:18.106Z	732cdc88-f3e5-4a63-a2ac-a4b41237a9d5	INFO	starting service 
{
    "desiredCount": 1,
    "service": [SERVICE_ARN],
    "cluster": [CLUSTER_ARN]
}

and

2023-01-20T08:37:06.306Z	ca083a87-1c9f-4d5d-91d3-d0ddc1c9fd8f	INFO	changing desiredCount to 0
2023-01-20T08:37:06.389Z	ca083a87-1c9f-4d5d-91d3-d0ddc1c9fd8f	INFO	starting service 
{
    "desiredCount": 0,
    "service": [SERVICE_ARN],
    "cluster": [CLUSTER_ARN]
}

... but this is having no effect.

The code lines are: https://github.com/rileydakota/valheim-ecs-fargate-cdk/blob/61bf2bb3db9207cd633a02969dc45522a9319b01/resources/startstopserver.ts#L34-L38

https://github.com/rileydakota/valheim-ecs-fargate-cdk/blob/61bf2bb3db9207cd633a02969dc45522a9319b01/resources/startstopserver.ts#L55-L60

These match up with use of the ECS client - docs here: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-ecs/classes/updateservicecommand.html

Updating cluster another way

As a sense check, you can use the aws cli to perform the same task:

aws ecs update-service --cluster [CLUSTER_ARN] --service [SERVICE_ARN] --desired-count [0|1]

After performing this command, the cluster responds. The serverstatus endpoint reflects the change and also the AWS console can be used to check, too.

This command is equivalent to what the Lambda is using - docs are here: https://docs.aws.amazon.com/cli/latest/reference/ecs/update-service.html

Hypotheses

  • Is this a permissions issue? Does the startstop lambda have the correct Role, security group, ..., whatever it needs, ..., to perform this task?
  • Something else ...?

rjaduthie avatar Jan 20 '23 09:01 rjaduthie

Worked it out. It isn't permissions: thankfully! The "startstop" lambda code doesn't await the resolution of the ECS client command - by adding the await keyword the endpoint works as expected.

See PR #35 for fix (and a couple of other improvements).

rjaduthie avatar Jan 22 '23 00:01 rjaduthie