ecs-deploy is not waiting for deployment to succeed

[FarhadF]

ecs-deploy used to wait for deployments to finish.

Current behavior:

• 1 deployment is already there in completed state
• ecs-deploy submits a new deployment
• ecs-deploy logs Service deployment successful and exists the loop.

Expected behavior:

• 1 deployment is already there in completed state
• ecs-deploy submits a new deployment
• ecs deploy waits until there is only 1 deployment remain (ecs will remove the old deployment and just keep the new on success with completed status)
• ecs-deploy logs Service deployment successful and exists the loop.

example command to reproduce:

ecs-deploy --timeout 600 --region region --cluster clustername  --service-name service --force-new-deployment -i repo/service:tag

[forevermatt]

Hi @FarhadF. Thanks for opening this issue. Until this is resolved, you can use ecs-deploy version 3.10.2 to have the previous behavior.

The change to the current behavior was made in response to issue #249. @fillup, would you mind taking a look at these two issues to see how best to resolve both concerns?

[rpaterson]

This just caught me out too. Can we revert #249?

[forevermatt]

Hi @rpaterson. Sorry for the delay. This is on our radar, we just haven't managed to sort out whether reverting this would re-add a bug re: #249.

@Rishang, could you clarify what problem you were experiencing that #249 fixed for you? (We'll probably need to revert that change, so more information from you could help us do so in a way that doesn't re-add any problem you experienced.)

Thanks.

[Rishang]

NUM_DEPLOYMENTS=$($AWS_ECS describe-services --services $SERVICE --cluster $CLUSTER | jq "[.services[].deployments[]] | length")

This was a variable in order to see if deployent gets to running healty sate, but previously it only check for single deployemnt, which should'nt be the case as if any service has more than 2 desired count it get fail in that case, so just had changed to heathcheck to be validate for 1 or more than one deployments as you can see in this change https://github.com/silinternational/ecs-deploy/pull/252/files

I don't think this can cause any issue, as i am using same code as my PR #252 in projects and is working well

[Rishang]

@FarhadF can you check for the condition in function waitForGreenDeployment for your code

[fillup]

I don't remember exactly and don't have easy access to an ECS cluster at the moment to test, but I don't think the deployment and the number of desired tasks are related. The check for desired count happens as part of updating the service in this section: https://github.com/silinternational/ecs-deploy/blob/develop/ecs-deploy#L454.

@forevermatt I'd recommend reverting #249.

@Rishang, if you can provide more details as to what you were seeing not working properly before that'd be helpful, but I don't believe the change you made is related the thing you're concerned with.

[fillup]

Looks like #261 will revert/fix this.

[forevermatt]

@fillup Thanks!

@FarhadF and @rpaterson, This has now been fixed and released as ecs-deploy 3.10.6 (also available via homebrew, if that's applicable to you)

[roy-moven]

Thank you!

[Rishang]

what about the case where my minimum desired count for ecs service is 2 or more than one

[forevermatt]

Hi @Rishang. That line in ecs-deploy is checking the number of deployments, not the desired count for a service.

For example, when I run the related command...

aws ecs describe-services --services my-service-name --cluster my-cluster-name | jq "[.services[].deployments[]]"

... right now, I see a single result (thus length is 1) which specifies a desiredCount of 2 (since I have that task definition running 2 instances of my container).

Was ecs-deploy doing something different than you expected? If you can provide us with some more details about that, we might be able to help resolve the problem you were experiencing.

Thanks.