aws-toolkit-azure-devops
aws-toolkit-azure-devops copied to clipboard
Execute Changeset appears to only work once
The first time execute changeset runs, it succeeds. The change is made and the task in Azure Devops completes.
Subsequent executions of the task do apply the changeset, but the task will never complete and has to be cancelled. I believe the reason is that the task is waiting for the stack to reach "create complete" status. However this never happens because thestack status enters "Update Complete" status instead following the initial deployment.
is this AWS CloudFormation Create/Update Stack task with execute change-set or AWS CloudFormation Execute Change Set task ?
The ExecuteChangeSet task should switch to different status waiters (create_complete, update_complete) based on whether the stack has resources at the time the task runs. In the log output, when you're doing an update, are you not seeing the text 'Waiting for stack stackname to reach update complete status'? If you're always seeing 'Waiting for stack stackname to reach create complete status' that would suggest the detection logic is failing for some reason.
Waiters (in the underlying sdk the tasks use) can only wait on one state, hence we have to detect-and-switch.
So I think that the logic must be failing. The execute changeset task works great the first time - here is the log for that:
2019-04-03T21:05:22.2061497Z Executing change set build-4-sbox, associated with stack mystack 2019-04-03T21:05:22.4094880Z Waiting for stack mystack to reach create complete status 2019-04-03T21:08:53.7647365Z Execution of change set to create stack mystack completed successfully 2019-04-03T21:08:53.7647811Z Executed change set build-4-sbox 2019-04-03T21:08:53.7729252Z ##[section]Finishing: Execute Change Set: build-4-sbox
When the next run happens, we get this:
2019-04-04T14:47:49.4540941Z Executing change set build-6-sbox, associated with stack mystack2019-04-04T14:47:49.7142750Z Waiting for stack mystack to reach create complete status 2019-04-04T14:50:04.9664798Z ##[error]The operation was canceled. 2019-04-04T14:50:04.9680950Z ##[section]Finishing: Execute Change Set: build-6-sbox
I cancel it because it never completes. Right now I have a powershell task instead that runs and watches for the update. The logic isn't perfect but the task completes and lets us move forwards.
One additional point I would mention is it would also be great to integrate logic that checks a changeset actually exists. When running ci/cd pipelines we may want to trigger them nightly to make sure we don't have drift. If that happens and there are no stack updates, the create/update stack task will pass successfully, but not create a changeset, leading to failure of the subsequent task. This is less of a big deal, but feel like the task should still pass with a log output that no changesets were detected.
Let me know if any further detail is needed!
Thanks, Matthew
Was this ever resolved? Facing this same exact issue currently.
Creating UPDATE type change set <CHANGESETNAME>
Waiting for change set <CHANGESETNAME> to be validated for stack <STACKNAME>
Change set successfully validated
Executing change set <CHANGESETNAME> <STACKNAME>
Waiting for stack <STACKNAME> to reach create complete status
It seems to be waiting for "create complete" when the original type is "UPDATE"
Task definition:
- task: CloudFormationCreateOrUpdateStack@1
displayName: Create and Execute ChangeSet
inputs:
awsCredentials: <service connection>
regionName: <region>
stackName: <STACKNAME>
templateSource: 'file'
templateFile: <template file>
templateParametersSource: 'inline'
templateParameters: |
[
<some parameters>
]
useChangeSet: true
changeSetName: <CHANGESETNAME>
capabilityIAM: false
capabilityNamedIAM: false
tags: |
<some tags>