Error: Another CLI (PID=72391) is currently synthing to cdk.out when having MaxConcurrentTasks > 0

[dannysteenman]

Subject of the issue

In the latest version of orgformation you introduced maxconcurrenttasks for aws-cdk tasks which is great! However when I try to set the concurrenttasks to e.g. 10 then the deploy fails because cdk.out is locked on 1 tasks causing the other tasks to fail.

This is the task:

AccountPasswordPolicy:
  Type: update-cdk
  Path: ../../../
  CustomDeployCommand: !Sub AWS_REGION=${AWS::Region} npx cdk deploy${CurrentTask.Parameters} --app "ts-node ./src/main.ts" '*account-pasword-policy' --require-approval never
  CustomRemoveCommand: !Sub AWS_REGION=${AWS::Region} npx cdk destroy${CurrentTask.Parameters} --app "ts-node ./src/main.ts" '*account-pasword-policy' --force
  Parameters:
    MAX_PASSWORD_AGE: 90
    MIN_PASSWORD_LENGTH: 18
  OrganizationBinding:
    IncludeMasterAccount: true
    Account: "*"
    Region: !Ref globalRegion
  MaxConcurrentTasks: 10

When the task gets deployed:

INFO: Executing: include src/templates/020-secure-defaults/_tasks.yml.
INFO: Executing: update-cdk AccountPasswordPolicy.
INFO: Executing command: AWS_REGION=us-east-1 npx cdk deploy -c 'MAX_PASSWORD_AGE=90' -c 'MIN_PASSWORD_LENGTH=18' --app "ts-node ./src/main.ts" '*account-pasword-policy' --require-approval never in account ...
INFO: Executing command: AWS_REGION=us-east-1 npx cdk deploy -c 'MAX_PASSWORD_AGE=90' -c 'MIN_PASSWORD_LENGTH=18' --app "ts-node ./src/main.ts" '*account-pasword-policy' --require-approval never in account ...
INFO: Executing command: AWS_REGION=us-east-1 npx cdk deploy -c 'MAX_PASSWORD_AGE=90' -c 'MIN_PASSWORD_LENGTH=18' --app "ts-node ./src/main.ts" '*account-pasword-policy' --require-approval never in account ...
INFO: Executing command: AWS_REGION=us-east-1 npx cdk deploy -c 'MAX_PASSWORD_AGE=90' -c 'MIN_PASSWORD_LENGTH=18' --app "ts-node ./src/main.ts" '*account-pasword-policy' --require-approval never in account ...

ERROR: Workload AccountPasswordPolicy in .../us-east-1 update failed. reason: error invoking external command AWS_REGION=us-east-1 npx cdk deploy -c 'MAX_PASSWORD_AGE=90' -c 'MIN_PASSWORD_LENGTH=18' --app "ts-node ./src/main.ts" '*account-pasword-policy' --require-approval never.
 error: Error: Command failed: AWS_REGION=us-east-1 npx cdk deploy -c 'MAX_PASSWORD_AGE=90' -c 'MIN_PASSWORD_LENGTH=18' --app "ts-node ./src/main.ts" '*account-pasword-policy' --require-approval never

Another CLI (PID=72391) is currently synthing to cdk.out. Invoke the CLI in sequence, or use '--output' to synth into different directories.

ERROR: Workload AccountPasswordPolicy in .../us-east-1 update failed. reason: error invoking external command AWS_REGION=us-east-1 npx cdk deploy -c 'MAX_PASSWORD_AGE=90' -c 'MIN_PASSWORD_LENGTH=18' --app "ts-node ./src/main.ts" '*account-pasword-policy' --require-approval never.
 error: Error: Command failed: AWS_REGION=us-east-1 npx cdk deploy -c 'MAX_PASSWORD_AGE=90' -c 'MIN_PASSWORD_LENGTH=18' --app "ts-node ./src/main.ts" '*account-pasword-policy' --require-approval never
 Another CLI (PID=72391) is currently synthing to cdk.out. Invoke the CLI in sequence, or use '--output' to synth into different directories.

Your environment

• version of org-foramtion: 1.0.11
• version of node: v18.16.0
• Linux/macOS

Steps to reproduce

Create a cdk stack, then the cdk task in orgformation and set maxconcurrenttasks to > 1

Expected behaviour

I expect that this feature allows you to deploy cdk stacks simultaneously on multiple accounts in parallel when MaxConcurrentTasks > 1.

Actual behaviour

cdk.out gets locked (normal cdk behaviour) and that causes the other deploys to fail.

Note: this could be solved by having the ability to add --output to a random cdk-<random hash>.out folder

[davidvpe]

Hi @dannysteenman ✋

Yes as part of the update to MaxConcurrentTasks we added an extra parameter to the Default Deploy Command --output cdk.out/${CurrentTask.AccountId}.

Since you have a CustomDeployCommand you also need to add that additional parameter

[dannysteenman]

Hi @dannysteenman ✋

Yes as part of the update to MaxConcurrentTasks we added an extra parameter to the Default Deploy Command --output cdk.out/${CurrentTask.AccountId}.

Since you have a CustomDeployCommand you also need to add that additional parameter

Yes I've tested it and it works perfectly! Thanks David!

[OlafConijn]

reopened this with the intention to clarify this in the documentation