ADF Pipeline deploying multiple stacks requiring different parameters

[ackjewtn]

I'm trying to deploy multiple stacks to the same account in a serial fashion (the ordering is key here). And the stacks require different parameters.

How do i define "per stack" parameters in the params-folder? In this case, i see that the TemplateConfiguration is this: my-multi-stack-pipeline-build::root_dir/params/{ACCOUNT_NAME}_{REGION}.json

I cannot use the parameter overrides either, because my parameters are dynamic. I have a script that is parsing the CDK manifest to populate the parameter files with Asset information (Like S3 bucket location, ArtifactHash etc for Lambda functions) in the buildspec file.

Initially i wrote this info to global.json, and it was working, but only when deploying one stack. Now it complains that the parameters defined doesn't exist in the second and third stack.

I've configured ADF like this:

  - name: my-multi-stack-pipeline
    default_providers:
      source:
        provider: github
        properties:
          repository: foobar
          owner: foo
          branch: develop
      build:
        provider: codebuild
        properties:
          spec_filename: root_dir/buildspec.yml
          image: AMAZON_LINUX_2_3
          size: medium
      deploy:
        provider: cloudformation
        properties:
          root_dir: root_dir
          action: CREATE_UPDATE
    regions: [eu-west-1, eu-north-1]
    targets:
      - name: stack-a-dev
        target: 111111111111
        properties:
          stack_name: "stack-a-dev"
          template_filename: "stack-a-dev.yml"
      - name: stack-b-dev
        target: 111111111111
        properties:
          stack_name: "stack-b-dev"
          template_filename: "stack-b-dev.yml"
      - name: stack-c-dev
        target: 111111111111
        properties:
          stack_name: "stack-c-dev"
          template_filename: "stack-c-dev.yml"

Buildspec:

version: 0.2
env:
  variables:
    ROOT_DIR: root_dir
  parameter-store:
    TOOLING_BUCKET: "/pipeline-tooling-bucket"
phases:
  install:
    runtime-versions:
      python: 3.8
      nodejs: 12
    commands:
      - n 16
      - cd $ROOT_DIR
      - aws s3 cp $TOOLING_BUCKET . --quiet --recursive
      - bash install.sh
  pre_build:
    commands:
      - bash pre-build.sh
  build:
    commands:
      - npm run co:login
      - npm install
      - npm run build
      - cdk synth stack-a-dev > stack-a-dev.yml
      - cdk synth stack-b-dev > stack-b-dev.yml
      - cdk synth stack-c-dev > stack-c-dev.yml
      - bash cdk-package.sh # parses the manifest and generating asset params
      - python adf-build/generate_params.py
  post_build:
    commands:
      - bash post-build.sh
artifacts:
  files:
    - '$ROOT_DIR/*.yml'
    - '$ROOT_DIR/params/*'

[ackjewtn]

I just stumbled over this issue and PR, and i think this change would solve this issue.

[sbkok]

Alternatively, you could use the mono repo approach as shown here.

You could generate the stacks, move those to their own folder. Then run generate params per folder. Please let me know if this approach fixes your issue.

If not, also please let us know.

Please note: As you linked an issue that is scheduled to be added in v3.2.1, I tagged this issue as such too. But it would be even better if we found a solution with the current version. I hope this fixes it for you.

[sbkok]

I'm closing this issue as it has been inactive for a long time. This probably means that it is not an issue and it has been fixed in the meanwhile. I believe the mono-repo approach as discussed above would solve it for you. You can use the same pattern in a single-repository and single-pipeline too.

Please reopen if you still require an enhancement or assistance regarding this issue.

Thank you!