Support for multiple AWS Organizations

[Nr18]

Introduction

Typically, within your AWS Organization you have some setup to provide DTAP environments for your workloads. However testing the your implementation of the Landing Zone is not so easy to setup.

For simple tests like SCP you could use a test OU to first validate if the SCP work as intended. But not all cases could be tested that easily and you even still need to tweak the SCP when you move it. But other landing zone features like deploying CloudTrail in your master account and the bucket in your logging/archiving account then it becomes even harder.

For this reason we are setting up a test organization. This is replica of the production organization and also uses ADF. All repositories that deploy the landing zone resources deploy them from a development branch. While production deploys them from a master/main branch.

Some stacks that we created contained a hardcoded master, logging, etc account id. Because what are the chances that those will change right ;-). With the introduction of a test organization they do change.

Goal

Collect information on what we need to change to ADF to support multiple organizations. And poll if there is a broader interest in this feature request. So, if you could use this 👍 this issue and leave your use-case, comments, suggestions in this thread.

Problem areas

Maintaining duplicate config

You now need to use duplicate repos of the aws-deployment-framework-bootstrap and aws-deployment-framework-pipelines. Maintaining two repositories could lead to configuration drift over time.

Options

• Keep 2 different repositories, it gives the greatest form of flexibility.
• ?

Code Repositories

When using CodeCommit the repositories live in the specific organization. But the two organizations are separated.

Options

• Create repositories in both organizations. Work only on 1 repository and sync a specific branch to the other organization for deployment. For example: if your working repo is in the testorg you could push master to the other repo when it's merged. But also the other way around. When you merge to development sync to the testorg when done testing merge to master.
  • We currently use a separate stack to do this. It's a Step Function that is triggered on a pull request being merged into development. When the event matches the requirements we will run a AWS Lambda that will checkout the code, assume a role in the other account. And do a git push.
• Allow cross organization access to the repository.

Parameters

When you deal with organization agnostic templates that you deploy using ADF. You have to pass in for example the organization id or the account ids of the master, logging, security accounts.

Options

• Manually render the <account-name>_<region>.json from the buildspec.yml
• ADF needs to pass in the OrganizationID by default to all CodeBuild Projects, and generate_params.py will look for param files like: o-myorgid.yml.
• Use SSM Parameter. Currently, the organization_id parameter already exists so adding it to the params/global.yml file like this:

  Parameters:
    OrganizationId: resolve:organization_id
    MasterAccountId: resolve:master_account_id
  

  Within ADF there is also a master_account_id parameter, so that is already available. But it would be nice if ADF manages a parameter for all AWS Account IDs. However, the account name and alias are different because it needs to be unique so we need a way to reference these accounts in a more generic way. By adding something like a unique goal to an account we could identify the accounts more easily:

  - account_full_name: acme-testorg-logging
    organizational_unit_path: /my/path
    email: [email protected]
    alias: acme-testorg-logging
    goal: logging
  

  For example we could do something like /adf/accounts/<goal>/account_id so for your logging account that would be: /adf/accounts/logging/account_id. This allows you to use the following snippet:

  Parameters:
    LoggingAccountId: resolve:/adf/accounts/logging/account_id
  

Open for any feedback!

[Nr18]

Forgot to mention it but had a chat with @sbkok about the parameters. Hence I updated the issue description. I validated the use cases we have not all but looking good. And we are ok with using parameters for now.

Any other insight is welcome 🙏.

[sbkok]

I'm happy to announce that DTAP ADF environment support is made possible with multiple AWS Organizations, as contributed by the following pull requests: #672, #668, #636, #634, and #633. These will be part of the next major version of ADF, v4.0. I appreciate your patience regarding this feature request.