Proposal for decentralised deployment maps and self-mutating pipelines.

[StewartW]

What and Why

Currently if creating a new pipeline or repository we have to create a deployment map in the pipelines repository. This is a fine approach at the moment. However it does create a disconnect between the deployable application (CFn, CDK, Terraform etc) and the targets.

I'd like to propose that we expand the ADF to support deployment_map.yml files that are in the source of the code being deployed.

How?

To start with, we would continue to use the existing pipelines pipeline to generate ADF pipelines, it's primary source would still be from CodeCommit as per the current setup. However we add in a new configuration property to the adfconfig. (Maybe additional_deployment_map_sources). These would then be added in as CodePipeline Secondary Sources to the pipelines pipeline.

Then when a project is pushed to, it would be responsible for pushing it's own deployment map to the configured location.

Pipeline generation then runs as intended, pulling the existing maps from the additional_deployment_map_sources.

This would also tie in to another proposal of self mutating pipelines. Where the pipelines themself would be responsible for updating their targets based on the deployment map. (Very similar to how CDK pipelines works)

TL;DR

Allow for deployment_maps from different sources (S3 to start with) This then enabled a new pipeline type (or new build provider/default functionality) that enables mutating pipelines.

Images

High level comparison of existing flow (left) and additional flow (right)

High Level idea for self-mutating pipeline

As always. Feedback, criticism and ideas are welcome.

[dsudduth]

@StewartW This is a great idea! I'm finding that ADF has value as a CICD framework. This step would move ADF toward the declarative nature seen in systems like Jenkins, CircleCI, Travis, etc. where the pipeline is defined as code and stored in the repository. This direction would lead to faster builds and isolated failures.