depends_on field in pipeline does not account for infrastructure dependencies

[JakeBrown]

Our use case is that after creation of a new Environment, the deployment of a Pipeline fails at the build step when the application contains a Worker Service.

SNS topic abc does not exist in environment xyz

This happens because the Pipeline attempts to generate the CloudFormation templates (copilot svc package) for all services, before any services have been deployed. Any SNS topics that the worker subscribes to have not been created yet.

We were excited by the new feature in v1.18 Control the order of services of jobs deployed in a pipeline (https://github.com/aws/copilot-cli/pull/3497) but then ran into this issue. It is probably by design but it wasn't clear from the docs.

Currently our workaround is to manually deploy the publishing service before the pipeline runs.

[dannyrandall]

Hey @JakeBrown! I was able to replicate the same error by creating a new worker service in my already existing environment, adding a publish block to an existing service in that environment, and letting the pipeline deploy the worker service for the first time.

I think this is a bug for us! We want you to be able to deploy services through pipelines.

[dannyrandall]

Thinking about the best way to solve this issue - I wonder if this (not currently implemented) feature might be the best solution. https://github.com/aws/copilot-cli/issues/3415

You'd modify your pipeline buildspec to have copilot svc package --validate=false and Copilot would skip validating that a topic exists during svc package. Instead, Copilot would just generate the CloudFormation template and deploy it in the order determined by your pipeline manifest. You'd still have to have depends_on: [publishers] set for worker services, otherwise the worker's CloudFormation deployment would fail.

Thoughts on that solution?

[JakeBrown]

@dannyrandall yep that'll cause it too! Great to hear you're treating it as a bug and not just a design decision/limitation. That is much appreciated.

I did go looking for an option like --validate=false, I think that's a good solution and hopefully a fairly obvious workaround for those running into this issue.

With that option it would be trivial to modify the pipeline buildspec to disable validation for all services, in all environments, on all deploys. That's a bit overkill but at least it lets us make progress. Disabling validation only for the worker service service (and only when needed) will be a bit trickier to script, but sounds doable.

Thanks!