feat: implement dependency order execution for terraform --all flag

[osterman]

what

• Implement dependency-ordered execution for atmos terraform apply --all command
• Create a reusable dependency graph package that both --all and --affected flags can use
• Ensure Terraform components are always processed in the correct order based on their dependencies

why

• The --all flag was processing components without respecting dependency order, which could lead to deployment failures
• The dependency logic was tightly coupled with the --affected functionality and needed to be generalized
• Users need a reliable way to deploy all components while respecting inter-component dependencies

Key Features

• ✅ Dependency order execution for --all flag
• ✅ Circular dependency detection with clear error messages
• ✅ Support for cross-stack dependencies
• ✅ Filtering by stack, components, and YQ queries
• ✅ Skipping of abstract and disabled components
• ✅ Dry-run mode support
• ✅ Reusable graph logic for both --all and --affected

Implementation Details

New Dependency Graph Package (pkg/dependency/)

Created a generalized, reusable dependency graph implementation:

• graph.go - Core graph structure and operations
• builder.go - Builder pattern for safe graph construction
• sort.go - Topological sort using Kahn's algorithm
• filter.go - Graph filtering operations
• types.go - Core types and interfaces

Terraform Execution Updates

• terraform_all.go - New ExecuteTerraformAll function for --all flag
• terraform_affected_graph.go - Refactored ExecuteTerraformAffected to use the graph
• terraform_executor.go - Shared execution logic for processing nodes

Testing

• Comprehensive unit tests for the dependency graph package
• Integration tests for terraform execution
• Test fixtures with complex dependency scenarios
• All existing tests continue to pass

Example

Given this configuration:

components:
  terraform:
    vpc:
      vars:
        cidr: "10.0.0.0/16"
    
    database:
      settings:
        depends_on:
          - component: vpc
    
    application:
      settings:
        depends_on:
          - component: database

Running atmos terraform apply --all will execute in order:

1. vpc (no dependencies)
2. database (depends on vpc)
3. application (depends on database)

Testing

• Run unit tests: go test ./pkg/dependency/...
• Run integration tests: go test ./internal/exec -run TestBuildTerraformDependencyGraph
• Test with fixture: atmos terraform plan --all --dry-run in tests/fixtures/scenarios/terraform-apply-all-dependencies/

references

• Closes #1242
• Closes DEV-3512
• Linear issue: https://linear.app/cloudposse/issue/DEV-3512/implement-atmos-terraform-apply-all-in-dependency-order

Summary by CodeRabbit

• New Features

  • Terraform runs for --all and --affected now respect dependency order (topological; reversed for destroy), with filtering by stack/component/metadata and per-component dry-run; enhanced execution logging and progress reporting.
  • Added reusable dependency-graph capabilities for building, sorting, filtering, and reachability/levels.

• Documentation

  • Added PRD describing dependency-ordered Terraform execution, architecture, and migration plan.

• Bug Fixes

  • Stricter validation and clearer errors around --all and related CLI usages.

• Tests

  • Extensive unit tests covering graph, filtering, parser, and execution flows.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai[bot]]

📝 Walkthrough

Walkthrough

This PR adds a reusable dependency graph system (pkg/dependency) and integrates it into Atmos Terraform execution flows. It implements graph construction, filtering, topological ordering, and per-node execution so --all and affected-component workflows run in dependency order with query/stack/component filters and cycle detection.

Changes

Cohort / File(s)	Summary
Product docs docs/prd/terraform-dependency-order.md	New PRD describing architecture, API, data models, algorithms, implementation plan, tests, migration and examples.
Error sentinels errors/errors.go	Added many new exported error variables for dependency parsing, graph/terraform execution, cache ops and aliases (e.g., ErrUnsupportedDependencyType, ErrBuildDepGraph, ErrTerraformExecFailed, ErrCacheWrite, ErrCreateTempDirectory).
Dependency types pkg/dependency/types.go	New public types: Node, Graph, Builder interface, ExecutionOrder, and Filter.
Graph core pkg/dependency/graph.go, pkg/dependency/errors.go, pkg/dependency/builder.go	Graph implementation (nodes, dependencies, roots, cycle detection), builder to incrementally construct/validate graphs and finalize into immutable Graph, package-level dependency errors.
Graph algorithms pkg/dependency/sort.go	Topological sort (Kahn), reverse sort, execution level grouping, path/find and reachability utilities.
Graph filtering & utilities pkg/dependency/filter.go	Subgraph extraction (include deps/dependents), FilterByType/Stack/Component, connected components, node removal and related helpers.
Dependency tests pkg/dependency/*_test.go	Unit tests covering graph behavior, sorting, filtering, cloning, cycles, connected components and removals.
Dependency parsing internal/exec/dependency_parser.go, internal/exec/dependency_parser_test.go	DependencyParser that extracts settings.depends_on (array/map variants), validates targets against node map, skips abstract/disabled components, and adds dependencies via GraphBuilder; comprehensive unit tests.
Terraform --all orchestration internal/exec/terraform_all.go, internal/exec/terraform_all_test.go, internal/exec/terraform_all_simple_test.go	ExecuteTerraformAll builds graph, applies filters (components/stack/query), computes (reverse) topological order for apply/destroy and iterates execution; tests for graph build, filtering and argument validation.
Terraform affected execution (graph) internal/exec/terraform_affected_graph.go	ExecuteTerraformAffectedWithGraph discovers affected components, builds filtered graph including dependents/dependencies, and executes nodes in topological order.
Terraform execution per-node internal/exec/terraform_executor.go	executeTerraformForNode plus helpers: skip logic (abstract/disabled), query-based filtering (YQ evaluation), info updates, command formatting and execution (dry-run support), and error wrapping.
Terraform utils & tests internal/exec/terraform_utils.go, internal/exec/terraform_utils_test.go	Reinstated recursive affected-component helper, added parseUploadStatusFlag, test renamed to call new affected-with-graph function.
Test fixtures & blog tests/fixtures/scenarios/terraform-apply-all-dependencies/*, website/blog/2025-12-16-terraform-all-dependency-order.mdx	New scenario fixtures (dev/prod stacks with depends_on graphs) and a blog post describing --all dependency ordering and examples.

Sequence Diagram(s)

sequenceDiagram
    actor User
    participant CLI as atmos CLI
    participant Exec as ExecuteTerraformAll / Affected
    participant Builder as GraphBuilder
    participant Parser as DependencyParser
    participant Graph as Graph
    participant Sort as TopologicalSort
    participant NodeExec as executeTerraformForNode
    participant TF as Terraform

    User->>CLI: atmos terraform apply --all -s prod
    CLI->>Exec: ExecuteTerraformAll(info)
    Exec->>Builder: NewBuilder()
    Exec->>Exec: addNodesToGraph (register nodes)
    Exec->>Parser: NewDependencyParser(builder, nodeMap)
    Exec->>Parser: ParseComponentDependencies(...) per component
    Parser->>Builder: AddDependency(from→to)
    Exec->>Builder: Build()
    Builder-->>Graph: Graph (validated)
    Exec->>Graph: applyFiltersToGraph (component/stack/query)
    Exec->>Sort: TopologicalSort()
    Sort-->>Exec: ExecutionOrder [vpc, db, cache, app]
    loop per node in order
      Exec->>NodeExec: executeTerraformForNode(node, info)
      NodeExec->>NodeExec: shouldSkipNode / query check
      NodeExec->>NodeExec: updateInfoFromNode
      NodeExec->>TF: run "atmos terraform <cmd> <component> -s <stack>"
      TF-->>NodeExec: result
    end
    Exec-->>CLI: completed

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related PRs

• cloudposse/atmos#1327 — related work on multi-component Terraform commands and dependency-ordered execution; touches exec package flows and flags.
• cloudposse/atmos#1405 — changes to dependency resolution and depends_on interpretation; overlaps with parsing and stack-context handling.
• cloudposse/atmos#1530 — modifications to errors package and sentinel error variables that align with newly added error sentinels here.

Suggested reviewers

• aknysh

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title clearly and specifically describes the main implementation: dependency-ordered execution for the terraform --all flag.
Linked Issues check	✅ Passed	The implementation fully satisfies both linked issues: #1242 (apply all components in a stack in order) and DEV-3512 (terraform apply --all in dependency order). All core requirements are met.
Out of Scope Changes check	✅ Passed	All changes directly support dependency-ordered execution for terraform --all. The PR introduces a focused package (pkg/dependency) and integrations without unrelated modifications.

✨ Finishing touches

• [ ] 📝 Generate docstrings

🧪 Generate unit tests (beta)

• [ ] Create PR with unit tests
• [ ] Post copyable unit tests in a comment
• [ ] Commit unit tests in branch feature/dev-3512-implement-atmos-terraform-apply-all-in-dependency-order

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[mergify[bot]]

[!WARNING]

This PR exceeds the recommended limit of 1,000 lines.

Large PRs are difficult to review and may be rejected due to their size.

Please verify that this PR does not address multiple issues. Consider refactoring it into smaller, more focused PRs to facilitate a smoother review process.

[codecov[bot]]

Codecov Report

:x: Patch coverage is 60.44341% with 339 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 73.35%. Comparing base (acaba0f) to head (6f105af).

Files with missing lines	Patch %	Lines
internal/exec/terraform_affected_graph.go	17.98%	113 Missing and 1 partial :warning:
internal/exec/terraform_all.go	48.97%	71 Missing and 4 partials :warning:
internal/exec/terraform_executor.go	0.00%	62 Missing :warning:
internal/exec/terraform_utils.go	0.00%	35 Missing :warning:
pkg/dependency/builder.go	45.16%	11 Missing and 6 partials :warning:
pkg/dependency/filter.go	89.55%	7 Missing and 7 partials :warning:
internal/exec/dependency_parser.go	87.20%	7 Missing and 4 partials :warning:
pkg/dependency/sort.go	92.15%	4 Missing and 4 partials :warning:
pkg/dependency/graph.go	96.15%	3 Missing :warning:

:x: Your patch check has failed because the patch coverage (60.44%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1516      +/-   ##
==========================================
- Coverage   73.47%   73.35%   -0.13%     
==========================================
  Files         633      642       +9     
  Lines       58652    59474     +822     
==========================================
+ Hits        43097    43627     +530     
- Misses      12587    12852     +265     
- Partials     2968     2995      +27     

Flag	Coverage Δ
unittests	73.35% <60.44%> (-0.13%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
errors/errors.go	100.00% <ø> (ø)
pkg/dependency/node_helpers.go	100.00% <100.00%> (ø)
pkg/dependency/graph.go	96.15% <96.15%> (ø)
pkg/dependency/sort.go	92.15% <92.15%> (ø)
internal/exec/dependency_parser.go	87.20% <87.20%> (ø)
pkg/dependency/filter.go	89.55% <89.55%> (ø)
pkg/dependency/builder.go	45.16% <45.16%> (ø)
internal/exec/terraform_utils.go	66.10% <0.00%> (ø)
internal/exec/terraform_executor.go	0.00% <0.00%> (ø)
internal/exec/terraform_all.go	48.97% <48.97%> (ø)
... and 1 more

... and 4 files with indirect coverage changes

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mergify[bot]]

💥 This pull request now has conflicts. Could you fix it @osterman? 🙏

[github-actions[bot]]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-06-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

[github-actions[bot]]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-09-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

[github-actions[bot]]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-13-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

[github-actions[bot]]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-14-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

[github-actions[bot]]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-16-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

[mergify[bot]]

💥 This pull request now has conflicts. Could you fix it @osterman? 🙏

[github-actions[bot]]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-16-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

[github-actions[bot]]

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

None

[mergify[bot]]

💥 This pull request now has conflicts. Could you fix it @osterman? 🙏