atmos icon indicating copy to clipboard operation
atmos copied to clipboard

Add vendor update and diff commands

Open osterman opened this issue 4 months ago β€’ 9 comments

what

  • Implement atmos vendor update command to check for and update vendored component versions
  • Implement atmos vendor diff command to show Git diffs between component versions
  • Add comprehensive test coverage (~80-85%) using gomock for mocking Git operations
  • Add complete documentation for both commands in Docusaurus format

why

  • Users need to see what changed between vendored component versions before updating
  • Users need an easy way to update vendor.yaml version references
  • YAML structure, comments, and anchors must be preserved during updates
  • Git operations should work without requiring local repository clones

references

  • Supersedes #1531 (previous implementation with issues)
  • See docs/prd/vendor-update.md for detailed design

Summary by CodeRabbit

Release Notes

  • New Features

    • Added atmos vendor update command with semantic version constraints, exclusions, and dry-run support for automated component version checking and YAML updates with structure preservation
    • Added atmos vendor diff command to display version differences with colorized output and configurable context
    • Added atmos vendor pull command for component vendoring with stack and tag filtering options
  • Documentation

    • Added comprehensive guides for vendor update, diff, and pull commands with examples and troubleshooting

✏️ Tip: You can customize this high-level summary in your review settings.

osterman avatar Oct 21 '25 22:10 osterman

Dependency Review

βœ… No vulnerabilities or license issues found.

Scanned Files

None

github-actions[bot] avatar Oct 21 '25 22:10 github-actions[bot]

[!WARNING]

This PR exceeds the recommended limit of 1,000 lines.

Large PRs are difficult to review and may be rejected due to their size.

Please verify that this PR does not address multiple issues. Consider refactoring it into smaller, more focused PRs to facilitate a smoother review process.

mergify[bot] avatar Oct 21 '25 22:10 mergify[bot]

[!WARNING]

Rate limit exceeded

@osterman has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 9 minutes and 45 seconds before requesting another review.

βŒ› How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

πŸ“₯ Commits

Reviewing files that changed from the base of the PR and between 97a023a588ebd7f62b32a4af4970bcbe2150ab8f and 2a639dc9b4432dfc134646259ae661ecf68ee6a7.

πŸ“’ Files selected for processing (64)
  • cmd/root.go (1 hunks)
  • cmd/vendor.go (0 hunks)
  • cmd/vendor/diff.go (1 hunks)
  • cmd/vendor/pull.go (1 hunks)
  • cmd/vendor/update.go (1 hunks)
  • cmd/vendor/utils.go (1 hunks)
  • cmd/vendor/vendor.go (1 hunks)
  • cmd/vendor_diff.go (0 hunks)
  • cmd/vendor_pull.go (0 hunks)
  • cmd/vendor_test.go (0 hunks)
  • docs/prd/vendor-refactoring-plan.md (1 hunks)
  • docs/prd/vendor-update.md (1 hunks)
  • errors/errors.go (1 hunks)
  • go.mod (1 hunks)
  • internal/exec/copy_glob.go (2 hunks)
  • internal/exec/copy_glob_error_paths_test.go (3 hunks)
  • internal/exec/copy_glob_test.go (5 hunks)
  • internal/exec/describe_affected_helpers.go (2 hunks)
  • internal/exec/docs_generate.go (2 hunks)
  • internal/exec/file_utils.go (1 hunks)
  • internal/exec/file_utils_test.go (1 hunks)
  • internal/exec/oci_utils.go (2 hunks)
  • internal/exec/oci_utils_test.go (2 hunks)
  • internal/exec/template_processing_test.go (3 hunks)
  • internal/exec/template_utils.go (3 hunks)
  • internal/exec/vendor.go (0 hunks)
  • pkg/datafetcher/schema/vendor/package/1.0.json (1 hunks)
  • pkg/schema/schema.go (1 hunks)
  • pkg/vender/component_vendor_test.go (1 hunks)
  • pkg/vender/vendor_config_test.go (1 hunks)
  • pkg/vendoring/component_utils.go (14 hunks)
  • pkg/vendoring/diff.go (1 hunks)
  • pkg/vendoring/diff_integration_test.go (1 hunks)
  • pkg/vendoring/git_diff.go (1 hunks)
  • pkg/vendoring/git_diff_test.go (1 hunks)
  • pkg/vendoring/git_interface.go (1 hunks)
  • pkg/vendoring/mock_git_interface.go (1 hunks)
  • pkg/vendoring/model.go (6 hunks)
  • pkg/vendoring/params.go (1 hunks)
  • pkg/vendoring/pull.go (1 hunks)
  • pkg/vendoring/pull_integration_test.go (6 hunks)
  • pkg/vendoring/source/git.go (1 hunks)
  • pkg/vendoring/source/github.go (1 hunks)
  • pkg/vendoring/source/mock_provider.go (1 hunks)
  • pkg/vendoring/source/provider.go (1 hunks)
  • pkg/vendoring/source/provider_test.go (1 hunks)
  • pkg/vendoring/source/unsupported.go (1 hunks)
  • pkg/vendoring/template_tokens_test.go (5 hunks)
  • pkg/vendoring/triple_slash_test.go (4 hunks)
  • pkg/vendoring/update.go (1 hunks)
  • pkg/vendoring/update_test.go (1 hunks)
  • pkg/vendoring/uri/helpers.go (5 hunks)
  • pkg/vendoring/uri/helpers_test.go (47 hunks)
  • pkg/vendoring/utils.go (19 hunks)
  • pkg/vendoring/utils_test.go (5 hunks)
  • pkg/vendoring/version/check.go (1 hunks)
  • pkg/vendoring/version/check_test.go (1 hunks)
  • pkg/vendoring/version/constraints.go (1 hunks)
  • pkg/vendoring/version/constraints_test.go (1 hunks)
  • pkg/vendoring/yaml_updater.go (1 hunks)
  • pkg/vendoring/yaml_updater_test.go (1 hunks)
  • website/blog/2025-10-21-vendor-update-and-diff.md (1 hunks)
  • website/docs/cli/commands/vendor/diff.mdx (1 hunks)
  • website/docs/cli/commands/vendor/update.mdx (1 hunks)
πŸ“ Walkthrough

Walkthrough

Replaces the legacy vendor CLI/exec with a new vendoring subsystem: adds pkg/vendoring (diff/pull/update, providers, git diff, semver constraints, YAML updater), new cmd/vendor subcommands and helpers, exports/instrumentation in internal/exec, schema/errors updates, docs, and many tests.

Changes

Cohort / File(s) Summary
Removed legacy CLI & exec
cmd/vendor_diff.go, cmd/vendor.go, cmd/vendor_pull.go, internal/exec/vendor.go, cmd/vendor_test.go
Deleted old vendor CLI commands and legacy internal execution code.
New CLI commands & wiring
cmd/vendor/vendor.go, cmd/vendor/pull.go, cmd/vendor/diff.go, cmd/vendor/update.go, cmd/vendor/utils.go, cmd/root.go
Added vendor parent command, subcommands (pull/diff/update), option structs, flag parsing/validation, shell completions, and Atmos config init helpers.
Vendoring core package
pkg/vendoring/*.go, pkg/vendoring/params.go, pkg/vendoring/model.go, pkg/vendoring/component_utils.go
Introduced pkg/vendoring with public entry points (Pull, Diff, Update), orchestration, and migrated utilities/error usage.
Provider abstraction & sources
pkg/vendoring/source/provider.go, pkg/vendoring/source/github.go, pkg/vendoring/source/git.go, pkg/vendoring/source/unsupported.go, pkg/vendoring/source/mock_provider.go
Added Provider interface, GitHub & Generic Git providers, Unsupported provider, factory and provider mocks.
Git diff & interface
pkg/vendoring/git_interface.go, pkg/vendoring/git_diff.go, pkg/vendoring/mock_git_interface.go, pkg/vendoring/git_diff_test.go
Added GitOperations abstraction, remote-diff implementation (bare repo fetch/diff), color/context handling, output writing, tests and mocks.
Version checks & constraints
pkg/vendoring/version/check.go, pkg/vendoring/version/constraints.go, pkg/vendoring/version/*_test.go
Remote tag retrieval, semver parsing, constraint resolution, selection logic and tests.
YAML updater
pkg/vendoring/yaml_updater.go, pkg/vendoring/yaml_updater_test.go
AST-based YAML lookup and in-place version updates while preserving structure/comments.
URI helpers refactor
pkg/vendoring/uri/helpers.go, pkg/vendoring/uri/helpers_test.go
Moved, renamed and exported URI helpers with perf instrumentation.
Internal exec exports & instrumentation
internal/exec/file_utils.go, internal/exec/copy_glob.go, internal/exec/oci_utils.go, internal/exec/template_utils.go, related tests
Exported helpers (RemoveTempDir, CopyToTargetWithPatterns, ProcessOciImage, GetSprigFuncMap), added perf tracking, updated callers/tests.
Vendoring package tests & migrations
pkg/vendoring/*_test.go, pkg/vender/*
Tests migrated from CLI-run to direct package calls; imports updated to new packages.
Errors, schema & modules
errors/errors.go, pkg/schema/schema.go, pkg/datafetcher/schema/vendor/package/1.0.json, go.mod
Added many vendor-related sentinel errors, introduced VendorConstraints in schema and JSON schema, added Masterminds semver v3 require.
Docs & planning
docs/prd/vendor-update.md, docs/prd/vendor-refactoring-plan.md, website/docs/cli/commands/vendor/*.mdx, website/blog/2025-10-21-vendor-update-and-diff.md
Added PRD, refactor plan, CLI docs for update/diff, and a blog post.

Sequence Diagram(s)

sequenceDiagram
    actor User as CLI User
    participant CLI as cmd/vendor/diff
    participant Parser as flag/viper parser
    participant Vend as pkg/vendoring.Diff
    participant Config as AtmosConfiguration
    participant Provider as source.Provider
    participant GitOps as GitOperations
    participant UI as Stdout/Stderr

    User->>CLI: atmos vendor diff --component foo [--from X --to Y]
    CLI->>Parser: bind & validate flags
    Parser->>Vend: Diff(atmosConfig, DiffParams)
    Vend->>Config: locate/read vendor.yaml
    Vend->>Vend: find component & extract source URI
    Vend->>Provider: GetProviderForSource(source)
    Provider->>GitOps: GetDiff / GetRemoteTags
    GitOps-->>Vend: diff bytes / error
    Vend->>UI: print diff or "No differences"
sequenceDiagram
    actor User as CLI User
    participant CLI as cmd/vendor/update
    participant Parser as flag/viper parser
    participant Vend as pkg/vendoring.Update
    participant Provider as source.Provider
    participant Version as version.constraints
    participant YAML as yaml_updater
    participant UI as Stdout

    User->>CLI: atmos vendor update --component foo --check
    CLI->>Parser: bind & validate flags
    Parser->>Vend: Update(atmosConfig, UpdateParams)
    Vend->>Provider: GetAvailableVersions(source)
    Provider->>Version: ResolveVersionConstraints
    Version-->>Vend: chosenVersion
    Vend->>UI: display check result
    alt pull/update requested
        Vend->>YAML: updateYAMLVersion(component, chosenVersion)
        YAML-->>Vend: updated file
    end

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

  • Areas to review closely:
    • pkg/vendoring/source/* (provider factory, GitHub parsing, rate-limit/auth)
    • pkg/vendoring/git_diff.go (bare-repo fetch, git command handling, ANSI/color logic)
    • pkg/vendoring/version/constraints.go (semver constraint filtering edge cases)
    • pkg/vendoring/yaml_updater.go (yaml.Node traversal and in-place updates)
    • Cross-package exported renames (internal/exec exports, uri helpers) and many updated import paths/tests

Possibly related PRs

  • cloudposse/atmos#1519 β€” overlapping vendor diff implementations and vendor CLI rework.
  • cloudposse/atmos#1639 β€” related internal/exec utilities export/rename and shared helpers used by vendoring.
  • cloudposse/atmos#1717 β€” changes to InitCliConfig/stack-validation behavior used by new vendor CLI init helpers.

Suggested reviewers

  • aknysh

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 63.84% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
βœ… Passed checks (2 passed)
Check name Status Explanation
Description Check βœ… Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check βœ… Passed The title 'Add vendor update and diff commands' directly and clearly summarizes the main changes: two new CLI commands (vendor update and vendor diff) are being added to the codebase.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❀️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot] avatar Oct 21 '25 22:10 coderabbitai[bot]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-10-21-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

github-actions[bot] avatar Oct 21 '25 22:10 github-actions[bot]

πŸ’₯ This pull request now has conflicts. Could you fix it @osterman? πŸ™

mergify[bot] avatar Oct 22 '25 03:10 mergify[bot]

πŸ’₯ This pull request now has conflicts. Could you fix it @osterman? πŸ™

mergify[bot] avatar Dec 05 '25 16:12 mergify[bot]

πŸ’₯ This pull request now has conflicts. Could you fix it @osterman? πŸ™

mergify[bot] avatar Dec 07 '25 14:12 mergify[bot]

πŸ’₯ This pull request now has conflicts. Could you fix it @osterman? πŸ™

mergify[bot] avatar Dec 10 '25 01:12 mergify[bot]

πŸ’₯ This pull request now has conflicts. Could you fix it @osterman? πŸ™

mergify[bot] avatar Dec 13 '25 00:12 mergify[bot]

πŸ’₯ This pull request now has conflicts. Could you fix it @osterman? πŸ™

mergify[bot] avatar Dec 17 '25 18:12 mergify[bot]

πŸ’₯ This pull request now has conflicts. Could you fix it @osterman? πŸ™

mergify[bot] avatar Dec 21 '25 05:12 mergify[bot]