feat: implement !append YAML function for list concatenation
what
- Implements the
!appendYAML function that allows fine-grained control over list merging behavior in Atmos stack configurations - Lists tagged with
!appendwill be concatenated with base values instead of replaced - Adds comprehensive unit tests and integration test fixtures
why
- Resolves the ongoing challenge of needing to concatenate lists on a case-by-case basis
- Currently, users have to fall back to using maps instead of lists when they need append behavior
- This is particularly important for fields like
depends_onwhere appending is often the desired behavior rather than replacement - The
!appendtag provides opt-in, per-field control that works alongside the globallist_merge_strategysetting
Key Features
-
Opt-in behavior: Only lists explicitly tagged with
!appenduse append mode -
Works alongside global settings: The
!appendtag works independently of the globallist_merge_strategysetting - Nested support: Works with deeply nested configurations
- Backward compatible: No impact on existing configurations without the tag
Example Usage
# base.yaml
components:
terraform:
eks:
settings:
depends_on:
- vpc
- iam-role
# override.yaml
components:
terraform:
eks:
settings:
depends_on: !append # This tag indicates append mode
- rds
- elasticache
# Result: depends_on = [vpc, iam-role, rds, elasticache]
Testing
- β All unit tests pass
- β Build succeeds without errors
- β Linting passes with no issues
- β Code follows Atmos conventions and patterns
references
- Closes #2980
- Linear issue: DEV-2980
π€ Generated with Claude Code
Summary by CodeRabbit
-
New Features
- Added support for a !append YAML function to append items to lists during configuration merging (per-field, preserves order, works with nested structures and global list-merge strategies).
-
Tests
- Added unit tests for append-tag utilities and merge behavior, plus integration-style test cases and expected results demonstrating !append semantics.
-
Documentation
- Added docs and examples explaining !append usage, behavior, and interactions with existing merge settings.
[!WARNING]
Rate limit exceeded
@osterman has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 5 minutes and 23 seconds before requesting another review.
β How to resolve this issue?
After the wait time has elapsed, a review can be triggered using the
@coderabbitai reviewcommand as a PR comment. Alternatively, push new commits to this PR.We recommend that you space out your commits to avoid hitting the rate limit.
π¦ How do rate limits work?
CodeRabbit enforces hourly rate limits for each developer per organization.
Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.
Please see our FAQ for further information.
π₯ Commits
Reviewing files that changed from the base of the PR and between ff9bca723c7fd7039db569c233ada54eaf6b32f5 and a0d392e1a2bc9de97477a361d25f8f40ea28b6d5.
β Files ignored due to path filters (2)
website/package-lock.jsonis excluded by!**/package-lock.jsonwebsite/pnpm-lock.yamlis excluded by!**/pnpm-lock.yamlπ Files selected for processing (1)
website/package.json(1 hunks)
π Walkthrough
Walkthrough
Adds per-field YAML !append handling: YAML loader records tagged sequences as append-wrapped values; merge pre-processing consumes those wrappers to concatenate lists into merged maps. New helpers, tests, docs, test fixtures, a CLI test helper, and one new exported error were added. No public API signatures were changed.
Changes
| Cohort / File(s) | Summary |
|---|---|
YAML processingpkg/config/process_yaml.go |
Decode YAML sequence nodes tagged !append, decode each list item, wrap with append metadata and store in Viper at the current path, then clear the node tag to avoid re-processing. |
Merge logicpkg/merge/merge.go |
Add pre-merge pass processAppendTags that walks input maps, detects append-wrapped lists and nested maps, and merges/concatenates lists into the running merged map before the normal merge flow. New internal helpers: processAppendTags, processValue, processAppendList, processNestedMap. No change to public Merge signature. |
Merge testspkg/merge/merge_append_test.go |
New unit tests for per-field !append semantics and interaction with a global list-merge strategy across multiple input documents. |
Append utilitiespkg/utils/yaml_func_append.go, pkg/utils/yaml_utils.go |
New helpers and constant: ProcessAppendTag, IsAppendTag, HasAppendTag, ExtractAppendListValue, WrapWithAppendTag; AtmosYamlFuncAppend = "!append". |
Utility testspkg/utils/yaml_func_append_test.go |
Unit tests for tag detection, presence checks, extraction, wrapping, and ProcessAppendTag validation. |
E2E test casestests/test-cases/append-function/base.yaml, tests/test-cases/append-function/override-with-append.yaml, tests/test-cases/append-function/expected-result.yaml |
Added base config, override using !append, and expected merged result demonstrating appended lists. |
Docswebsite/docs/functions/yaml/append.mdx, website/docs/functions/yaml/index.mdx |
New documentation and examples describing !append usage, semantics, and interaction with global list merge strategies. |
Errorserrors/errors.go |
Added exported error ErrRefusingToDeleteSymlink = errors.New("refusing to delete symbolic link"). |
Test harnesstests/cli_test.go |
Added validateAtmosBinary(repoRoot string) (string, string) and integrated it into TestMain to surface a skip reason or the binary path for CLI-related tests. |
Sequence Diagram(s)
sequenceDiagram
autonumber
participant YAML as YAML Loader
participant PY as process_yaml.go
participant U as utils (append helpers)
participant V as Viper (config store)
YAML->>PY: Parse node
alt node is sequence with tag "!append"
PY->>PY: Decode each list item
PY->>U: WrapWithAppendTag(items)
U-->>PY: {"__atmos_append__": items}
PY->>V: Set(path, wrappedValue)
PY->>PY: Clear node.Tag
else other node
PY->>V: Set(path, decoded value)
end
sequenceDiagram
autonumber
participant M as MergeWithOptions
participant P as processAppendTags
participant U as utils (append helpers)
participant S as MergedState
M->>P: Pre-process input map
loop each key,value
P->>P: processValue(key, value)
alt value is append-wrapped list
P->>U: ExtractAppendListValue(value)
U-->>P: items
P->>S: Append items into S[key] (create if missing)
else value is nested map
P->>P: processAppendTags(nestedMap)
P->>S: Merge/assign nested map into S[key]
else
P->>S: Assign/override S[key] = value
end
end
M->>M: Continue normal merge flow with processed map
Estimated code review effort
π― 3 (Moderate) | β±οΈ ~25 minutes
Possibly related PRs
- cloudposse/atmos#1506 β modifies YAML custom-tag handling and clears node tags after processing; strongly related to the
!appendtag implementation.
Pre-merge checks and finishing touches
β Failed checks (1 warning)
| Check name | Status | Explanation | Resolution |
|---|---|---|---|
| Out of Scope Changes Check | β οΈ Warning | The PR also adds an unrelated file-operation error constant (ErrRefusingToDeleteSymlink) and CLI binary validation logic in tests/cli_test.go, which are not part of the !append feature scope defined in DEV-2980. |
Please remove or relocate the new symlink error constant and CLI test helper changes into a separate pull request so that this PR focuses solely on the !append YAML functionality. |
β Passed checks (4 passed)
| Check name | Status | Explanation |
|---|---|---|
| Description Check | β Passed | Check skipped - CodeRabbitβs high-level summary is enabled. |
| Title Check | β Passed | The title clearly and concisely describes the primary purpose of the changeset, which is adding the !append YAML function for list concatenation, matching the main feature implemented in the pull request. |
| Linked Issues Check | β Passed | The pull request fully implements the per-field !append tag mechanism described in DEV-2980 by detecting and wrapping append-tagged lists in the YAML parser, merging them correctly in the merge logic, providing utility helpers and unit tests, and documenting usage, all of which satisfy the linked issueβs requirements. |
| Docstring Coverage | β Passed | Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. |
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands and usage tips.
Codecov Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 72.84%. Comparing base (8e75aa1) to head (3e8e709).
Additional details and impacted files
@@ Coverage Diff @@
## main #1513 +/- ##
==========================================
+ Coverage 72.81% 72.84% +0.02%
==========================================
Files 530 531 +1
Lines 50667 50715 +48
==========================================
+ Hits 36892 36941 +49
+ Misses 11002 11000 -2
- Partials 2773 2774 +1
| Flag | Coverage Ξ | |
|---|---|---|
| unittests | 72.84% <100.00%> (+0.02%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
| Files with missing lines | Coverage Ξ | |
|---|---|---|
| errors/errors.go | 100.00% <ΓΈ> (ΓΈ) |
|
| pkg/merge/merge.go | 86.71% <100.00%> (+1.49%) |
:arrow_up: |
| pkg/utils/yaml_func_append.go | 100.00% <100.00%> (ΓΈ) |
|
| pkg/utils/yaml_utils.go | 81.10% <ΓΈ> (ΓΈ) |
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
π₯ This pull request now has conflicts. Could you fix it @osterman? π
[!WARNING]
This PR exceeds the recommended limit of 1,000 lines.
Large PRs are difficult to review and may be rejected due to their size.
Please verify that this PR does not address multiple issues. Consider refactoring it into smaller, more focused PRs to facilitate a smoother review process.
π₯ This pull request now has conflicts. Could you fix it @osterman? π
[!WARNING]
Changelog Entry Required
This PR is labeled
minorormajorbut doesn't include a changelog entry.Action needed: Add a new blog post in
website/blog/to announce this change.Example filename:
website/blog/2025-12-08-feature-name.mdxAlternatively: If this change doesn't require a changelog entry, remove the
minorormajorlabel.
[!WARNING]
Changelog Entry Required
This PR is labeled
minorormajorbut doesn't include a changelog entry.Action needed: Add a new blog post in
website/blog/to announce this change.Example filename:
website/blog/2025-12-09-feature-name.mdxAlternatively: If this change doesn't require a changelog entry, remove the
minorormajorlabel.
π₯ This pull request now has conflicts. Could you fix it @osterman? π
[!WARNING]
Changelog Entry Required
This PR is labeled
minorormajorbut doesn't include a changelog entry.Action needed: Add a new blog post in
website/blog/to announce this change.Example filename:
website/blog/2025-12-09-feature-name.mdxAlternatively: If this change doesn't require a changelog entry, remove the
minorormajorlabel.
[!WARNING]
Changelog Entry Required
This PR is labeled
minorormajorbut doesn't include a changelog entry.Action needed: Add a new blog post in
website/blog/to announce this change.Example filename:
website/blog/2025-12-17-feature-name.mdxAlternatively: If this change doesn't require a changelog entry, remove the
minorormajorlabel.
[!WARNING]
Changelog Entry Required
This PR is labeled
minorormajorbut doesn't include a changelog entry.Action needed: Add a new blog post in
website/blog/to announce this change.Example filename:
website/blog/2025-12-18-feature-name.mdxAlternatively: If this change doesn't require a changelog entry, remove the
minorormajorlabel.
π₯ This pull request now has conflicts. Could you fix it @osterman? π
[!WARNING]
Changelog Entry Required
This PR is labeled
minorormajorbut doesn't include a changelog entry.Action needed: Add a new blog post in
website/blog/to announce this change.Example filename:
website/blog/2025-12-22-feature-name.mdxAlternatively: If this change doesn't require a changelog entry, remove the
minorormajorlabel.
[!WARNING]
This PR exceeds the recommended limit of 1,000 lines.
Large PRs are difficult to review and may be rejected due to their size.
Please verify that this PR does not address multiple issues. Consider refactoring it into smaller, more focused PRs to facilitate a smoother review process.
Dependency Review
The following issues were found:- β 0 vulnerable package(s)
- β 3 package(s) with incompatible licenses
- β 1 package(s) with invalid SPDX license definitions
- β οΈ 7 package(s) with unknown licenses.
License Issues
website/pnpm-lock.yaml
| Package | Version | License | Issue Type |
| dompurify | 3.3.1 | (Apache-2.0 AND GPL-2.0-only AND MPL-2.0 AND MS-PL) OR (Apache-2.0 AND GPL-2.0-only AND MPL-2.0) | Incompatible License |
| node-forge | 1.3.3 | (BSD-3-Clause AND GPL-1.0-or-later AND GPL-2.0 AND GPL-2.0-only) OR (BSD-3-Clause AND GPL-1.0-or-later AND GPL-2.0-only) | Incompatible License |
| sax | 1.4.3 | BlueOak-1.0.0 | Incompatible License |
| posthog-js | 1.302.2 | SEE LICENSE IN LICENSE | Invalid SPDX License |
| @csstools/postcss-position-area-property | 1.0.0 | Null | Unknown License |
| @csstools/postcss-system-ui-font-family | 1.0.0 | Null | Unknown License |
| @docsearch/core | 4.3.1 | Null | Unknown License |
| @ai-sdk/gateway | 2.0.18 | Null | Unknown License |
| @ai-sdk/react | 2.0.109 | Null | Unknown License |
| @posthog/core | 1.7.1 | Null | Unknown License |
| ai | 5.0.108 | Null | Unknown License |
Allowed Licenses:
MIT, MIT-0, Apache-2.0, BSD-2-Clause, BSD-2-Clause-Views, BSD-3-Clause, ISC, MPL-2.0, 0BSD, Unlicense, CC0-1.0, CC-BY-3.0, CC-BY-4.0, CC-BY-SA-3.0, Python-2.0, OFL-1.1, LicenseRef-scancode-generic-cla, LicenseRef-scancode-unknown-license-reference, LicenseRef-scancode-unicode, LicenseRef-scancode-google-patent-license-golang
Scanned Files
- website/pnpm-lock.yaml