atmos icon indicating copy to clipboard operation
atmos copied to clipboard

feat: add validation for unsupported YAML tags

Open osterman opened this issue 5 months ago • 12 comments

what

  • Add validation to detect and error on unsupported YAML tags instead of silently ignoring them
  • Create comprehensive list of all supported YAML tags for validation
  • Provide clear error messages that include the unsupported tag, file location, and list of supported tags

why

Currently, when Atmos encounters an unsupported YAML tag (like !invalid or typos like !envv instead of !env), it silently strips the tag and keeps the value. This makes it very difficult to track down configuration errors and typos in YAML files.

By explicitly validating tags and providing clear error messages, we:

  • Help users catch typos and invalid tags early
  • Prevent silent configuration failures
  • Improve debugging experience with clear error messages
  • Make the system more reliable and predictable

Example Error Messages

When an unsupported tag is encountered:

unsupported YAML tag '!invalid' found in file '/path/to/stack.yaml'. 
Supported tags are: !exec, !store, !store.get, !template, !terraform.output, 
!terraform.state, !env, !include, !include.raw, !repo-root

When there's a typo in a tag:

unsupported YAML tag '!envv' found in file '/path/to/stack.yaml'. 
Supported tags are: !exec, !store, !store.get, !template, !terraform.output, 
!terraform.state, !env, !include, !include.raw, !repo-root

Test Results

  • Added comprehensive unit tests for unsupported tag detection
  • Tests verify that unsupported tags trigger errors
  • Tests verify that valid tags continue to work correctly
  • Tests verify error message content and clarity

references

  • Closes DEV-2986
  • Linear issue: https://linear.app/cloudposse/issue/DEV-2986/atmos-should-error-for-unsupported-yaml-functions

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added explicit validation for custom YAML tags with clear errors when unsupported tags are used, including a list of supported tags.
    • Skips processing of standard YAML tags (!!) to avoid false positives.
    • Expanded and documented the set of supported tags (e.g., include-related tags recognized).
  • Tests

    • Introduced comprehensive test coverage for supported/unsupported tags, nested structures, edge cases, and performance.
    • Added multiple YAML fixtures to validate error messages, type preservation, and mixed valid/invalid scenarios.

osterman avatar Sep 25 '25 05:09 osterman

[!WARNING]

Rate limit exceeded

@osterman has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 16 minutes and 6 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 79d472baba010f19fa9ad6fa85ba282f78b1556a and 7a5572b1ecd1b785cc5a194e1725ddd06c06957e.

📒 Files selected for processing (2)
  • errors/errors.go (1 hunks)
  • pkg/utils/yaml_utils.go (3 hunks)
📝 Walkthrough

Walkthrough

Adds explicit validation and erroring for unsupported YAML tags. Introduces ErrUnsupportedYamlTag and AllSupportedYamlTags, updates processing to skip standard YAML (!!) tags, and to return errors on unknown custom tags. Extensive tests added across exec and utils, plus new YAML fixtures for negative cases.

Changes

Cohort / File(s) Summary
Errors package
errors/errors.go
Added exported error variable ErrUnsupportedYamlTag.
Exec tag processing logic
internal/exec/yaml_func_utils.go
Extended processCustomTags default path: detect !tag, verify against supported list, return input for unhandled-but-supported tags, and error for unsupported tags.
Exec tests (custom/unsupported/coverage)
internal/exec/yaml_func_process_custom_tags_test.go, internal/exec/yaml_func_tag_processing_test.go, internal/exec/yaml_func_unsupported_tags_test.go, internal/exec/yaml_func_utils_coverage_test.go
Added comprehensive tests covering supported/unsupported tags, skip behavior, recursion via processNodes, JSON handling for !template, !exec handling, and type preservation.
Config YAML processing
pkg/config/process_yaml.go
Skip standard YAML !! tags; return a descriptive error for non-standard, unsupported tags during scalar processing.
Utils core (YAML)
pkg/utils/yaml_utils.go
Added exported AllSupportedYamlTags and ErrUnsupportedYamlTag; enforced runtime validation in processCustomTags to error on unknown ! tags.
Utils tests (unsupported tags)
pkg/utils/yaml_utils_unsupported_test.go
Added tests verifying supported tag set, error messages, inclusion tags recognition, and mixed valid/invalid content handling.
Test fixtures for unsupported tags
tests/test-cases/yaml-unsupported-tags/*
Added YAML files with unsupported/mistyped tags and mixed cases for negative testing.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Y as YAML Loader
  participant C as Config Processor
  participant U as Utils (processCustomTags)

  Y->>C: Parse YAML nodes
  alt Scalar with standard tag (!!)
    C-->>Y: Skip custom processing
  else Scalar with custom tag (!...)
    C->>U: processCustomTags(value, filename, skipTags)
    alt Tag supported and handled above
      U-->>C: Processed value
    else Tag supported but unhandled here
      U-->>C: Return as-is
    else Tag unsupported
      U-->>C: ErrUnsupportedYamlTag (includes tag + supported list)
      C-->>Y: Propagate error
    end
  else Non-tag or other nodes
    C-->>Y: Return as-is / recurse
  end

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • cloudposse/atmos#865: Also updates YAML function/tag handling around store-related tags and shared utils.
  • cloudposse/atmos#1493: Touches pkg/utils/yaml_utils.go for include/raw and supported-tags validation, overlapping this change.
  • cloudposse/atmos#1506: Alters YAML tag-processing control flow in yaml_utils.go (clearing node.Tag), intersecting with current validation logic.

Suggested labels

major

Suggested reviewers

  • aknysh

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 17.14% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title concisely highlights the primary change of adding validation for unsupported YAML tags and uses clear, descriptive phrasing without vague terms.
Linked Issues Check ✅ Passed The changes implement explicit error handling for unsupported YAML tags by introducing a comprehensive supported-tags list, emitting ErrUnsupportedYamlTag with detailed messages including the tag and file location, and adding tests that confirm this behavior while preserving valid tag processing.
Out of Scope Changes Check ✅ Passed All modifications focus squarely on enhancing YAML tag processing—adding validation for unsupported tags, defining the supported-tags list, and extending tests for both supported and unsupported scenarios—so there are no unrelated or out-of-scope changes.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot] avatar Sep 25 '25 05:09 coderabbitai[bot]

Codecov Report

:x: Patch coverage is 73.33333% with 16 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 73.28%. Comparing base (827b828) to head (13c0517). :warning: Report is 17 commits behind head on main.

Files with missing lines Patch % Lines
pkg/config/process_yaml.go 16.66% 10 Missing :warning:
internal/exec/yaml_func_utils.go 85.71% 3 Missing and 3 partials :warning:

:x: Your patch check has failed because the patch coverage (73.33%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1515      +/-   ##
==========================================
+ Coverage   73.10%   73.28%   +0.17%     
==========================================
  Files         550      551       +1     
  Lines       53176    53494     +318     
==========================================
+ Hits        38876    39201     +325     
- Misses      11443    11449       +6     
+ Partials     2857     2844      -13     
Flag Coverage Δ
unittests 73.28% <73.33%> (+0.17%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
errors/errors.go 100.00% <ø> (ø)
pkg/utils/yaml_utils.go 81.94% <100.00%> (+0.84%) :arrow_up:
internal/exec/yaml_func_utils.go 91.45% <85.71%> (+1.76%) :arrow_up:
pkg/config/process_yaml.go 66.38% <16.66%> (-2.82%) :arrow_down:

... and 21 files with indirect coverage changes

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Sep 25 '25 15:09 codecov[bot]

[!WARNING]

This PR exceeds the recommended limit of 1,000 lines.

Large PRs are difficult to review and may be rejected due to their size.

Please verify that this PR does not address multiple issues. Consider refactoring it into smaller, more focused PRs to facilitate a smoother review process.

mergify[bot] avatar Sep 25 '25 19:09 mergify[bot]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-08-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

github-actions[bot] avatar Dec 08 '25 21:12 github-actions[bot]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-09-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

github-actions[bot] avatar Dec 08 '25 21:12 github-actions[bot]

💥 This pull request now has conflicts. Could you fix it @osterman? 🙏

mergify[bot] avatar Dec 09 '25 04:12 mergify[bot]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-09-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

github-actions[bot] avatar Dec 09 '25 04:12 github-actions[bot]

💥 This pull request now has conflicts. Could you fix it @osterman? 🙏

mergify[bot] avatar Dec 09 '25 05:12 mergify[bot]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-09-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

github-actions[bot] avatar Dec 09 '25 15:12 github-actions[bot]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-10-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

github-actions[bot] avatar Dec 09 '25 18:12 github-actions[bot]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-12-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

github-actions[bot] avatar Dec 10 '25 18:12 github-actions[bot]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-17-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

github-actions[bot] avatar Dec 12 '25 18:12 github-actions[bot]

💥 This pull request now has conflicts. Could you fix it @osterman? 🙏

mergify[bot] avatar Dec 17 '25 14:12 mergify[bot]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-23-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

github-actions[bot] avatar Dec 17 '25 15:12 github-actions[bot]

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

None

github-actions[bot] avatar Dec 23 '25 01:12 github-actions[bot]