atmos icon indicating copy to clipboard operation
atmos copied to clipboard

refactor: Replace CSV-based YAML function argument parsing

Open osterman opened this issue 3 months ago β€’ 7 comments

what

  • Eliminated painful CSV-based quote escaping in !terraform.state and !terraform.output functions
  • Implemented a simple, grammar-aware function argument parser that exploits semantic properties of component/stack naming
  • Updated documentation with clean syntax examples and YAML block scalar demonstrations

why

The CSV parser was the wrong tool for this job. It required double-quote escaping ("") to include JSON literals in expressions, creating ugly, hard-to-read YAML. The new parser uses a lightweight heuristic: component and stack names never contain spaces and never start with special characters (., |, [, {, ", '), while expressions always do. This enables clean, readable syntax without escaping.

references

  • Addresses usability pain point for complex YAML function expressions
  • Part of ongoing effort to improve YAML function syntax

Summary by CodeRabbit

  • New Features

    • Simplified YAML function syntax for terraform operationsβ€”no more complex quote escaping required.
    • Support for YAML block scalars to format complex expressions for better readability.
  • Documentation

    • Updated function guides with cleaner syntax examples and best practices.
    • Added guidance on block scalar usage for long expressions.
    • Published documentation on syntax improvements and backward compatibility.

✏️ Tip: You can customize this high-level summary in your review settings.

osterman avatar Dec 04 '25 03:12 osterman

Dependency Review

βœ… No vulnerabilities or license issues found.

Scanned Files

None

github-actions[bot] avatar Dec 04 '25 03:12 github-actions[bot]

[!WARNING]

Changelog Entry Required

This PR is labeled minor or major but doesn't include a changelog entry.

Action needed: Add a new blog post in website/blog/ to announce this change.

Example filename: website/blog/2025-12-04-feature-name.mdx

Alternatively: If this change doesn't require a changelog entry, remove the minor or major label.

github-actions[bot] avatar Dec 04 '25 03:12 github-actions[bot]

[!WARNING]

Rate limit exceeded

@osterman has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 8 minutes and 30 seconds before requesting another review.

βŒ› How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

πŸ“₯ Commits

Reviewing files that changed from the base of the PR and between 5336979f8b49bdb6d638651e962fed75a8288fcb and 29e17e90ee37c6122c77857263b9ad53d740060f.

πŸ“’ Files selected for processing (2)
  • website/docs/functions/yaml/terraform.output.mdx (2 hunks)
  • website/docs/functions/yaml/terraform.state.mdx (2 hunks)
πŸ“ Walkthrough

Walkthrough

Introduces a new parser utility in pkg/function/parser.go to standardize YAML function argument parsing, then refactors yaml_func_terraform_output.go and yaml_func_terraform_state.go to use this parser instead of manual string splitting. Updates test fixtures and documentation to reflect cleaner single-quoted syntax.

Changes

Cohort / File(s) Summary
New Parser Implementation
pkg/function/parser.go, pkg/function/parser_test.go
Introduces ParseArgs function to parse YAML function arguments into component, stack, and expression; includes helper isExpressionStart for detecting expression-start characters; comprehensive test suite covering backward compatibility, new syntax, edge cases, and YAML block scalars.
YAML Function Refactors
internal/exec/yaml_func_terraform_output.go, internal/exec/yaml_func_terraform_state.go
Replaces manual string-splitting with fn.ParseArgs for argument parsing; adds explicit validation for missing component and output; defaults stack to currentStack when not provided; moves dependency tracking inline with deferred context cleanup.
Test Cleanup
internal/exec/yaml_func_utils_context_test.go
Minor formatting: removes extra blank lines after error assertions; no logic changes.
Test Fixture Syntax Updates
tests/fixtures/scenarios/atmos-terraform-output-yaml-function/stacks/deploy/nonprod.yaml, tests/fixtures/scenarios/atmos-terraform-state-yaml-function/stacks/deploy/nonprod.yaml, tests/fixtures/scenarios/atmos-terraform-state-yaml-function/stacks/deploy/prod.yaml
Updates terraform output and state references from escaped double-quoted syntax to cleaner single-quoted syntax with unescaped inline JSON literals.
Documentation
website/docs/functions/yaml/terraform.output.mdx, website/docs/functions/yaml/terraform.state.mdx, website/blog/2025-12-03-yaml-function-clean-syntax.mdx
Replaces escaping guidance with "Clean Syntax" tips; adds examples using single-quoted YAML expressions; introduces new section on YAML block scalars for complex expressions; includes blog post detailing syntax improvements.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~30 minutes

  • Parser logic introduces new string-processing utility with edge cases that need validation
  • Dual refactors of terraform YAML functions follow consistent pattern but require verification of integration
  • Changes touch core argument-parsing infrastructure; test fixture updates validate syntax compatibility
  • Documentation aligns well with code changes
  • Areas requiring attention: ParseArgs behavior with whitespace and edge-case inputs; integration of deferred context cleanup in both YAML functions; backward compatibility verification with existing simple usages

Possibly related PRs

  • cloudposse/atmos#944 β€” Modifies same yaml_func_terraform_output.go file and !terraform.output parsing logic
  • cloudposse/atmos#810 β€” Refactors the same terraform YAML functions with new ParseArgs parser and validation handling
  • cloudposse/atmos#1708 β€” Overlaps on terraform.state/terraform.output processing and resolution-context dependency-tracking mechanics

Suggested reviewers

  • aknysh

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 32.26% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
βœ… Passed checks (2 passed)
Check name Status Explanation
Description Check βœ… Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check βœ… Passed The title accurately captures the main refactor: replacing CSV-based parsing with a cleaner argument parser for YAML functions.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❀️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot] avatar Dec 04 '25 04:12 coderabbitai[bot]

[!WARNING]

Rate limit exceeded

@osterman has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 24 minutes and 19 seconds before requesting another review.

βŒ› How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

πŸ“₯ Commits

Reviewing files that changed from the base of the PR and between 63866570aad405bb644adc616c56c9d34d67e326 and 61db2034a16fceb101acf5d344020b810e399e1d.

πŸ“’ Files selected for processing (10)
  • internal/exec/yaml_func_terraform_output.go (2 hunks)
  • internal/exec/yaml_func_terraform_state.go (2 hunks)
  • pkg/function/parser.go (1 hunks)
  • pkg/function/parser_test.go (1 hunks)
  • tests/fixtures/scenarios/atmos-terraform-output-yaml-function/stacks/deploy/nonprod.yaml (1 hunks)
  • tests/fixtures/scenarios/atmos-terraform-state-yaml-function/stacks/deploy/nonprod.yaml (1 hunks)
  • tests/fixtures/scenarios/atmos-terraform-state-yaml-function/stacks/deploy/prod.yaml (1 hunks)
  • website/blog/2025-12-03-yaml-function-clean-syntax.mdx (1 hunks)
  • website/docs/functions/yaml/terraform.output.mdx (3 hunks)
  • website/docs/functions/yaml/terraform.state.mdx (3 hunks)
✨ Finishing touches
  • [ ] πŸ“ Generate docstrings
πŸ§ͺ Generate unit tests (beta)
  • [ ] Create PR with unit tests
  • [ ] Post copyable unit tests in a comment
  • [ ] Commit unit tests in branch osterman/yaml-func-quoting

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❀️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot] avatar Dec 04 '25 04:12 coderabbitai[bot]

Codecov Report

:x: Patch coverage is 73.00000% with 27 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 72.81%. Comparing base (8e75aa1) to head (f37e03e). :warning: Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
internal/exec/yaml_func_terraform_output.go 59.09% 5 Missing and 4 partials :warning:
internal/exec/yaml_func_terraform_state.go 35.71% 5 Missing and 4 partials :warning:
internal/exec/yaml_func_utils.go 81.08% 5 Missing and 2 partials :warning:
pkg/function/parser.go 92.30% 1 Missing and 1 partial :warning:

:x: Your patch check has failed because the patch coverage (73.00%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1834      +/-   ##
==========================================
- Coverage   72.81%   72.81%   -0.01%     
==========================================
  Files         530      531       +1     
  Lines       50667    50677      +10     
==========================================
+ Hits        36892    36898       +6     
+ Misses      11002    10997       -5     
- Partials     2773     2782       +9     
Flag Coverage Ξ”
unittests 72.81% <73.00%> (-0.01%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Ξ”
internal/exec/yaml_processor.go 100.00% <100.00%> (ΓΈ)
pkg/function/parser.go 92.30% <92.30%> (ΓΈ)
internal/exec/yaml_func_utils.go 90.10% <81.08%> (-5.13%) :arrow_down:
internal/exec/yaml_func_terraform_output.go 74.35% <59.09%> (-10.55%) :arrow_down:
internal/exec/yaml_func_terraform_state.go 72.97% <35.71%> (-16.16%) :arrow_down:

... and 3 files with indirect coverage changes

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Dec 04 '25 04:12 codecov[bot]

πŸ’₯ This pull request now has conflicts. Could you fix it @osterman? πŸ™

mergify[bot] avatar Dec 09 '25 04:12 mergify[bot]

πŸ’₯ This pull request now has conflicts. Could you fix it @osterman? πŸ™

mergify[bot] avatar Dec 09 '25 05:12 mergify[bot]