atlantis icon indicating copy to clipboard operation
atlantis copied to clipboard

Cloudformation stacks in terraform containing list items can show up as line removals due to the diff replacement regex

Open nitrocode opened this issue 7 months ago • 1 comments

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Overview of the Issue

Cloudformation stacks in terraform containing list items can show up as line removals due to the diff replacement regex

e.g. This yaml from a wiz stackset

                  ManagedPolicyArns:
                    - Fn::Sub: arn:${AWS::Partition}:iam::aws:policy/job-function/ViewOnlyAccess
                    - Fn::Sub: arn:${AWS::Partition}:iam::aws:policy/SecurityAudit

gets converted to this incorrectly

                  ManagedPolicyArns:
-                     Fn::Sub: arn:${AWS::Partition}:iam::aws:policy/job-function/ViewOnlyAccess
-                     Fn::Sub: arn:${AWS::Partition}:iam::aws:policy/SecurityAudit

However, this gets converted semi-correctly

          -   isOrg:
          -     AllowedValues:
          -       - Enabled
          -       - Disabled
          -     Default: Disabled
          -     Description: Enable org deploy
          -     Type: String
          -   orgId:
          -     Default: ""
          -     Description: The OU ID of the AWS Organization where we should deploy, preferably the root OU. This value is mandatory when isOrg is Enabled. You can submit one value, or a space separated list of multiple OUs
          -     Type: String
          +   # isOrg:
          +   #   AllowedValues:
          +   #     - Enabled
          +   #     - Disabled
          +   #   Default: Disabled
          +   #   Description: Enable org deploy
          +   #   Type: String
          +   # orgId:
          +   #   Default: ""
          +   #   Description: The OU ID of the AWS Organization where we should deploy, preferably the root OU. This value is mandatory when isOrg is Enabled. You can submit one value, or a space separated list of multiple OUs
          +   #   Type: String

to this

-             isOrg:
          +   # isOrg:
-               AllowedValues:
+             #   AllowedValues:
          -       - Enabled
          +   #     - Enabled
          -       - Disabled
          +   #     - Disabled
-               Default: Disabled
+             #   Default: Disabled
-               Description: Enable org deploy
+             #   Description: Enable org deploy
-               Type: String
+             #   Type: String
-             orgId:
          +   # orgId:
-               Default: ""
+             #   Default: ""
-               Description: The OU ID of the AWS Organization where we should deploy, preferably the root OU. This value is mandatory when isOrg is Enabled. You can submit one value, or a space separated list of multiple OUs
+             #   Description: The OU ID of the AWS Organization where we should deploy, preferably the root OU. This value is mandatory when isOrg is Enabled. You can submit one value, or a space separated list of multiple OUs
-               Type: String
+             #   Type: String

Reproduction Steps

I'm having trouble reproducing this

Here is the related code https://github.com/runatlantis/atlantis/blob/a7c712b921d9c4de15bd70d035c95be9a667a4c2/server/events/models/models.go#L428-L441

Here is the regexr reproduction using

  1. DiffKeywordRegex https://regexr.com/8fdf9
  2. DiffListRegex https://regexr.com/8fden

I believe the above contents are getting picked up by the former.

Logs

Environment details

  • Atlantis version: 0.34.0
  • Deployment method: eks
  • If not running the latest Atlantis version have you tried to reproduce this issue on the latest version:
  • Atlantis flags: n/a

options

  1. Improve the regex to exclude yaml lists somehow
  2. Exempt aws_cloudformation_stack resources from the diff conversion
  3. Use hcl instead of diff and don't modify the terraform output at all

Additional Context

nitrocode avatar Jun 13 '25 15:06 nitrocode

This issue happens because the regex patterns used by Atlantis to format diffs are tuned for simple HCL or plain text, and can overmatch or misinterpret YAML/CloudFormation list structures—especially when list items use constructs like Fn::Sub. The key patterns are diffKeywordRegex and diffListRegex, which try to match lines starting with diff symbols and certain value assignments, but they don't always handle YAML lists or indentation correctly. This can cause list items to show up as removals rather than as changed or added lines, especially for CloudFormation blocks with nested or complex list items [source].

To address this, you could experiment with refining the diffListRegex to be less greedy and more context-aware—possibly by matching indentation and optional list indicators, or by pre-processing YAML blocks before applying the regex. Another approach is to add detection for CloudFormation-style YAML blocks and handle them differently in the diff formatting logic. There haven't been recent changes to this logic, so the behavior is likely inherent to the current implementation.

If you're having trouble reproducing the issue, it may depend on the exact formatting of the Terraform plan output and how Atlantis parses it. Try to ensure your test cases match the indentation and structure of the problematic YAML.

Let me know if you want to discuss specific regex tweaks or see example adjustments!

To reply, just mention @dosu.


How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Join Discord Share on X

dosubot[bot] avatar Jun 13 '25 15:06 dosubot[bot]

@dosu Can you solve this one?

fzipi avatar Sep 02 '25 15:09 fzipi