MetaData formatting diferences when used in LaunchConfigurations

[cbrowningcp]

It seems there are some formatting differences when you add metadata to the whole template, versus just in a launch configuration. As an example, see this gist.

Ideally, the metadata under the launch configuration would have the same formatting as what is under the generic metadata area at the top of the template, specifically ,without the "LogicalId:" in front of the provided ID or "Props:" header.

[scrthq]

Hey @cbrowningcp - I should probably clarify that the New-Vapor* functions are typically reserved for those top-level nodes in the template. I should also expand on the input type validation against the Metadata property on Resource Types to provide better contextual awareness on this (and/or add handling in the event that New-VaporMetadata is used so it resolves as expected.

Currently, the Metadata property on individual resource types expects a PSCustomObject. The input to that object is translated literally to the resulting template, so using New-VaporMetadata will resolve to the actual object that New-VaporMetadata produces vs the massaged object that is visible when you use it on the top-level Metadata node.

Here's the actual parameter and it's ValidateScript attribute for visibility:

[parameter(Mandatory = $false)]
[ValidateScript( {
        $allowedTypes = "System.Management.Automation.PSCustomObject"
        if ([string]$($_.PSTypeNames) -match "($(($allowedTypes|ForEach-Object{[RegEx]::Escape($_)}) -join '|'))") {
            $true
        }
        else {
            $PSCmdlet.ThrowTerminatingError((New-VSError -String "The UpdatePolicy parameter only accepts the following types: $($allowedTypes -join ", "). The current types of the value are: $($_.PSTypeNames -join ", ")."))
        }
    })]
$Metadata

Here's how I would approach your example of adding the same cfn-init metadata directly to the resource itself (casting to PSCustomObject is currently needed, will work on enabling more types there like Hashtables or OrderedDictionary's as well):

$t = Initialize-Vaporshell "testing resource metadata"
$asLC = New-VSAutoScalingLaunchConfiguration "LaunchConfigurationApplicationOnDemand" -ImageId 'ami-1234567' -InstanceType 't2.micro' -Metadata ([PSCustomObject]@{
    'AWS::CloudFormation::Init' = @{
        configSets  = @{
            WindowsConfig = @(
                'SetTimezone'
                'ConfigureAWSTools'
                'InstallDeploymentBundle'
                'SetEnvironmentVariables'
                'SetEnvironmentVariables'
            )
        }
        SetTimezone = @{
            commmands = [ordered]@{
                '01-set_timezone' = [ordered]@{
                    command                  = 'tzutil /s "Eastern Standard Time"'
                    waitUntilAfterCompletion = 0
                }
            }
        }
    }
})
$t.AddResource($asLc)
$t.ToYaml()

Resulting YAML:

Description: testing resource metadata
Resources:
  LaunchConfigurationApplicationOnDemand:
    Type: AWS::AutoScaling::LaunchConfiguration
    Properties:
      ImageId: ami-1234567
      InstanceType: t2.micro
    Metadata:
      AWS::CloudFormation::Init:
        configSets:
          WindowsConfig:
            - SetTimezone
            - ConfigureAWSTools
            - InstallDeploymentBundle
            - SetEnvironmentVariables
            - SetEnvironmentVariables
        SetTimezone:
          commmands:
            '01-set_timezone':
              command: tzutil /s "Eastern Standard Time"
              waitUntilAfterCompletion: 0

---

Hopefully that makes sense! I appreciate you raising these concerns and will get some clarification implemented in the code to assist with that, for sure 😃

[cbrowningcp]

Yes, this does make sense. Thank you for the assistance once again.

If I could ask another question - I am having trouble getting add-fnsub to format in the form of a literal block, like this:

content: !Sub |
    [cfn-auto-reloader-hook]
    triggers=post.update
    path=Resources.LaunchConfigurationApplication.Metadata.AWS::CloudFormation::Init
    action=c:\\"Program Files"\\Amazon\\cfn-bootstrap\\cfn-init.exe -v --stack ${AWS::StackName} --resource LaunchConfigurationApplication --region ${AWS::Region}

my attempts produce something like

c:\\cfn\\hooks.d\\cfn-auto-reloader.conf:
                content: "!Sub | \r\n   [cfn-auto-reloader-hook]\r\
                  \n                                    triggers=post.update\r\n \
                  \                                   path=Resources.LaunchConfigurationApplication.Metadata.AWS::CloudFormation::Init\r\
                  \n                                    action=c:\\\\\"Program Files\"\
                  \\\\Amazon\\\\cfn-bootstrap\\\\cfn-init.exe -v --stack $_AWSStackName\
                  \ --resource LaunchConfigurationApplicationOnDemand --region $_AWSRegion"

or

content: !Sub "[main]\rstack=AWS::StackId\rregion=AWS::Region"

when using literal whitespace or escaped characters, respectively. Is there some way to manipulate the sub function to perform this or another method to accomplish it?

[scrthq]

hey @cbrowningcp - I'm thinking this may be an oddity with necessiting converting to JSON from a PSObject first, then converting to YAML from there. Based on the conversion, it seems like the resulting template still works but it's not aesthetically ideal? Or does it also not work the same for you?

Going to do some testing and see if I can replicate it outside of PowerShell just using cfn-flip

[cbrowningcp]

@scrthq, Sorry it took so long for me to get back to you. This setup does not appear to work fo rme - it causes some formatting errors when pulling the metada down, such as

"command": "tzutil /s \\\"Eastern Standard Time\\\"",

or

"command": "powershell.exe -ExecutionPolicy Unrestricted c:\\\\deploy\\\\app\\\\deploy.ps1",

with a hilarious amount of \'s. This might be solved by formatting my input differently, but I have not been able to get it in a good spot yet. I was able to improve the layour some by forgoing the use of "new-vapormertadata" in favor of something like

$CloudformationAuthenticationMetaData = [PSCustomObject]@{
    "AWS::CloudFormation::Authentication" = @{
                          S3AccessCreds = @{
                              type = "S3"
                              roleName = (Add-FnRef -Ref "RoleApplication")
                              buckets = (Add-FnRef -Ref "ArtifactsBucket")
                          }
                      }
                    }

and for the cfn-hup

 "c:\\cfn\\cfn-hup.conf" = @{
                                    content = "Fn::Sub | ", (Add-FnJoin -Delimiter "\n" -ListOfValues ("[main]","stack=`${$_AWSStackId}","region=`${$_AWSRegion}"))
                                }

I am beginning to suspect that this is something in cfn-flip, as you stated, but don't have the necessary experience to pinpoint any specific problems there.

[scrthq]

hey @cbrowningcp - this looks like it's just going wild with escaping the two special characters in that command, \ and ". I see this a lot with Chef/Ruby while dealing with anything Windows path related since backslash is the escape character for most languages outside of PowerShell, lol.

Have you tried using the normal PowerShell new-line for the delimiter instead of \n?

"c:\\cfn\\cfn-hup.conf" = @{
    content = "Fn::Sub | ", (Add-FnJoin -Delimiter "`n" -ListOfValues ("[main]","stack=`${$_AWSStackId}","region=`${$_AWSRegion}"))
}

[cbrowningcp]

@scrthq - sorry i have been away for the past couple of weeks. I did in fact try the powershell newline, but it had more or less the same effect - just printing "`n" instead of a newline character.

[scrthq]

Here's the raw conversion for just that sample section using cfn-flip; it looks to be closer to what you'd expect. Still not perfect, but the conversion of new lines to \n appears to be happening within cf-flip. I'm checking if there's anything pertinent happening within VaporShell in between the ConvertTo/From-Json calls while it's a PSObject.

<#[PS 6.2]#> @'
content: !Sub |
    [cfn-auto-reloader-hook]
    triggers=post.update
    path=Resources.LaunchConfigurationApplication.Metadata.AWS::CloudFormation::Init
    action=c:\\"Program Files"\\Amazon\\cfn-bootstrap\\cfn-init.exe -v --stack ${AWS::StackName} --resource LaunchConfigurationApplication --region ${AWS::Region}
'@ | cfn-flip | ConvertFrom-Json | ConvertTo-Json | cfn-flip

content: !Sub "[cfn-auto-reloader-hook]\ntriggers=post.update\npath=Resources.LaunchConfigurationApplication.Metadata.AWS::CloudFormation::Init\n\
  action=c:\\\\\"Program Files\"\\\\Amazon\\\\cfn-bootstrap\\\\cfn-init.exe -v --stack\
  \ ${AWS::StackName} --resource LaunchConfigurationApplication --region ${AWS::Region}\n"

[scrthq]

@cbrowningcp - it looks like it's handling it as it should actually; a single, multi-line string with line breaks in it (due to using a pipe | to break into the first line. If you were to use a > instead of the pipe, it would join the strings with a space (no line breaks). Since YAML parses JSON fine and understands the \n as a new line in the same string, the resulting YAML may look different, but the expected behavior should be the same

Example showing the different operators:

<#[PS 6.2]#> @'
content: |
    testing
    first
    second
    third
'@ | cfn-flip 

{
    "content": "testing\nfirst\nsecond\nthird\n"
}

<#[PS 6.2]#> @'
content: >
    testing
    first
    second
    third
'@ | cfn-flip

{
    "content": "testing first second third\n"
}

[scrthq]

alright, after spending a bit more time on this, this would be the correct approach (IMO), as it is handling it the same way as it would be handled on YAML/JSON at the end of it and joining an array of strings with a newline char then passing that off as the main parameter to Fn::Sub:

<#[PS 6.2]#> @{
    content = (
        Add-FnSub -String (
            @(
                "[cfn-auto-reloader-hook]"
                "triggers=post.update"
                "path=Resources.LaunchConfigurationApplication.Metadata.AWS::CloudFormation::Init"
                "action=c:\\`"Program Files`"\\Amazon\\cfn-bootstrap\\cfn-init.exe -v --stack `${AWS::StackName} --resource LaunchConfigurationApplication --region `${AWS::Region}"
            ) -join "`n"
        )
    )
} | ConvertTo-Json | cfn-flip

content: !Sub "[cfn-auto-reloader-hook]\ntriggers=post.update\npath=Resources.LaunchConfigurationApplication.Metadata.AWS::CloudFormation::Init\n\
  action=c:\\\\\"Program Files\"\\\\Amazon\\\\cfn-bootstrap\\\\cfn-init.exe -v --stack\
  \ ${AWS::StackName} --resource LaunchConfigurationApplication --region ${AWS::Region}"

Still has the odd line breaks and escaped white space, but should functionally be the same.