Force resolve dynamic secret references in change set creation

[mschweitzer-sd]

1. Title

Force resolve dynamic secret references in change set creation

2. Scope of request

Currently, if you have a dynamic reference to a secret, and don't specify the version, change set creation fails with the error message stating that there are no changes to the template, even if the secret value changed.

It would be great if we could pass either a flag or a capability to change set creation to force CloudFormation to detect and re-resolve dynamic references to secrets when you want the latest version of the secret.

We use a lot of CodePipeline, and for us, it is especially necessary for this to be supported by the CHANGE_SET_REPLACE CloudFormation action.

3. Expected behavior

When I leave off the version of a secret, it means I want to use the latest version, and that means when I change the secret value, I expect a change set to be created and for the change set to use the updated value.

4. Suggest specific test cases

Say you have a reference in a template Foo like so: {{resolve:secretsmanager:MySecret:SecretString:password}}

1. Go to SecretsManager
2. Retrieve and edit the password for MySecret
3. Create a change set for Foo (management console or CLI) --> Change set should be created and executed with the new password

Currently, change set creation fails.

5. Helpful Links to speed up research and evaluation

• https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html#dynamic-references-secretsmanager
• https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-updating-stacks-changesets-create.html
• https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-CloudFormation.html
• https://docs.aws.amazon.com/secretsmanager/latest/userguide/integrating_cloudformation.html

6. Category (required) - Will help with tagging and be easier to find by other users to +1

Security, Developer Tools, Enhancement

[mliner]

This issue is quite a blocker for me (I am using complex cloudformation setup) - same as referencing latest version of ssm parameter. This two improvements would really help hundreds of people working with complex cf setups.

@mschweitzer-sd I was thinking of a workaround - create a macro or custom resource which would update a cf output with time info in it - every create changeset will trigger the update as the time info won't be the same as previously set.

[mschweitzer-sd]

@mliner lol yeah I could see that working, neat idea!

[dentonmwood]

Please please please add this. CloudFormation doesn't have a great way to roll out changes which affect resources in other templates, and we would really appreciate having that functionality. We already had to give up native imports since errors are thrown when the changes cause new resources to be created, changing the values of the exported services.

[dudeitssm]

+1 for this. For now, I am ok with adding an extra input param (date), but a native solution would be much preferred.

[mcIovin]

Yes, please please please add this. We have some SAM stacks that use 'resolve' from secretsmanager in template.yaml, and without this functionality we cannot rotate the secret.

[fabiatz]

+1. If this is working with SSM parameters, it would be consistent to implement with SecretsManager entries, too.

[yongzhang]

Also blocked by this.

[InQuirer]

+1. our team ran into this problem too (e.g. if you want the API Gateway authorizer token to be copied up from the latest version of a secret)

  MyAuthorizer:
    Properties:
      IdentityValidationExpression: '{{resolve:secretsmanager:my-authorizer:SecretString:token}}'

[yongzhang]

Has anyone tried this? https://www.amazonaws.cn/en/new/2024/amazon-cloudformation-improves-changesets-to-enable-safer-deployment-practices/ https://aws.amazon.com/about-aws/whats-new/2024/04/aws-cloudformation-changesets-enhanced-change-visibility-deployments/

I did a test today but no luck, but I do see cfn created changeset successfully by updating secret values, unfortunately cfn didn't update my resource (ECS task definition) with updated secret value even though I can see "Replacement: true" in changeset.

[InQuirer]

for now, I did the following workaround @mliner proposed:

AWSTemplateFormatVersion: 2010-09-09
Resources:

    SecretExpanderMacro:
        Type: AWS::CloudFormation::Macro
        Properties:
            Name: SecretExpanderMacro # use this macro name is Transform section
            Description: Replaces resolve:secretsmanager with the exact latest version
            FunctionName: !Ref SecretExpanderLambda

    SecretExpanderLambda:
        Type: AWS::Serverless::Function
        Properties:
            CodeUri: ./expand_secrets_macro.py # see lambda code gist link below
            MemorySize: 128
            Timeout: 60 # seconds
            Handler: expand_secrets_macro.lambda_handler
            Runtime: python3.11
            Architectures:
                - x86_64
            EventInvokeConfig:
                MaximumEventAgeInSeconds: 60
                MaximumRetryAttempts: 2
            Policies:
                - Statement:
                      - Effect: Allow
                        Action:
                            - logs:CreateLogGroup
                            - logs:CreateLogStream
                            - logs:PutLogEvents
                        Resource:
                            - !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:*
                      - Effect: Allow
                        Action:
                            - secretsmanager:GetSecretValue
                        Resource:
                            - !Sub arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:*

Transform: AWS::Serverless-2016-10-31

usage:

AWSTemplateFormatVersion: 2010-09-09
# ...

Transform:
  - SecretExpanderMacro

lambda code: https://gist.github.com/InQuirer/a7c5be8004ab4b78744a992ef58facc1