BUG: Can not use secrets manager when using step function, ssm:/ does not work

[300LiterPropofol]

This is a (Bug Report)

Description

For bug reports:

• What went wrong?

I added

custom:
  nestingVariable: ${ssm:/aws/reference/secretsmanager/${env:secret_name}}

in my serverless.yml, but the deployment kept failing with an error A valid SSM parameter to satisfy the declaration 'ssm:/aws/reference/secretsmanager/my-variable' could not be found. I tested this exact syntax for retrieving exactly the same secret from AWS secrets manager and it worked in other places. Only caused error in this step function included serverless.yml. And if I remove everything about step function in my serverless.yml, it can be deployed normally.

• What did you expect should have happened?

Normal deployment, deploy successfully.

• What was the config you used?

My serverless.yml file is as below:

service: my-service
frameworkVersion: '2'
useDotenv: true
unresolvedVariablesNotificationMode: error
variablesResolutionMode: 20210326
configValidationMode: error

provider:
  name: aws
  runtime: python3.8
  stage: ${opt.stage, 'temp'}
  region: eu-west-1
  logRetentionInDays: 14
  lambdaHashingVersion: 20201221
  vpc:
      securityGroupIds:
        - ${env:security_group_ID}
      subnetIds:
        - ${env:subnet_IDs_1}
        - ${env:subnet_IDs_2}
        - ${env:subnet_IDs_3}
  endpointType: regional
  apiGateway:
    shouldStartNameWithService: true
  versionFunctions: true

custom:
  nestingVariable: ${ssm:/aws/reference/secretsmanager/${env:secret_name}}
  mongourl: ${self:custom.nestingVariable}
  pythonRequirements:
    dockerizePip: false
  kmsGrants:
      kmsKeyId: ${env:kms_key_ID}

package:
  patterns:
    - '!README.md'

functions:
  some_handler:
    handler: handler.some_handler
    description: Lambda function that
    environment:
      mongourl: ${self:custom.mongourl}
      mongodbname: ${env:mongo_dbname}
    memorySize: 1024
    timeout: 120
  other_handler:
    handler: handler.other_handler
    description: Lambda function that
    environment:
      sqsurl: ${env:sqs_url}
    memorySize: 1024
    timeout: 120

stepFunctions:
  stateMachines:
    parallelMachine:
      events:
        - http:
            path: my-path
            method: post 
            cors: true
      definition:
        Comment: "A parallel state to execute two branchesS."
        StartAt: Parallel
        States:
          Parallel:
            Type: Parallel
            Branches:
            - StartAt: Some_Insertion
              States:
                Some_Insertion:
                  Type: Task
                  Resource:
                    Fn::GetAtt: [some_handler, Arn]
                  End: true
            - StartAt: Some_Upload
              States:
                Some_Upload:
                  Type: Task
                  Resource:
                    Fn::GetAtt: [other_handler, Arn]
                  End: true
            End: true   

plugins:
  - serverless-python-requirements
  - serverless-kms-grants
  - serverless-step-functions

• What stacktrace or error message from your provider did you see?

ServerlessError: A valid SSM parameter to satisfy the declaration 'ssm:/aws/reference/secretsmanager/my-variable' could not be found.
      at Variables.handleUnresolved (/snapshot/serverless/lib/classes/Variables.js:986:13)
      at Variables.renderMatches (/snapshot/serverless/lib/classes/Variables.js:375:12)
      at /snapshot/serverless/lib/classes/Variables.js:396:31
      at tryCatcher (/snapshot/serverless/node_modules/bluebird/js/release/util.js:16:23)
      at Promise._settlePromiseFromHandler (/snapshot/serverless/node_modules/bluebird/js/release/promise.js:547:31)
      at Promise._settlePromise (/snapshot/serverless/node_modules/bluebird/js/release/promise.js:604:18)
      at Promise._settlePromise0 (/snapshot/serverless/node_modules/bluebird/js/release/promise.js:649:10)
      at Promise._settlePromises (/snapshot/serverless/node_modules/bluebird/js/release/promise.js:729:18)
      at Promise._fulfill (/snapshot/serverless/node_modules/bluebird/js/release/promise.js:673:18)
      at PromiseArray._resolve (/snapshot/serverless/node_modules/bluebird/js/release/promise_array.js:127:19)
      at PromiseArray._promiseFulfilled (/snapshot/serverless/node_modules/bluebird/js/release/promise_array.js:145:14)
      at Promise._settlePromise (/snapshot/serverless/node_modules/bluebird/js/release/promise.js:609:26)
      at Promise._settlePromise0 (/snapshot/serverless/node_modules/bluebird/js/release/promise.js:649:10)
      at Promise._settlePromises (/snapshot/serverless/node_modules/bluebird/js/release/promise.js:729:18)
      at _drainQueueStep (/snapshot/serverless/node_modules/bluebird/js/release/async.js:93:12)
      at _drainQueue (/snapshot/serverless/node_modules/bluebird/js/release/async.js:86:9)
      at Async._drainQueues (/snapshot/serverless/node_modules/bluebird/js/release/async.js:102:5)
      at Immediate._onImmediate (/snapshot/serverless/node_modules/bluebird/js/release/async.js:15:14)
      at processImmediate (internal/timers.js:456:21)
      at process.topLevelDomainCallback (domain.js:137:15)

• What is the use case that should be solved. The more detail you describe this in the easier it is to understand for us.

I changed the serverless file to as below, i.e., remove all content related to step-function, and the folder can be deployed successfully without ssm:/ error. Even with just adding - serverless-step-functions inside plugin field (without adding any actual stepFunctions field) will cause the same ServerlessError: A valid SSM parameter to satisfy the declaration 'ssm:/aws/reference/secretsmanager/my-variable' could not be found. error. It seems to me that there is something in this plugin that has conflict with the core of serverless.

service: my-service
frameworkVersion: '2'
useDotenv: true
unresolvedVariablesNotificationMode: error
variablesResolutionMode: 20210326
configValidationMode: error

provider:
  name: aws
  runtime: python3.8
  stage: ${opt.stage, 'temp'}
  region: eu-west-1
  logRetentionInDays: 14
  lambdaHashingVersion: 20201221
  vpc:
      securityGroupIds:
        - ${env:security_group_ID}
      subnetIds:
        - ${env:subnet_IDs_1}
        - ${env:subnet_IDs_2}
        - ${env:subnet_IDs_3}
  endpointType: regional
  apiGateway:
    shouldStartNameWithService: true
  versionFunctions: true

custom:
  nestingVariable: ${ssm:/aws/reference/secretsmanager/${env:secret_name}}
  mongourl: ${self:custom.nestingVariable}
  pythonRequirements:
    dockerizePip: false
  kmsGrants:
      kmsKeyId: ${env:kms_key_ID}

package:
  patterns:
    - '!README.md'

functions:
  some_handler:
    handler: handler.some_handler
    description: Lambda function that
    environment:
      mongourl: ${self:custom.mongourl}
      mongodbname: ${env:mongo_dbname}
    memorySize: 1024
    timeout: 120
  other_handler:
    handler: handler.other_handler
    description: Lambda function that
    environment:
      sqsurl: ${env:sqs_url}
    memorySize: 1024
    timeout: 120 

plugins:
  - serverless-python-requirements
  - serverless-kms-grants

• If there is additional config how would it look

See above

Similar or dependent issues:

• #439 and #320 may be slightly related, it seems to me the step-function plugin impacts the serverless core behavior.

Additional Data

• Serverless Framework Core Version you're using: 2.41.2 (standalone)
• The Plugin Version you're using: 5.0.0
• Operating System: linux
• Stack Trace:

Serverless: Load command interactiveCli
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command config:tabcompletion
Serverless: Load command config:tabcompletion:install
Serverless: Load command config:tabcompletion:uninstall
Serverless: Load command create
Serverless: Load command install
Serverless: Load command package
Serverless: Load command deploy
Serverless: Load command deploy:function
Serverless: Load command deploy:list
Serverless: Load command deploy:list:functions
Serverless: Load command invoke
Serverless: Load command invoke:local
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command metrics
Serverless: Load command print
Serverless: Load command remove
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command slstats
Serverless: Load command plugin
Serverless: Load command plugin
Serverless: Load command plugin:install
Serverless: Load command plugin
Serverless: Load command plugin:uninstall
Serverless: Load command plugin
Serverless: Load command plugin:list
Serverless: Load command plugin
Serverless: Load command plugin:search
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command upgrade
Serverless: Load command uninstall
Serverless: Load command requirements
Serverless: Load command requirements:clean
Serverless: Load command requirements:install
Serverless: Load command requirements:cleanCache
Serverless: Load command createKmsGrant
Serverless: Load command revokeKmsGrant
Serverless: Load command invoke
Serverless: Load command invoke:stepf
Serverless: Load command login
Serverless: Load command logout
Serverless: Load command generate-event
Serverless: Load command test
Serverless: Load command dashboard
Serverless: Load command output
Serverless: Load command output:get
Serverless: Load command output:list
Serverless: Load command param
Serverless: Load command param:get
Serverless: Load command param:list
Serverless: Load command studio
Serverless: Deprecation warning: CLI options definitions were upgraded with "type" property (which could be one of "string", "boolean", "multiple"). Below listed plugins do not predefine type for introduced options:
             - ServerlessStepFunctions for "name", "data", "path", "stage", "region"
            Please report this issue in plugin issue tracker.
            Starting with next major release, this will be communicated with a thrown error.
            More Info: https://www.serverless.com/framework/docs/deprecations/#CLI_OPTIONS_SCHEMA
Serverless: Skipping variables resolution with old resolver (new resolver reported no more variables to resolve)
Serverless: Invoke deploy
Serverless: Invoke package
Serverless: Invoke aws:common:validate
Serverless: Invoke aws:common:cleanupTempDir

• Provider Error messages: See above.

Please have a check at this bug. I really appreciate if it can be fixed soon. Thank you!

[medikoo]

This will most likely be fixed with https://github.com/serverless-operations/serverless-step-functions/pull/451

[300LiterPropofol]

Thanks! I can see #451 is still open and I guess this patch hasn't been released in the latest version of serverless. I will close this issue once a new version is released and fix my bug.