add ability to access custom CDK resource outputs from Lambda function

[shishkin]

Note: If your question is regarding the AWS Amplify Console service, please log it in the AWS Amplify Console repository

Which Category is your question related to? custom and function

Amplify CLI Version

7.5.2 (Why does a CLI tool need 9.35 seconds to print its version? 🤔 )

What AWS Services are you utilizing?

Amplify

Provide additional details e.g. code snippets

I'm trying to add a dependency via backend-config.json for my custom CDK resource. Somehow I managed to add a dependency to a function. That worked well and I deployed (amplify push) it. Now I've added another dependency to another custom CDK resource and whenever I try to amplify env checkout dev or amplify build or amplify push CLI keeps reverting my change in the backend-config.json and removing the second dependency. What am I doing wrong?

So far my experience with Amplify has been extremely frustrated. So much magic happening under the hood that I keep more time spent debugging and troubleshooting then I would have just building the stack with CDK.

[lazpavel]

Hi @shishkin, using the backend-config is not the right approach, you can use the steps here to add custom resources: https://docs.amplify.aws/cli/custom/cdk/#reference-amplify-environment-name.

Please let us know if you're still having issues adding custom resources

[shishkin]

Thanks for the link @lazpavel . I've studied it several times already. Could you please outline the right approach of how to make Amplify function depend on a custom CDK resource and consume one of its outputs as an environment variable?

[lazpavel]

Hi @shishkin, can you provide more information about the specific problem you're trying to solve in order for us to see if there would be a different approach to solve it.

[shishkin]

Specific scenario is that I'm creating an SNS topic as CDK construct and provide topic ARN as an output. I need an Amplify lambda function to get the topic ARN as an env var so that it can publish to the topic.

[nathanagez]

Specific scenario is that I'm creating an SNS topic as CDK construct and provide topic ARN as an output. I need an Amplify lambda function to get the topic ARN as an env var so that it can publish to the topic.

Hi, have you been able to do it ?

[nathanagez]

Hi @shishkin, can you provide more information about the specific problem you're trying to solve in order for us to see if there would be a different approach to solve it.

Hello @lazpavel, any hints on how we can achieve this ? Or maybe some directives on how we can contribute to implement this feature ?

[josefaidt]

Hey @shishkin :wave: apologies for the delay! Are you still having trouble with this? We can run amplify update function and grant access to the custom resource

[nathanagez]

Hey @shishkin 👋 apologies for the delay! Are you still having trouble with this? We can run amplify update function and grant access to the custom resource

Hello @josefaidt, it looks like that it's not supported for custom resources.

What I'm trying to achieve is access custom stack outputs from the lambda.

Here is my CDK stack code:

    new cdk.CfnOutput(this, "inputBucket", {
      value: "TEST_FROM_CDK"
    });

Here is my backend-config.json:

"outputBucketWatcher": {
      "build": true,
      "providerPlugin": "awscloudformation",
      "service": "Lambda",
      "dependsOn": [
        {
          "category": "custom",
          "resourceName": "video",
          "attributes": [
            "inputBucket"
          ]
        }
      ]
    }

Here is the Lambda CloudFormation stack:

  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "{\"createdOn\":\"Mac\",\"createdBy\":\"Amplify\",\"createdWith\":\"7.6.3\",\"stackType\":\"function-Lambda\",\"metadata\":{}}",
  "Parameters": {
    "CloudWatchRule": {
      "Type": "String",
      "Default": "NONE",
      "Description": " Schedule Expression"
    },
    "deploymentBucketName": {
      "Type": "String"
    },
    "env": {
      "Type": "String"
    },
    "s3Key": {
      "Type": "String"
    },
    "customvideoinputBucket": {
      "Type": "String"
    }
  }

And I'm getting the following error:

I also tested to run amplify env checkout <my-current-env> it doesn't change anything

[shishkin]

I actually reduced Amplify footprint by deploying backend infra with AWS CDK. CDK is a much more straightforward and stable abstraction. So instead of a small custom extension point in Amplify framework I have full flexibility of CDK constructs, including SSR-enabled Next.js websites hosting that I originally used Amplify for. Amplify had a very nice value proposition for people starting with AWS to quickly get an app up and running. Unfortunately I ended up wasting more time troubleshooting Amplify's limitations and issues. This time would have been better spent learning the actual AWS services that Amplify abstracts over.

[josefaidt]

Hey @shishkin and @nathanagez :wave: unfortunately it is not yet possible to reference a custom CDK resource output in a Lambda function. I'll mark this as a feature request 🙂

[nathanagez]

Hey @josefaidt, any hints on how we can add this feature ? If we can accelerate the process by opening a PR :)

[filod]

@josefaidt any workaround for this? it's really annoying and full of false toturial on the internet. e.g. in the official article it says "For demo purposes hard-coded, normally recommended to use environment variable", and i nerve find a way to inject my sqs arn into lambda. how can this even be a problem....it's 2022 now...

[dwil618]

Here is the method I have managed to use for my project, in case it's any use to others. It allowed me to access a custom stack's state machine ARN in my lambda function. The parameter naming I believe is case sensitive, and must match your resource/output names.

Note: I did have an issue while deploying this at first, when adding to an existing project. It didn't like the Fn::GetAtt. But when I deleted the app and redeployed it was happy, so maybe I had something wrong with my existing project.

1. Output value from stack

   amplify/backend/custom/<custom-stack-name>/cdk-stack.ts

   new cdk.CfnOutput(this, "<output-name>", {
       value: <output-value>
   })
   

2. Tell amplify to import the value as a parameter to the function

   amplify/backend/function/<function-name>/parameters.json

   {
       "custom<custom-stack-name><output-name>": {
           "Fn::GetAtt": [
               "<custom-stack-name>",
               "<output-name>"
           ]
       }
   }
   

3. Modify the function's cloudformation template to use the parameter

   amplify/backend/function/<function-name>/<function-name>-cloudformation-template.json

   {
       ...
       "Parameters": {
           ...
           "custom<custom-stack-name><output-name>": {
               "Type": "String"
           }
       },
       ...
       "Resources": {
           "LambdaFunction": {
               ...
               "Properties": {
                   ...
                   "Environment": {
                       "Variables": {
                           ...
                           "<environment-variable-name>": {
                               "Ref": "custom<custom-stack-name><output-name>"
                           }
                       }
                   }
               }
           }
           ...
       }
       ...
   }
   

4. Update backend config dependencies, to make sure resources are built in correct order

   amplify/backend/backend-config.json

   {
       ...
       "function": {
           "<function-name>": {
               ...
               "dependsOn": [
                   ...
                   {
                       "category": "custom",
                       "resourceName": "<custom-stack-name>",
                       "attributes": [
                           "<output-name>"
                       ]
                   }
               ]
           }
           ...
       }
       ...
   }
   

5. Access the value in the function

   amplify/backend/function/<function-name>/src/index.js

   const cdkOutputVal = process.env.<environment-variable-name>
   

[ivadenis]

Here is the method I have managed to use for my project, in case it's any use to others. It allowed me to access a custom stack's state machine ARN in my lambda function. The parameter naming I believe is case sensitive, and must match your resource/output names.

Note: I did have an issue while deploying this at first, when adding to an existing project. It didn't like the Fn::GetAtt. But when I deleted the app and redeployed it was happy, so maybe I had something wrong with my existing project.

1. Output value from stack amplify/backend/custom/<custom-stack-name>/cdk-stack.ts

   new cdk.CfnOutput(this, "<output-name>", {
       value: <output-value>
   })
   

2. Tell amplify to import the value as a parameter to the function amplify/backend/function/<function-name>/parameters.json

   {
       "custom<custom-stack-name><output-name>": {
           "Fn::GetAtt": [
               "<custom-stack-name>",
               "<output-name>"
           ]
       }
   }
   

3. Modify the function's cloudformation template to use the parameter amplify/backend/function/<function-name>/<function-name>-cloudformation-template.json

   {
       ...
       "Parameters": {
           ...
           "custom<custom-stack-name><output-name>": {
               "Type": "String"
           }
       },
       ...
       "Resources": {
           "LambdaFunction": {
               ...
               "Properties": {
                   ...
                   "Environment": {
                       "Variables": {
                           ...
                           "<environment-variable-name>": {
                               "Ref": "custom<custom-stack-name><output-name>"
                           }
                       }
                   }
               }
           }
           ...
       }
       ...
   }
   

4. Update backend config dependencies, to make sure resources are built in correct order amplify/backend/backend-config.json

   {
       ...
       "function": {
           "<function-name>": {
               ...
               "dependsOn": [
                   ...
                   {
                       "category": "custom",
                       "resourceName": "<custom-stack-name>",
                       "attributes": [
                           "<output-name>"
                       ]
                   }
               ]
           }
           ...
       }
       ...
   }
   

5. Access the value in the function amplify/backend/function/<function-name>/src/index.js

   const cdkOutputVal = process.env.<environment-variable-name>
   

No need to use parameters.json. The parameter will be already available in the template as long as you specify it in the backend-config.json as a dependency.

the naming convention for params is: <category><resource-name><parameter>

[armenr]

@ivadenis - I think you are mistaken.

I tried it the way you mentioned, and it didn't work.

Here's my backend-config.json

I am depending on cloudformation outputs from my custom CDK resource that I added via amplify CLI

    "projectV2BucketGuardSigner": {
      "build": true,
      "dependsOn": [
        {
          "attributes": [
            "privateKeySecretName",
            "cloudfrontURL",
            "privateKeySecretArn",
            "publicKeyId"
          ],
          "category": "custom",
          "resourceName": "projectV2CDN"
        }
      ],
      "providerPlugin": "awscloudformation",
      "service": "Lambda"
    },

After an amplify push --> those don't show us parameters inside my projectV2BucketGuardSigner cloudformation template as parameters.

[iShavgula-TacTill]

@armenr you need to run amplify env checkout your-current-env-name so that manual modifications ofbackend-config.json are taken into account.