aws-sam-cli

aws-sam-cli copied to clipboard

Description

When using AWS SAM for local development, I can introduce environment variables by setting them in the template with no value, and then defining them in my environment. However, when I go to deploy, the environment variables do not appear to be inserted by sam package or sam deploy, and I get the following error on deploy:

Failed to create the changeset: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state Status: FAILED. Reason: [/Resources/GeneratePlan/Type/Environment/Variables/SECRET_ACCESS_KEY] 'null' values are not allowed in templates

Where SECRET_ACCESS_KEY is one of my environment variables. I cannot find any documentation detailing how to deploy projects with environment variables, either by having them defined in my environment or providing them in an alternate config.

I don't want to add the environment variables to my template.yml directly because this is stored in Git, and I don't want to edit them manually into the packaged.yml file each time between the package and deploy steps as that's cumbersome.

I haven't seen any steps in the documentation or similar issues, so I presume this is either an edge case bug or I am just missing something simple (in which case I might file this as a documentation bug) 😄

Steps to reproduce

The following config is a minimal excerpt from mine:

AWSTemplateFormatVersion: '2010-09-09'

Transform: AWS::Serverless-2016-10-31

Globals:

  Function:
    Environment:
      Variables:
        SECRET_ACCESS_KEY:

Resources:

  AnswerChallenge:
    Type: "AWS::Serverless::Function"
    Properties:
      Runtime: nodejs8.10
      Handler: build/functions/answerChallenge.default
      CodeUri: ./
      Policies: AmazonDynamoDBFullAccess
      Events:
        GetRequest:
          Type: Api
          Properties:
            Path: /event
            Method: get

Observed result

When I run sam start-api with SECRET_ACCESS_KEY defined, the lambda works as expected. When I attempt to deploy with sam package and sam deploy, I receive an error about undefined variables.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: MacOS
2. sam --version: 0.11

Same issue here. sam package/sam deploy should accept the --env-vars flag.

My current workaround is to use CF parameters, since sam package/sam deploy are aliases for cloudformation package/cloudformation deploy.

template.yml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
  SomeFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: some-function/
      Handler: index.handler
      Runtime: nodejs8.10
      Environment:
        Variables:
          SOME_VAR: !Ref SomeVar
Parameters:
  SomeVar:
    Type: String

$ sam package --template-file template.yml --s3-bucket BUCKET_NAME --output-template-file packaged.yml

$ sam deploy --template-file packaged.yml --stack-name STACK_NAME --capabilities CAPABILITY_IAM --parameter-overrides SomeVar=123

@jsonmaur Thanks for sharing a workaround! Do you know if that is compatible with local development (i.e. start-api and start-lambda)?

Works fine for me when I run sam local invoke --env-vars env.json. --env-vars seems to override the parameters.

Is there any response to this? I'm seeing the same thing.

sam package creates a YAML file containing my environment variables as expected, but after running sam deploy I can see that the Lambda configuration in the AWS console hasn't been updated to include them.

Trying to execute the Lambda then fails with the predictable "environment variable not defined" result.

Hi!

Any updates on this issue?

Thanks!

Wow does SAM make setting environment variables complicated!

@jsonmaur's workaround seems the best, using the (undocumented in sam docs) --parameter-overrides argument to sam deploy using the Name1=Value1 Name2=Value2 syntax.

sam build documents a --parameter-overrides argument, but it uses the the JSON shorthand syntax: ParameterKey=KeyPairName, ParameterValue=MyKey ParameterKey=InstanceType, ParameterValue=t1.micro .

So sam and cloudformation have multiple same-named arguments with entirely incompatible value syntax? Not helpful!

I tried using the sam build --parameter-overrides option but the parameters don't see to go into the build, and the values you specify for build are lost when you deploy. So not even sure what that is for?

Yeah the sam local start-api also uses the mismatched format in --parameter-overrides, which is frustrating! Having to use two different config formats between local dev and prod deploy is not great... I had tried the --env-vars solution but had issues with the different cases expected between uppercase and snake case. This is what I had to do to get some resemblance of consistency:

I store my config file as a JSON file, such as config.json:

{
  "Region": "us-west-1",
  "Secret": "my-real-secret",
  [...]
}

And define them as parameters in the template.yaml:

Parameters:
  Secret:
    Type: String
  [...]

[... later ...]

Globals:
  Function:
    Environment:
      Variables:
        SECRET: !Ref Secret
        [...]

Then for development I have the following in package.json:

"scripts": {
  "api": "sam local start-api --region us-west-2 --parameter-overrides \"$(jq -r -j 'to_entries[] | \"ParameterKey=\\(.key),ParameterValue=\\(.value) \"' config.json)\"",
}

So I can run npm run api to get the api server going with the config file.

Then for deployment our CI runs:

> sam package --template-file ./template.yaml --s3-bucket bucketname --output-template-file ./packaged.yaml
> sam deploy --template-file ./packaged.yaml --stack-name stackname --capabilities CAPABILITY_IAM --parameter-overrides $(jq -r 'to_entries[] | "\(.key)=\(.value)"' config.json)

Surprising amount of work to get sane environment variables in SAM... I think this needs some developer experience TLC or maybe I'm missing something blatantly obvious 😅

this is an issue. My Environment.variables just disappear upon command sam deploy --template-file packaged.yaml --region us-west-2 --capabilities CAPABILITY_IAM --stack-name lambda-stack-test

  LambdaStackFunction:
Type: AWS::Serverless::Function
Properties:
  FunctionName: LambdaStack-Local
  Timeout: 15
  Description: Local stack use to test -> deployed version
  CodeUri: lambda_stack/
  Handler: main.handler
  Role: !GetAtt LambdaExecutionRole.Arn
  Runtime: python3.7
  Layers:
    - arn:aws:lambda:us-west-xxxxxxxxxxxxxxx
    - arn:aws:lambda:us-west-xxxxxxxxxxxxxxx
Environment:
  variables:
    S3_ENDPOINT: https://s3.us-west-2.amazonaws.com
    SQS_ENDPOINT: https://sqs.us-west-2.amazonaws.com
    REGION: us-west-2
    TIMEZONE: US/Pacific

@LEscobar-Driver capital ‘V’ in Variables

@whereisaaron does nothing at all. Still the same

For local testing, you can override env variables in a few ways, including the --env-vars parameter. However, in the deploy step, you're just passing along the packaged template. For that, you must define all environment variables, either directly or as a parameter.

If this doesn't cover your needs, it could help to explain more about what you're trying to do.

@awood45 Thanks for jumping in! I will see if I can provide some context. Essentially, I'm looking for a unified method for configuring the application that works both in sam local and sam deploy.

For example, when I create a non-serverless Node.js application based on express, in the documentation I can provide information on what environment variables are expected to be present. Developers can define these in their environment for local dev, and in production the server can set those environment variables. It's the same interface, it's well defined intuitively, and it's conventional.

I thought I had similar behaviour in SAM, as I found a Github issue mentioning how to pass through environment variables by setting them to an empty value in the template. That was great! It was frustrating to learn this does not work when deploying.

Some notes on my goals:

• I do not want to put the environment variables directly into the configuration file, as I can then no longer safely check that file into version control
• There are a lot of environment variables in any moderately complex app. I'd like to not have to supply all of them on the command line, or use a hack like jq to get them to parse from a file. This is particularly annoying as sam local and sam deploy expect them on the command line in different formats.
• I'd like to be able to use the same interface / format for describing my environment variables in both local dev and when deploying. Setting environment variables is the ideal method here for me personally but even a common format config file would be significantly better.

FWIW I think there is both dev UX aspects and documentation aspects here. I have ideals about how to deploy, which would be great if we could improve the dev UX there, but realistically documentation is what originally made me create the issue here. When I went to figure out how to add environment variables in dev I couldn't find any documentation on this at all. I did find an answer through Github issues. When I found that didn't work in production, I again couldn't find any documentation at all on how to actually deploy environment variables, and had to get help on this issue here from @jsonmaur to get deploy working. It may seem intuitive as it uses another AWS command under the hood, but do consider that users of SAM for the first time might not be familiar with how to deploy lambda's under cloudformation. This has been my experience over many months so it's possible that docs have improved since then but I definitely found it unclear how to actually deploy an app with env variables.

Does that clarify a bit? Happy to help elaborate some more if needed.

I too expected SAM would have a consistent way to handle env vars between running locally and deploying. Like @jsonmaur I expected --env-vars to work for deployment too.

Right now sam local, sam build, and sam deploy have three, mutually incompatible ways to specify environment variable values.

We'll definitely want to improve documentation on the deploy difference, though I would agree we should at minimum converge options on build/local to match.

One issue with deployments is that they are simply a CloudFormation deployment. What you'd be asking for in that case is for --env-vars to transform the template before deployment, without those changes being saved anywhere.

Unfortunately the ship has sailed to an extent for naming this better without it being a breaking change, but it's really a way to override your production env variables for local testing.

To add more context to the "We have 3 different ways to do environment variables".

The --env-vars options pre-dates use resolving Parameters. During that time, we needed a way to allow customers to change their Env Vars defined in the template. This still has applications today (overriding dynamic references that are inlined in the Env Var definition for example) but most of the time --parameter-overrides will be what you will want. Especially, if you are following the pattern of using Parameters to allow changes to the template at deploy (or in this case local invoke) time.

Now to the "build and local have different way than deploy for parameter-overrides.

Sadly, we didn't have a better way. Currently deploy still shells out to the AWS CLI (which we will be addressing separately). When we introduced the --parameter-overrides into the local suite (and later build), we attempted to match what AWS CLI was doing to keep things consistent. AWS CLI, however, does not use click (they use argparse if I recall correctly). Because the the different ways we have the cli command defined, we couldn't match AWS CLI's way and were forced to deviate. When we revisit deploy, this will most likely change to follow the pattern we have in local/build.

Hopefully that sheds some light on why, it is what it is right now. We know the inconsistency is a UX problem and rough edge right now.

Thank you both for the context!

Thanks for the background. Everyone here seems to agree (!) that --parameter-overrides on sam deploy to set environment variables in the template via parameters is the current best way to go. However that option is not listed in the command or online documentation:

https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-deploy.html

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
  SomeFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: some-function/
      Handler: index.handler
      Runtime: nodejs8.10
      Environment:
        Variables:
          SOME_VAR: !Ref SomeVar
Parameters:
  SomeVar:
    Type: String

I'm trying to follow the example you shared above, but instead of the value of the parameter, the name of the parameter itself getting into the env variable i.e., SOME_VAR = 'SomeVar', also I want to set the default value of parameters; based on my understanding it should be done as below but none of the way it is working

**Parameters:

SomeVar: value**

I agree that documentation could be better - the most important is to add missing param --parameter-overrides to aws deploy command.

sam deploy is an alias to aws cloudformation deploy, I think - when you run aws cloudformation deploy help you can find information about missing param.

Working example

template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
  
Globals:

  Function:
    Timeout: 3

Parameters:

  SomeVar:
    Type: String
    Description: My SomeVar
    Default: default value

Resources:

  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: hello-world/
      Handler: app.lambdaHandler
      Runtime: nodejs12.x
      Environment:
        Variables:
          SOME_VAR: !Ref SomeVar
      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /hello
            Method: get

Outputs:

  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"

  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn

  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

When you run:

sam build
sam local invoke HelloWorldFunction -e events/event.json

the default value will be used - in our case: SOME_VAR=default value

To deploy run commands:

sam package --template-file template.yaml \
  --s3-bucket my-bucket \
  --output-template-file packaged.yaml

sam deploy --template-file packaged.yaml \
  --stack-name my-application \
  --capabilities CAPABILITY_IAM \
  --parameter-overrides SomeVar=other_value

The last line do "the magic" and allow you to override default value:

--parameter-overrides SomeVar=other_value

When the real function will be invoked: SOME_VAR=other_value.

I think is important to remember that SAM templates are compatible with CloudFormation templates and all techniques/concepts could be used in SAM development.

Thanks @wojtekk! It's really helpful!

I get "unable to use container host config override file from '$HOME/.config/aws-sam-local/container-config.json': HOME env variable is not set" after get request "127.0.0.1:3000/ping" when use "sam local start-api --template sam.yaml".

This is my log: "2020/02/28 10:22:06 Invoking com.amazonaws.serverless.proxy.struts2.Struts2LambdaHandler::handleRequest (java8) 2020/02/28 10:22:07 WARNING: No AWS credentials found. Missing credentials may lead to slow startup times as detailed in #134 2020/02/28 10:22:07 Mounting /c/Users/huyho/OneDrive/Documents/Java Project/aws-serverless-struts2-archetype as /var/task:ro inside runtime container 2020/02/28 10:22:07 unable to use container host config override file from '$HOME/.config/aws-sam-local/container-config.json': HOME env variable is not set 2020/02/28 10:22:07 Error invoking java8 runtime: Error: No such image: lambci/lambda:java8 "

I have followed the guide in README.md. Please help me.

Local env variables work fine with sam local invoke -n env.json where env.json looks like this

{
  "Parameters": {
    "KEY1: "value1"
    "KEY2": "value2"
    ...
  }
}

I can't figure out how to list the K/V pair the parameter-overrides. I've tried many permutations. Here's one

sam deploy \
  --template-file package.yml \
  --stack-name my-stack \
  --s3-bucket my_bucket \
  --region us-east-1 \
  --capabilities CAPABILITY_IAM \
  --parameter-overrides KEY1=valu1e, KEY2=value2, KEY3=value3

I've setup env variables in the web UI lambda definition area but that doesn't seem to help. The --help is not very helpful for this parameter. I don't know what ParameterKey=KeyPairName, ParamterValue=MyKey means.
What I see in. help for deploy:

 --parameter-overrides           Optional. A string that contains AWS
                                  CloudFormation parameter overrides encoded
                                  as key=value pairs.For example, 'ParameterKe
                                  y=KeyPairName,ParameterValue=MyKey Parameter
                                  Key=InstanceType,ParameterValue=t1.micro' or
                                  KeyPairName=MyKey InstanceType=t1.micro

Can I get some help with how to list multiples K/V pairs in the parameter-overrides

Hello, @rebeccapeltz

For multiple parameter overrides, you could write it like this (without comma, just space):

--parameter-overrides KEY1=valu1e KEY2=value2 KEY3=value3

I'm working now on just adding one parameter and getting the same error. Here's the error:

Error: Invalid value for '--parameter-overrides':  KEY=value is not in valid format. It must look something like 'ParameterKey=KeyPairName,ParameterValue=MyKey ParameterKey=InstanceType,ParameterValue=t1.micro' or 'KeyPairName=MyKey InstanceType=t1.micro'
I don't know what `KeyPairName` refers to, or why it says `ParameterValue=MyKey ParameterKey=InstanceType`.  

My current deloy command looks like this

sam deploy \
  --template-file package.yml \
  --stack-name my-stack \
  --s3-bucket my_bucket \
  --region us-east-1 \
  --capabilities CAPABILITY_IAM \
  --parameter-overrides KEY=value

I've tried another way of submitting parameter-overrides base on the error message. This doesn't fail due to missing parameter-overrides.

sam deploy \
  --template-file package.yml \
  --stack-name my-stack \
  --s3-bucket my_bucket \
  --region us-east-1 \
  --capabilities CAPABILITY_IAM \
  --parameter-overrides ParameterKey=Key1, ParameterValue=value1

  Error: Failed to create changeset for the stack: my-stack, ex: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state Status: FAILED. Reason: Template format error: Unresolved resource dependencies [Key1] in the Resources block of the template

The failure points to a problem in the package which looks like this

Globals:
  Function:
    Timeout: 3
    Environment:
      Variables:
        Key1:
          Ref: Key1

The template from which the package is made looks like this

Globals:
  Function:
    Timeout: 3
    Environment:
      Variables:
        Key1: !Ref Key1

Again, this works fine locally, so I think the problem is in specifying the environment variable.

@rebeccapeltz --parameter-overrides Overrides the Parameter section of the template not the Environment.Variables on a Function. To be able to override on deploy, you need to reference a Parameter defined on your template. Do you have that sections defined? A good example can be found in this comment above.

Thanks @jfuss! I got it deployed. I found that I then had to add the environment variable using the Web UI. Here's what worked.

sam package --template-file template.yml --output-template-file package.yml --s3-bucket my-bucket
sam local invoke -n config.json
sam deploy --template-file package.yml --stack-name my-stack --s3-bucket my-bucket --region us-east-1 --capabilities CAPABILITY_IAM  --parameter-overrides 'ParameterKey=Key1, ParameterValue=Value1'

in template.yml

Parameters:
  Key1:
    Type: String
    Description: Key
    Default: "default value 1"
  
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 3
    Environment:
      Variables:
        KEY1: !Key1

contents of local invoke config.json

{
  "HelloWorldFunction": {
    "KEY1": "Value1",
  }
}

in the app.js, a value of process.env.KEY1 is available

Has anyone found a good way to address this with quite a few different env variables (say 10-20 or so)?

The two things I want to avoid are:

• Committing any files containing my env variables to a repo.
• Manually typing env variables on every deploy. That seems like a living nightmare.

The only ways I could think of would be adding them to the template.yaml file or adding the script with the --parameter-overrides to package.json file. Both would mean I have to commit my env variables to a repo.

I think one fo the option is, you can set the env variables and then use the variable while executing the command. Or create a shell script wrapper which can supply these value from env. Ideally once you are done with development, these variable will be injected through ci/cd pipelines

Regards Adnan

On Tue, Aug 4, 2020 at 11:27 AM Nick Bordeau [email protected] wrote:

Has anyone found a good way to address this with quite a few different env variables (say 10-20 or so)?

The two things I want to avoid are:

• I definitely do not want to commit any files containing my env variables to a repo.
• I do not want to manually type them on every deploy. That seems like a living nightmare.

The only ways I could think of would be adding them to the template.yaml file or adding the script with the --parameter-overrides to package.json file. Both would mean I have to commit my env variables to a repo.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/awslabs/aws-sam-cli/issues/1163#issuecomment-668696414, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACIH7AHZUTZWG5WEFCCIEYTR7AZITANCNFSM4HK5LDNA .

-- Regards, Adnan Yaqoob

After reading through this thread, I think one main problem is the docs for --parameter-overrides are extremely confusing.

'ParameterKey=KeyPairName, ParameterValue=MyKey ParameterKey=InstanceType,ParameterValue=t1.micro'.

Whereas it should be simply:

ParameterKey1=Value1, ParameterKey2=Value2