terraform-provider-aws icon indicating copy to clipboard operation
terraform-provider-aws copied to clipboard

Unable to run `terraform apply` successfully after creating provisioned concurrency config for a lambda alias

Open sarahbeharryovo opened this issue 4 years ago • 15 comments

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.14 aws provider version 2.58.0

Affected Resource(s)

aws_provisioned_concurrency_config aws_lambda_alias

Terraform Configuration Files

resource "aws_lambda_alias" "my-alias" {
  name = "alias-one"
  description = "The current version of the lambda"
  function_name = "${aws_lambda_function.my-function.arn}"
  function_version = "${aws_lambda_function.my-function.version}"
}

resource "aws_lambda_provisioned_concurrency_config" "my_concurrency_config" {
  function_name = "${aws_lambda_alias.my-alias.function_name}"
  provisioned_concurrent_executions = 1
  qualifier = "${aws_lambda_alias.my-alias.name}"
}

Expected Behavior

After running Terraform once to create the alias, and the provisioned concurrency config associated with the alias, running the same Terraform again runs without error

Actual Behavior

The second run fails with:

Error: Error applying plan:
1 error occurred:
    * module.my-terraform-module.aws_lambda_provisioned_concurrency_config.my_api_concurrency_config: 1 error occurred:
    * aws_lambda_provisioned_concurrency_config.my_api_concurrency_config: error getting Lambda Provisioned Concurrency Config (arn:aws:lambda:REGION:AWS_ACCOUNT_ID:function:my-function:alias-one): ValidationException: 
    status code: 400, request id: $REQUEST_ID

Steps to Reproduce

  1. Create a lambda, with an alias and provisioned concurrency associated with the alias
  2. Run terraform apply
  3. Run terraform apply again.

sarahbeharryovo avatar Apr 21 '20 15:04 sarahbeharryovo

I had this issue when creating the aws_lambda_provisioned_concurrency_config resource timed out which spinning up the provisioned execution environments. When this occured, the only way to recover was to remove the aws_lambda_provisioned_concurrency_config resource from the state, manualled deleted it from aws, and try again.

ccolefax avatar Apr 28 '20 04:04 ccolefax

To add another detail I forgot: the first run is "successful" (i.e. provisioned concurrency is set up), but you do get that ValidationException error so it looks like it hadn't worked.

sarahbeharryovo avatar Apr 28 '20 07:04 sarahbeharryovo

Was this ever addressed? I just had it occur for me

RalphBragg avatar Dec 21 '20 22:12 RalphBragg

An error happened while I was raising this issue, and I'm not sure it properly passed through the auto-labelling to be triaged, etc. Not sure what to do to get it looked at, or whether I should just re-raise the issue.

sarahbeharryovo avatar Dec 22 '20 07:12 sarahbeharryovo

I was able to work around this bug by not applying the aws_lambda_provisioned_concurrency_config to the alias name but to the version that the alias refers to.

MarkusKramer avatar Feb 03 '21 17:02 MarkusKramer

I have my concurrency config connected to an alias and I ran into this issue.

My concurrency config references the alias for both function_name and qualifier.

I removed the concurrency config from state and imported it. It wants to change the function_name field to an ARN no matter what I set function_name to in the alias.

Workaround for my case: reference the function itself for the function_name attribute in the concurrency config (this is probably what you're doing @MarkusKramer, but it's not clear where you're getting qualifier from - I'm still getting mine from the alias). I think the issue must be with aws_lambda_alias's function_name attribute always being an ARN.

jeffcasavant avatar Mar 17 '21 19:03 jeffcasavant

I am unable to run terraform plan, terraform apply or terraform destroy! Is there any other work around other than manually pulling the resource from state?

tjjosep avatar Aug 27 '21 00:08 tjjosep

Does any one have any work around to this? I tried using all sort of combinations (func name + alias, func name + version, func arn + alias, func arn + version) - any quick help would be greatly appreciated.

sriniavgs avatar Feb 17 '22 03:02 sriniavgs

We're seeing a similar issue, I'm even seeing the same ValidationException when attempting to import the resources I made manually within the console. Similar to the above, I've tried different combinations for the function_name and qualifier within the aws_lambda_provisioned_concurrency_config with no luck

armandobelardo avatar Mar 31 '22 19:03 armandobelardo

This confiugration looks to be working for me

resource "aws_lambda_provisioned_concurrency_config" "test_concurrency" {
  function_name                     = var.rta-lambda-function-name
  provisioned_concurrent_executions = 3
  qualifier                         = aws_lambda_alias.test_alias.function_version
}

var.rta-lambda-function-name is a local variable hard-coding the function name in plain text (just the name, no ARNs)

Hope it can help!

MrMavin avatar Jun 29 '22 09:06 MrMavin

This is still an issue for me. I followed the documentation at https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_provisioned_concurrency_config

On the first run, the TF Plan step succeeds but the Apply steps fails with the following:

module.myLambda.aws_lambda_provisioned_concurrency_config.myLambda: Still creating... [50s elapsed] module.myLambda.aws_lambda_provisioned_concurrency_config.myLambda: Still creating... [1m0s elapsed] module.myLambda.aws_lambda_provisioned_concurrency_config.myLambda: Still creating... [1m10s elapsed] module.myLambda.aws_lambda_provisioned_concurrency_config.myLambda: Still creating... [1m20s elapsed] module.myLambda.aws_lambda_provisioned_concurrency_config.myLambda: Still creating... [1m30s elapsed] module.myLambda.aws_lambda_provisioned_concurrency_config.myLambda: Still creating... [1m40s elapsed] module.myLambda.aws_lambda_provisioned_concurrency_config.myLambda: Still creating... [1m50s elapsed] module.myLambda.aws_lambda_provisioned_concurrency_config.myLambda: Still creating... [2m0s elapsed]

│ Error: error getting Lambda Provisioned Concurrency Config (arn:aws:lambda:*****): ValidationException: │ status code: 400, request id: da0e...

What's interesting is that the concurrency is actually provisioned successfully. Here's a screenshot from the console.

Screen Shot 2022-12-27 at 7 35 46 PM

If I make no changes and run the TF apply again, I get the the same error but this time during the plan stage.

│ Error: error getting Lambda Provisioned Concurrency Config (arn:aws:lambda:*****): ValidationException: │ status code: 400, request id: d07e...

I've tried to remove the provisioned concurrency from my TF code but the TF apply process always fails with the message above. The only thing I can do is manually delete the provisioned concurrency and modify the TF state in order to restore my ability to push changes.

I've also tried to customize the timeouts but the deployment still fails after 2 minutes of attempting to create the concurrency.

krikork avatar Dec 27 '22 17:12 krikork

I've noticed most issues in this project get a needs-triage label attached to them, and this one didn't. Maybe this is why it's still lingering unsolved, unsure. Does anyone know how to add that label to this issue?

sarahbeharryovo avatar Dec 28 '22 09:12 sarahbeharryovo

We have the same issue here (TF v0.12.6). We define an alias

resource "aws_lambda_alias" "lambda_alias" {
  name             = "${var.AliasName}"
  description      = "alias pinned to latest lambda version"
  function_name    = aws_lambda_function.lambda[0].function_name
  function_version = aws_lambda_function.lambda[0].version
}

We put this in a module and export the function name (alias_function_name) and version (alias_function_version) with output statements.

Then we want to provision a concurrency with this resource block:

resource "aws_lambda_provisioned_concurrency_config" "app_provisioning1" {
  function_name                     = module.app1.alias_function_name
  provisioned_concurrent_executions = 1
  qualifier                         = module.app1.alias_function_version
}

This leads to the concurrency being destroyed and recreated. On creation AWS needs a very long time to get this concurrency configuration done and Terraform always gives up too early. After 2 to 5 minutes we get the error message

Error: error waiting for Lambda Provisioned Concurrency Config (app_function_name1:74) to be ready: status reason: FUNCTION_ERROR_INIT_FAILURE

Setting the timeouts in aws_lambda_provisioned_concurrency_config resource does not change anything.

It seems AWS just takes a very long time to deploy this concurrency change and we can not make TF wait for it to complete. We tried about a dozen ways to make this work but without success.

xformerfhs avatar Feb 24 '23 15:02 xformerfhs

I believe the short answer here is everyone needs to update to the current Terraform version and Provider versions. we had this problem in 0.12x - 0.14.x before the 4.x provider was released. moving to the latest Terraform or at least 1.x or newer with the 4.x provider - we have not seen this issue in months.

tburow avatar Feb 24 '23 15:02 tburow

Thanks for your really quick answer, This will help me to convince our customer to migrate to a newer Terraform version.

xformerfhs avatar Feb 24 '23 16:02 xformerfhs

anyone with a fix, been trying for a week to solve it

IrvingRs avatar Mar 22 '23 23:03 IrvingRs

With terraform 1.3.9 and aws provider > 4.60 I also had this exact error: Error: error getting Lambda Provisioned Concurrency Config (arn:aws:lambda:*****): ValidationException @krikork

By using function NAME and not ARN this problem was fixed. Despite the docs (below) saying use either of them, for me only using the function name worked.

function_name - (Required) Name or Amazon Resource Name (ARN) of the Lambda Function.

BrianSigafoos avatar Apr 28 '23 16:04 BrianSigafoos

This functionality has been released in v5.4.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

github-actions[bot] avatar Jun 15 '23 23:06 github-actions[bot]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

github-actions[bot] avatar Jul 16 '23 02:07 github-actions[bot]