terraform-provider-aws icon indicating copy to clipboard operation
terraform-provider-aws copied to clipboard

[Bug]: Publishing Lambda Function (<name>) version: waiting for completion: exceeded max wait time for FunctionUpdated waiter

Open AlisherUrunovMimacom opened this issue 1 year ago • 4 comments

Terraform Core Version

1.9.5

AWS Provider Version

5.60.0

Affected Resource(s)

  • aws_lambda_function
  • aws_lambda_alias
  • aws_lambda_provisioned_concurrency_config

Expected Behavior

No issues during apply

Actual Behavior

First apply times out on aws_lambda_function resource. Second apply goes through. During apply provisioned concurrency is marked as ready in console UI.

Relevant Error/Panic Output Snippet

Error: publishing Lambda Function (<name>) version: waiting for completion: exceeded max wait time for FunctionUpdated waiter
--
1913 |  
1914 | with module.whatever_service.aws_lambda_function.lambda,
1915 | on ../../modules/lambda/lambda.tf line 7, in resource "aws_lambda_function" "lambda":
1916 | 7: resource "aws_lambda_function" "lambda" {

Terraform Configuration Files

resource "aws_lambda_function" "lambda" {
  function_name       = "${var.function_name}_${var.environment_name}"
  description             = var.function_description
  role                          = aws_iam_role.lambda.arn
  handler                    = "main.handler"
  runtime                    = var.runtime
  timeout                    = var.timeout
  memory_size           = var.memory_size
  filename                   = data.archive_file.artifact.output_path
  source_code_hash = data.archive_file.artifact.output_base64sha256
  publish                      = true

  environment {
    variables = var.environment_variables
  }

  vpc_config {
    security_group_ids  = [aws_security_group.lambda.id]
    subnet_ids               = data.aws_subnets.subnets-private-v2.ids
  }
}

data "archive_file" "artifact" {
  type        = "zip"
  output_path = "${path.module}/../../../../dist/apps/${var.app_name}.zip"
  source_dir  = "${path.module}/../../../../dist/apps/${var.app_name}"
}

resource "aws_lambda_alias" "lambda" {
  name                    = "alias_concurrency"
  description          = "Lambda alias used for concurrency control of the latest version of the lambda function."
  function_name    = aws_lambda_function.lambda.function_name
  function_version = aws_lambda_function.lambda.version
}

resource "aws_lambda_provisioned_concurrency_config" "lambda" {
  function_name                                    = aws_lambda_function.lambda.function_name
  qualifier                                                = aws_lambda_alias.lambda.name
  provisioned_concurrent_executions = var.provisioned_concurrent_executions
}

Steps to Reproduce

terraform init terraform apply -auto-approve

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

AlisherUrunovMimacom avatar Aug 26 '24 09:08 AlisherUrunovMimacom

Community Note

Voting for Prioritization

  • Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

  • If you are interested in working on this issue, please leave a comment.
  • If this would be your first contribution, please review the contribution guide.

github-actions[bot] avatar Aug 26 '24 09:08 github-actions[bot]

Hey @AlisherUrunovMimacom 👋 Thank you for taking the time to raise this! The aws_lambda_function resources has configurable timeouts, including for create. Can you configure explicit timeout(s) and let us know if that resolves the issue?

justinretzolk avatar Aug 26 '24 19:08 justinretzolk

Hi @justinretzolk ,

We did try this out as well and increased create and update timeouts for lambda to 30 min. But terraform just was hanging forever.

PS I also want to outline that from AWS console everything looks ready and updated after some minutes.

ghost avatar Aug 27 '24 06:08 ghost

Thanks for the update @AlisherUrunovMimacom! Are you able to provide debug logs (redacted as needed) so that whoever picks this up has the information they need in order to look into it? For this particular resource, things can get pretty configuration-specific fairly quickly (given that you're uploading a file, publishing, etc.), so that logging may prove to be critical information.

justinretzolk avatar Aug 27 '24 14:08 justinretzolk

Facing the same issue lambda publishing just hangs there, we had to create lambda function on every attempt of this to bypass this issue.

Vishnunkumar avatar Sep 02 '24 10:09 Vishnunkumar

Facing the same issue lambda publishing just hangs there, we had to create lambda function on every attempt of this to bypass this issue.

Update: We had an issue with IAM policies as one of the required permission was missing for the publishing lambda part. lambda:getFunctionConfiguration was missing. It was available in the logs tagged as DEBUG while the same was not thrown out as an ERROR hence we took some to resolve it. Suggestion: Tagging logs can be made better from terraform

Vishnunkumar avatar Sep 02 '24 14:09 Vishnunkumar

Thanks for following up here @Vishnunkumar!

@AlisherUrunovMimacom -- can you review your debug logging and see if you're experiencing the same issue as mentioned above?

justinretzolk avatar Sep 03 '24 16:09 justinretzolk

This issue was solved for me after adding the lambda:GetFunctionConfiguration permission to the iam management role, thanks to @Vishnunkumar comment - https://github.com/hashicorp/terraform-provider-aws/issues/39037#issuecomment-2324849420 :)

jackwills04 avatar Sep 07 '24 11:09 jackwills04

Sorry for late response, it is indeed a missing permissions. Thanks for some insights.

ghost avatar Sep 09 '24 07:09 ghost

[!WARNING] This issue has been closed, meaning that any additional comments are hard for our team to see. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

github-actions[bot] avatar Sep 09 '24 07:09 github-actions[bot]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

github-actions[bot] avatar Oct 10 '24 02:10 github-actions[bot]