terraform-aws-lambda Provider produced inconsistent final plan for source_code

Description

I had an issue with changes and deployment of lambda function when using auto build. Maybe this is more a provoder bug?

Versions

Module version: 7.20.1
Terraform version:

Terraform v1.10.3
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.84.0
+ provider registry.terraform.io/hashicorp/external v2.3.4
+ provider registry.terraform.io/hashicorp/http v3.4.0
+ provider registry.terraform.io/hashicorp/local v2.5.2
+ provider registry.terraform.io/hashicorp/null v3.2.3
+ provider registry.terraform.io/hashicorp/random v3.6.0
+ provider registry.terraform.io/hashicorp/tls v4.0.4
+ provider registry.terraform.io/jianyuan/sentry v0.13.1

Provider version(s): Providers required by configuration:

.
├── provider[registry.terraform.io/hashicorp/random] 3.6.0
├── provider[registry.terraform.io/hashicorp/http] 3.4.0
├── provider[registry.terraform.io/hashicorp/tls] 4.0.4
├── provider[registry.terraform.io/jianyuan/sentry] 0.13.1
├── provider[registry.terraform.io/hashicorp/aws] ~> 5.0

Reproduction Code

module "cdn_function_reisemagazin_rewrite" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "7.20.1"

  function_name                = "cdn-reisemagazin-rewrite"
  description                  = "Rewrite headers"
  handler                      = "dist/main.handler"
  runtime                      = "nodejs22.x"
  trigger_on_package_timestamp = false
  recreate_missing_package     = false

  publish        = true
  lambda_at_edge = true
  source_path = [
    {
      path = "cdn-functions/reisemagazin-rewrite"
      commands = [
        "npm ci",
        "npm run build",
        "npm prune --production",
        ":zip"
      ]
      prefix_in_zip = "dist"
    }
  ]

  providers = {
    aws = aws.us-east-1
  }
  tags = local.holidayguruAllTags
}

Steps to reproduce the behavior:

terraform apply

Expected behavior

Should be deployed.

Actual behavior

Deployment fails. After two retries, it works fine.

Terminal Output Screenshot(s)

│ Error: Provider produced inconsistent final plan │ │ When expanding the plan for │ module.cdn_function_reisemagazin_rewrite.aws_lambda_function.this[0] to include new │ values learned so far during apply, provider │ "registry.terraform.io/hashicorp/aws" produced an invalid new value for │ .source_code_hash: was │ cty.StringVal("pL+RtsVm8jioo3I9uYiR4yPfc0hfbKtBz1zkSqbuMWk="), but now │ cty.StringVal("Gli5ygJZc6siVv3F42MWnJnROrQcNfTgVgzScmsw2CY="). │ │ This is a bug in the provider, which should be reported in the provider's own │ issue tracker.

Feb 05 '25 13:02 djakielski

I'm encountering this on a similar example. I ran into it specifically (so far) when I changed from true to false on trigger_on_package_timestamp and recreate_missing_package settings. I haven't tested it enough to know if it was only a one-time mishap when changing the settings but it happened consistently on every environment (like 5 times so far) where I changed the settings - had to re-run apply again and then it went through successfully.

Feb 12 '25 00:02 mmerickel

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days

Mar 15 '25 00:03 github-actions[bot]

Not stale

Mar 15 '25 05:03 mmerickel

having same issue as well. initial apply works fine. but once i modify the index.py script. i get this error

│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.backend_bucket_event_notification.module.lambda.aws_lambda_function.this[0] to include new values learned so far during apply, provider "registry.terraform.io/hashicorp/aws" produced an invalid new value for .source_code_hash: was
│ cty.StringVal("GdPtjAWNuoWRS6HyfU3p6IWp7r7ElPUKSwO2RYPim/Y="), but now cty.StringVal("iXM0Tq3dgmF/mSdlHY5KWpvwAaDcYwRl2wgz9ZuiiV8=").
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Apr 01 '25 05:04 cwong-archy

having same issue as well. initial apply works fine. but once i modify the index.py script. i get this error

│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.backend_bucket_event_notification.module.lambda.aws_lambda_function.this[0] to include new values learned so far during apply, provider "registry.terraform.io/hashicorp/aws" produced an invalid new value for .source_code_hash: was
│ cty.StringVal("GdPtjAWNuoWRS6HyfU3p6IWp7r7ElPUKSwO2RYPim/Y="), but now cty.StringVal("iXM0Tq3dgmF/mSdlHY5KWpvwAaDcYwRl2wgz9ZuiiV8=").
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

well. i re-ran terraform apply and it worked. ¯_(ツ)_/¯

Apr 01 '25 05:04 cwong-archy

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days

May 02 '25 00:05 github-actions[bot]

not stale

May 02 '25 01:05 mmerickel

same issue here

Terraform v1.11.4
on linux_amd64
+ provider registry.terraform.io/hashicorp/archive v2.7.0
+ provider registry.terraform.io/hashicorp/aws v5.97.0
+ provider registry.terraform.io/hashicorp/external v2.3.4
+ provider registry.terraform.io/hashicorp/local v2.5.2
+ provider registry.terraform.io/hashicorp/null v3.2.4

2025-05-02T16:58:51.255Z [WARN]  Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for module.lambda.aws_lambda_function.this[0], but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .image_uri: planned value cty.StringVal("") for a non-computed attribute
      - .kms_key_arn: planned value cty.StringVal("") for a non-computed attribute
      - .code_signing_config_arn: planned value cty.StringVal("") for a non-computed attribute
      - .skip_destroy: planned value cty.False for a non-computed attribute
      - .logging_config[0].application_log_level: planned value cty.StringVal("") for a non-computed attribute
      - .logging_config[0].system_log_level: planned value cty.StringVal("") for a non-computed attribute
      - .tracing_config: block count in plan (1) disagrees with count in config (0)

2025-05-02T16:58:51.255Z [ERROR] vertex "module.lambda.aws_lambda_function.this[0]" error: Provider produced inconsistent final plan

2025-05-02T16:58:51.257Z [DEBUG] states/remote: state read serial is: 7; serial is: 7
2025-05-02T16:58:51.257Z [DEBUG] states/remote: state read lineage is: 29383748-4a04-6c15-f754-46e34d5823d5; lineage is: 29383748-4a04-6c15-f754-46e34d5823d5

2025-05-02T16:58:51.262Z [INFO]  backend-s3: Uploading remote state: tf_backend.operation=Put tf_backend.req_id=01181a20-b5df-51ed-ebad-cfd59cce01e4 tf_backend.s3.bucket=<bucket_name> tf_backend.s3.path=<backend_path>/terraform.tfstate

2025-05-02T16:58:51.263Z [DEBUG] backend-s3: HTTP Request Sent: aws.region=eu-west-3 aws.s3.bucket=<bucket_name> aws.s3.key=<backend_path>/terraform.tfstate rpc.method=PutObject rpc.service=S3 rpc.system=aws-api tf_aws.sdk=aws-sdk-go-v2 tf_aws.signing_region="" tf_backend.operation=Put tf_backend.req_id=01181a20-b5df-51ed-ebad-cfd59cce01e4 tf_backend.s3.bucket=<bucket_name> tf_backend.s3.path=<backend_path>/terraform.tfstate net.peer.name=<bucket_name>.s3.eu-west-3.amazonaws.com http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.11.3 (+https://www.terraform.io/) aws-sdk-go-v2/1.36.0 ua/2.1 os/linux lang/go#1.23.3 md/GOOS#linux md/GOARCH#amd64 api/s3#1.75.2 ft/s3-transfer m/G,Z" http.request.header.x_amz_decoded_content_length=107951 http.request.header.x_amz_server_side_encryption=AES256 http.request.header.x_amz_date=20250502T165851Z http.method=PUT http.request.header.content_encoding=aws-chunked http.request.header.amz_sdk_invocation_id=3ce85f27-2f0d-4467-889c-d3d0dbe346c5 http.request.header.x_amz_content_sha256=STREAMING-UNSIGNED-PAYLOAD-TRAILER http.request.header.content_type=application/json http.request.body="[Redacted: 105.5 KB (108,033 bytes), Type: application/json]" http.request_content_length=108033 http.request.header.authorization="AWS4-HMAC-SHA256 Credential=ASIA************DT3X/20250502/eu-west-3/s3/aws4_request, SignedHeaders=accept-encoding;amz-sdk-invocation-id;amz-sdk-request;content-encoding;content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-sdk-checksum-algorithm;x-amz-security-token;x-amz-server-side-encryption;x-amz-trailer, Signature=*****" http.request.header.x_amz_security_token="*****" http.request.header.x_amz_sdk_checksum_algorithm=SHA256 http.request.header.accept_encoding=identity http.url=https://<bucket_name>.s3.eu-west-3.amazonaws.com/<backend_path>/terraform.tfstate?x-id=PutObject http.request.header.amz_sdk_request="attempt=1; max=5" http.request.header.x_amz_trailer=x-amz-checksum-sha256

2025-05-02T16:58:51.357Z [DEBUG] backend-s3: HTTP Response Received: aws.region=eu-west-3 aws.s3.bucket=<bucket_name> aws.s3.key=<backend_path>/terraform.tfstate rpc.method=PutObject rpc.service=S3 rpc.system=aws-api tf_aws.sdk=aws-sdk-go-v2 tf_aws.signing_region="" tf_backend.operation=Put tf_backend.req_id=01181a20-b5df-51ed-ebad-cfd59cce01e4 tf_backend.s3.bucket=<bucket_name> tf_backend.s3.path=<backend_path>/terraform.tfstate http.response.header.x_amz_request_id=SP1SH4D0HCN2N8TD http.response.header.etag="\"ee0c4cd70502159840cbaf12924656ad\"" http.response.header.date="Fri, 02 May 2025 16:58:52 GMT" http.response.header.x_amz_server_side_encryption=AES256 http.response.header.x_amz_checksum_sha256="1sN+4agm1SX8dnOk3PXVrQJt8MQfnzaorer58qRWoDg=" http.response.body="" http.duration=93 http.status_code=200 http.response.header.x_amz_version_id=LA595x2bC6kqWSzFpoPSZa50iQQm3TZp http.response.header.x_amz_id_2="ZEwPXPfX9lLf5o6D8XxrNE3KgS9ptRWt3CiFtBVatrlZTpP/UUlaWdoyO8ZiUbp3x9xkJEFze78gLOvYJ4vLaOh1wizQ/J+J" http.response.header.server=AmazonS3 http.response.header.x_amz_checksum_type=FULL_OBJECT

│ Error: Provider produced inconsistent final plan
│ 
│ When expanding the plan for module.lambda.aws_lambda_function.this[0] to
│ include new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .source_code_hash: was
│ cty.StringVal("eJtNL22SQynso3BANmWCGaA4B3q7/i/SoxddGABd3js="), but now
│ cty.StringVal("ZZXy+dAo8vXB0H0JrTb6RUdwK5po5pfVTPmWlEtfwvA=").
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.

I have this issue when I'm on a multi stage within my Azure Devops CICD I do not have the issue locally when I perform an apply directly (apply -auto-approve after doing a plan before)

May 02 '25 17:05 remi-f-artelia

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days

Jun 02 '25 00:06 github-actions[bot]

Not stale.

Jun 03 '25 17:06 jkampfwealthbox

In case it helps. I ran into same error setting up a new pipeline with separate steps for plan and apply. I had forgotten to include the folder with lambdas pre-packaged as an artifact between steps. ie. the plan had the prepackaged lambda, but the apply did not.

Jun 27 '25 03:06 chris-mackubin

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days

Jul 28 '25 00:07 github-actions[bot]

This issue was automatically closed because of stale in 10 days

Aug 08 '25 00:08 github-actions[bot]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

Sep 07 '25 02:09 github-actions[bot]

Provider produced inconsistent final plan for source_code_hash

Description

Versions

Reproduction Code

Expected behavior

Actual behavior

Terminal Output Screenshot(s)