data "aws_lambda_invocation" "invoke_codebuild_job" The aft-lambda-layer-codebuild-invocations function is looping

[jannyg]

Terraform Version & Prov:

AFT Version: 1.5.1

Terraform Version & Provider Versions Please provide the outputs of terraform version and terraform providers from within your AFT environment

terraform version

Terraform v1.2.4
on darwin_amd64
+ provider registry.terraform.io/cloudposse/awsutils v0.11.1
+ provider registry.terraform.io/hashicorp/archive v2.2.0
+ provider registry.terraform.io/hashicorp/aws v4.22.0
+ provider registry.terraform.io/hashicorp/local v2.2.3
+ provider registry.terraform.io/hashicorp/null v3.1.1
+ provider registry.terraform.io/hashicorp/random v3.3.2
+ provider registry.terraform.io/hashicorp/time v0.7.2

terraform providers

Providers required by configuration:
.
├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
├── provider[registry.terraform.io/hashicorp/random] ~> 3.0
├── module.remote_state_replica
│   ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│   ├── provider[registry.terraform.io/hashicorp/time] >= 0.7.0
│   ├── module.s3_user
│   │   ├── provider[registry.terraform.io/hashicorp/aws] >= 2.0.0
│   │   ├── module.s3_user
│   │   │   ├── provider[registry.terraform.io/hashicorp/aws] >= 2.0.0
│   │   │   ├── provider[registry.terraform.io/cloudposse/awsutils] >= 0.11.0
│   │   │   ├── module.store_write
│   │   │   │   ├── provider[registry.terraform.io/hashicorp/local] >= 1.2.0
│   │   │   │   ├── provider[registry.terraform.io/hashicorp/null] >= 2.0.0
│   │   │   │   ├── provider[registry.terraform.io/hashicorp/aws] >= 2.0.0
│   │   │   │   └── module.this
│   │   │   └── module.this
│   │   └── module.this
│   └── module.this
├── module.this
├── module.aft
│   ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0, < 5.0.0
│   ├── provider[registry.terraform.io/hashicorp/local]
│   ├── module.aft_feature_options
│       └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│   ├── module.aft_lambda_layer
│       ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│       ├── provider[registry.terraform.io/hashicorp/random]
│       └── provider[registry.terraform.io/hashicorp/local]
│   ├── module.aft_code_repositories
│       ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│       └── provider[registry.terraform.io/hashicorp/local]
│   ├── module.aft_customizations
│       ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│       └── provider[registry.terraform.io/hashicorp/local]
│   ├── module.packaging
│       └── provider[registry.terraform.io/hashicorp/archive]
│   ├── module.aft_account_request_framework
│       ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│       └── provider[registry.terraform.io/hashicorp/time]
│   ├── module.aft_backend
│       └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│   ├── module.aft_ssm_parameters
│       └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│   ├── module.aft_account_provisioning_framework
│       └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│   └── module.aft_iam_roles
│       ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│       ├── module.aft_service_role
│           └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│       ├── module.audit_exec_role
│           └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│       ├── module.audit_service_role
│           └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│       ├── module.ct_management_exec_role
│           └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│       ├── module.ct_management_service_role
│           └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│       ├── module.log_archive_exec_role
│           └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│       ├── module.log_archive_service_role
│           └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
│       └── module.aft_exec_role
│           └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0
└── module.remote_state
    ├── provider[registry.terraform.io/hashicorp/local] >= 1.3.0
    ├── provider[registry.terraform.io/hashicorp/aws] >= 2.0.0
    ├── module.dynamodb_table_label
    ├── module.log_storage
        ├── provider[registry.terraform.io/hashicorp/time] >= 0.7.0
        ├── provider[registry.terraform.io/hashicorp/aws] >= 3.0.0
        └── module.this
    └── module.this

Providers required by state:

    provider[registry.terraform.io/hashicorp/archive]

    provider[registry.terraform.io/hashicorp/local]

    provider[registry.terraform.io/hashicorp/time]

    provider[registry.terraform.io/hashicorp/aws]

    provider[registry.terraform.io/hashicorp/random]

Bug Description invoke_codebuild_job is running on every plan and apply. The layer is created and uploaded successfully to s3, but the layer version in Lambda is not updated after the first initial run (e.g when changing versions of AFT). From the logs it could look like there's no output from the module (aws_lambda_layer_version.layer_version.arn) which in turn gives empty var.aft_common_layer_arn

I have tested this on both 1.5.1 and 1.4.2 with the same issue. I also experience the same issue both when running terraform locally and running from Github Actions. I have also tested on a different network to ensure that there's no proxy issues.

To Reproduce Steps to reproduce the behavior:

1. Deploy AFT 1.5.1 or 1.4.2
2. Run terraform plan and module.aft.module.aft_lambda_layer.data.aws_lambda_invocation.invoke_codebuild_job should be reading each plan and apply run.

Expected behavior Only be applied if there are any changes and not to be invoked on every plan.

Related Logs I can share terraform debug logs in another channel to avoid masking all accounts numbers and other things are deployed at the same time. I will continue to test and debug, but creating this here in case other have similar issues and has found a solution.

Additional context

[snebhu3]

@jannyg thank you for reaching out. I suspect there could be certain network configuration in the environment causing this. However, I would recommend reaching out to AWS Premium support for better assistance on this issue as they would be able to dive deeper and help troubleshoot the issue.

[jannyg]

Thanks @snebhu3, I will give that a shot and report back. Not sure with the network issue could be as this is a fresh account for AFT and has not been used for anything else. It could be related to the aft_feature_delete_default_vpcs_enabled being set to true maybe. If support can figure it out, I will try with a fresh environment when I'm allowed to close enough accounts.