Unstable json in s3 bucket policy

[booi]

Describe the Bug

Applying this terraform module appears to work fine but the s3 bucket policy json is unstable and terraform wants to update it every run.

Expected Behavior

Subsequent terraform runs should not trigger updates to the provisioned resources

Steps to Reproduce

1. Provision

module "cloudtrail_s3_bucket" {
  source = "cloudposse/cloudtrail-s3-bucket/aws"
  version = "0.23.1"

  name                     = "cloudtrail_logs"
  stage                    = var.env_name
  namespace                = var.namespace
  standard_transition_days = 30
  glacier_transition_days  = 60
  expiration_days          = 365
  force_destroy = var.force_destroy
}

module "cloudtrail" {
  source = "cloudposse/cloudtrail/aws"
  version = "0.21.0"

  namespace = var.namespace
  stage = var.env_name
  name = "cloudtrail"
  enable_log_file_validation    = true
  include_global_service_events = true
  is_multi_region_trail         = true
  s3_bucket_name                = module.cloudtrail_s3_bucket.bucket_id
}

2. terraform apply This correctly creates all resources.

3. terraform apply

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

  # module.cloudtrail.module.cloudtrail_s3_bucket.module.s3_bucket.aws_s3_bucket.default[0] has been changed
  ~ resource "aws_s3_bucket" "default" {
        id                          = "REDACTED-prod-cloudtraillogs"
      ~ policy                      = jsonencode(
          ~ {
              ~ Statement = [
                    {
                        Action    = "s3:GetBucketAcl"
                        Effect    = "Allow"
                        Principal = {
                            Service = "cloudtrail.amazonaws.com"
                        }
                        Resource  = "arn:aws:s3:::REDACTED-prod-cloudtraillogs"
                        Sid       = "AWSCloudTrailAclCheck"
                    },
                  ~ {
                      ~ Principal = {
                          ~ Service = [
                              - "cloudtrail.amazonaws.com",
                                "config.amazonaws.com",
                              + "cloudtrail.amazonaws.com",
                            ]
                        }
                        # (5 unchanged elements hidden)
                    },
                  + {
                      + Action    = "s3:*"
                      + Condition = {
                          + Bool = {
                              + aws:SecureTransport = "false"
                            }
                        }
                      + Effect    = "Deny"
                      + Principal = "*"
                      + Resource  = [
                          + "arn:aws:s3:::REDACTED-prod-cloudtraillogs/*",
                          + "arn:aws:s3:::REDACTED-prod-cloudtraillogs",
                        ]
                      + Sid       = "ForceSSLOnlyAccess"
                    },
                ]
                # (1 unchanged element hidden)
            }
        )
        tags                        = {
            "Name"      = "REDACTED-prod-cloudtraillogs"
            "Namespace" = "REDACTED"
            "Stage"     = "prod"
        }
        # (10 unchanged attributes hidden)



        # (3 unchanged blocks hidden)
    }
  # module.cloudtrail.module.cloudtrail_s3_bucket.module.s3_bucket.aws_s3_bucket_policy.default[0] has been changed
  ~ resource "aws_s3_bucket_policy" "default" {
        id     = "REDACTED-prod-cloudtraillogs"
      ~ policy = jsonencode(
          ~ {
              ~ Statement = [
                    {
                        Action    = "s3:GetBucketAcl"
                        Effect    = "Allow"
                        Principal = {
                            Service = "cloudtrail.amazonaws.com"
                        }
                        Resource  = "arn:aws:s3:::REDACTED-prod-cloudtraillogs"
                        Sid       = "AWSCloudTrailAclCheck"
                    },
                  ~ {
                      ~ Condition = {
                          ~ StringEquals = {
                              ~ s3:x-amz-acl = [
                                  - "bucket-owner-full-control",
                                ] -> "bucket-owner-full-control"
                            }
                        }
                      ~ Principal = {
                          ~ Service = [
                              - "config.amazonaws.com",
                                "cloudtrail.amazonaws.com",
                              + "config.amazonaws.com",
                            ]
                        }
                        # (4 unchanged elements hidden)
                    },
                  ~ {
                      ~ Condition = {
                          ~ Bool = {
                              ~ aws:SecureTransport = [
                                  - "false",
                                ] -> "false"
                            }
                        }
                        # (5 unchanged elements hidden)
                    },
                ]
                # (1 unchanged element hidden)
            }
        )
        # (1 unchanged attribute hidden)
    }

The changes seem to be essentially no-ops but it would be great if they were consistent. It appears that amazon reformats inputted JSON and this collides with what is generated by terraform.

[ivanmartos]

any update pls?

[leb4r]

Is this issue actually associated with https://github.com/cloudposse/terraform-aws-cloudtrail-s3-bucket instead ?

[enver-multibank]

Problem is indeed in https://github.com/cloudposse/terraform-aws-cloudtrail-s3-bucket. Latest version of this module uses s3-log-storage module version 0.26 (https://github.com/cloudposse/terraform-aws-cloudtrail-s3-bucket/blob/master/main.tf#L13) which is managing bucket policy in 2 different places: https://github.com/cloudposse/terraform-aws-s3-log-storage/blob/0.26.0/main.tf#L8 and https://github.com/cloudposse/terraform-aws-s3-log-storage/blob/0.26.0/main.tf#L162

Solution is to upgrade version of terraform-aws-s3-log-storage to 0.27 (at least) in terraform-aws-cloudtrail-s3-bucket module. However 0.27 is preparation release for upgrade of aws provider to version 4, so it will require some manual state changes :(

Update: There is a pull request: https://github.com/cloudposse/terraform-aws-cloudtrail-s3-bucket/pull/59 which seems to be stalled.

[Nuru]

This issue relates to terraform-aws-s3-log-storage, not this module, and has been fixed.