terraform-provider-awscc awscc_redshiftserverless_namespace - "NotUpdatableException: │ Invalid patch update: createOnlyProperties [/properties/Tags] cannot be updated"

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment
The resources and data sources in this provider are generated from the CloudFormation schema, so they can only support the actions that the underlying schema supports. For this reason submitted bugs should be limited to defects in the generation and runtime code of the provider. Customizing behavior of the resource, or noting a gap in behavior are not valid bugs and should be submitted as enhancements to AWS via the CloudFormation Open Coverage Roadmap.

Terraform CLI and Terraform AWS Cloud Control Provider Version

Terraform v1.9.3 on linux_amd64

awscc = {
      source  = "hashicorp/awscc"
      version = "1.7.0"
    }

Affected Resource(s)

awscc_redshiftserverless_namespace

Terraform Configuration Files

terraform {
  required_providers {
    awscc = {
      source  = "hashicorp/awscc"
      version = "1.7.0"
    }
  }
}
variable "target_account_id" {} # TF_VAR_target_account_id
variable "iam_assume_role" {}   # TF_VAR_iam_assume_role

provider "awscc" {
  region = "eu-west-1"

  assume_role = {
    role_arn = "arn:aws:iam::${var.target_account_id}:role/${var.iam_assume_role}"
  }
}

locals {
  tags = {
    terraform   = "true"
    environment = "test"
    workspace   = terraform.workspace
    project     = "hs-test"
  }
  awscc_tags = [for k, v in local.tags : { key : k, value : v }]
}

resource "awscc_redshiftserverless_namespace" "this" {
  namespace_name = "hs-testing-123"

  #   log_exports = toset(["userlog", "connectionlog", "useractivitylog"])
  log_exports = ["userlog", "connectionlog", "useractivitylog"]

  tags = local.awscc_tags
}

Debug Output

https://gist.github.com/hszalwinski/079b98bb2ba5c935c5e65ce7118a18f8
https://gist.github.com/hszalwinski/768575fa8a43e39edef275622c94a96f
https://gist.github.com/hszalwinski/1ea3af8016959c0bb8bfef73c73ffd37

Expected Behavior

Docs state, that log_exports should be a List of String. I would expect either the documentation to be fixed or implementation to handle the bug case.

Additionally and even more important, it is not only an issue with log_exports. When I try to update existing resource by e.g. changing the admin_password_secret_kms_key_id or adding another role to iam_roles. I receive the same error from Cloud Control API. I expect the issue not to appear, so no in-place updates should be blocked by this.

Actual Behavior and Steps to Reproduce

When I create a awscc_redshiftserverless_namespace providing log_exports as a list the terraform apply command succeeds (1st gist).
When I hit this command once again I expect No changes. output like in 3rd gist. Instead of this, I receive an error (2nd gist):

Error: AWS SDK Go Service Operation Unsuccessful
│ 
│   with awscc_redshiftserverless_namespace.this,
│   on main.tf line 31, in resource "awscc_redshiftserverless_namespace" "this":
│   31: resource "awscc_redshiftserverless_namespace" "this" {
│ 
│ Calling Cloud Control API service UpdateResource operation returned: operation error CloudControl: UpdateResource, https response error StatusCode: 400, RequestID: 62f9e2fb-99c4-4506-bd26-b87dd844a73e, NotUpdatableException:
│ Invalid patch update: createOnlyProperties [/properties/Tags] cannot be updated

If I change the log_exports from list to a set and run terraform apply once again it produces desired output. I've also checked, that it works okay if I use set from the beginning.

Jul 30 '24 09:07 hszalwinski

Thank you for opening the issue @hszalwinski .I can reproduce it and it does look like the ordering on the log_exports leading to a drift. Adding some additional details for further debugging.

Debug log

2024-07-30T18:04:18.156-0400 [DEBUG] provider.terraform-provider-awscc_v1.7.0_x5: 
Detected value change between proposed new state and prior state: 
@module=sdk.framework tf_req_id=0804e250-80f3-4008-8c50-77c82671d84e 
@caller=github.com/hashicorp/[email protected]/internal/fwserver/server_planresourcechange.go:208 
tf_attribute_path=log_exports 
tf_provider_addr=registry.terraform.io/hashicorp/awscc 
tf_resource_type=awscc_redshiftserverless_namespace 
tf_rpc=PlanResourceChange timestamp=2024-07-30T18:04:18.156-0400

PlanResourceChange_Request_PriorState


{
  "admin_password_secret_kms_key_id": null,
  "admin_user_password": null,
  "admin_username": "admin",
  "db_name": null,
  "default_iam_role_arn": null,
  "final_snapshot_name": null,
  "final_snapshot_retention_period": null,
  "iam_roles": null,
  "id": "hs-testing-123",
  "kms_key_id": "AWS_OWNED_KMS_KEY",
  "log_exports": [
    "connectionlog",
    "useractivitylog",
    "userlog"
  ],
  "manage_admin_password": null,
  "namespace": {
    "admin_password_secret_arn": null,
    "admin_password_secret_kms_key_id": null,
    "admin_username": "admin",
    "creation_date": "2024-07-30T22:03:41.759Z",
    "db_name": null,
    "default_iam_role_arn": null,
    "iam_roles": null,
    "kms_key_id": "AWS_OWNED_KMS_KEY",
    "log_exports": [
      "connectionlog",
      "useractivitylog",
      "userlog"
    ],
    "namespace_arn": "arn:aws:redshift-serverless:us-east-1:############:namespace/74e6ded1-fd65-43c3-b79a-76e9a18cde38",
    "namespace_id": "74e6ded1-fd65-43c3-b79a-76e9a18cde38",
    "namespace_name": "hs-testing-123",
    "status": "AVAILABLE"
  },
  "namespace_name": "hs-testing-123",
  "namespace_resource_policy": null,
  "redshift_idc_application_arn": null,
  "snapshot_copy_configurations": null,
  "tags": [
    {
      "key": "environment",
      "value": "test"
    },
    {
      "key": "project",
      "value": "hs-test"
    },
    {
      "key": "terraform",
      "value": "true"
    },
    {
      "key": "workspace",
      "value": "default"
    }
  ]
}

PlanResourceChange_Request_ProposedNewState

{
  "admin_password_secret_kms_key_id": null,
  "admin_user_password": null,
  "admin_username": "admin",
  "db_name": null,
  "default_iam_role_arn": null,
  "final_snapshot_name": null,
  "final_snapshot_retention_period": null,
  "iam_roles": null,
  "id": "hs-testing-123",
  "kms_key_id": "AWS_OWNED_KMS_KEY",
  "log_exports": [
    "userlog",
    "connectionlog",
    "useractivitylog"
  ],
  "manage_admin_password": null,
  "namespace": {
    "admin_password_secret_arn": null,
    "admin_password_secret_kms_key_id": null,
    "admin_username": "admin",
    "creation_date": "2024-07-30T22:03:41.759Z",
    "db_name": null,
    "default_iam_role_arn": null,
    "iam_roles": null,
    "kms_key_id": "AWS_OWNED_KMS_KEY",
    "log_exports": [
      "connectionlog",
      "useractivitylog",
      "userlog"
    ],
    "namespace_arn": "arn:aws:redshift-serverless:us-east-1:############:namespace/74e6ded1-fd65-43c3-b79a-76e9a18cde38",
    "namespace_id": "74e6ded1-fd65-43c3-b79a-76e9a18cde38",
    "namespace_name": "hs-testing-123",
    "status": "AVAILABLE"
  },
  "namespace_name": "hs-testing-123",
  "namespace_resource_policy": null,
  "redshift_idc_application_arn": null,
  "snapshot_copy_configurations": null,
  "tags": [
    {
      "key": "environment",
      "value": "test"
    },
    {
      "key": "project",
      "value": "hs-test"
    },
    {
      "key": "terraform",
      "value": "true"
    },
    {
      "key": "workspace",
      "value": "default"
    }
  ]
}

PlanResourceChange_Response_PlannedState


{
  "admin_password_secret_kms_key_id": "\u0000",
  "admin_user_password": "\u0000",
  "admin_username": "admin",
  "db_name": "\u0000",
  "default_iam_role_arn": "\u0000",
  "final_snapshot_name": "\u0000",
  "final_snapshot_retention_period": "\u0000",
  "iam_roles": "\u0000",
  "id": "hs-testing-123",
  "kms_key_id": "AWS_OWNED_KMS_KEY",
  "log_exports": [
    "connectionlog",
    "useractivitylog",
    "userlog"
  ],
  "manage_admin_password": "\u0000",
  "namespace": {
    "admin_password_secret_arn": null,
    "admin_password_secret_kms_key_id": null,
    "admin_username": "admin",
    "creation_date": "2024-07-30T22:03:41.759Z",
    "db_name": null,
    "default_iam_role_arn": null,
    "iam_roles": null,
    "kms_key_id": "AWS_OWNED_KMS_KEY",
    "log_exports": [
      "connectionlog",
      "useractivitylog",
      "userlog"
    ],
    "namespace_arn": "arn:aws:redshift-serverless:us-east-1:############:namespace/74e6ded1-fd65-43c3-b79a-76e9a18cde38",
    "namespace_id": "74e6ded1-fd65-43c3-b79a-76e9a18cde38",
    "namespace_name": "hs-testing-123",
    "status": "AVAILABLE"
  },
  "namespace_name": "hs-testing-123",
  "namespace_resource_policy": "\u0000",
  "redshift_idc_application_arn": "\u0000",
  "snapshot_copy_configurations": "\u0000",
  "tags": [
    {
      "key": "environment",
      "value": "test"
    },
    {
      "key": "project",
      "value": "hs-test"
    },
    {
      "key": "terraform",
      "value": "true"
    },
    {
      "key": "workspace",
      "value": "default"
    }
  ]
}

Jul 30 '24 22:07 quixoticmonk

From the schema, insertionorder shouldn't be a factor.

"LogExports": {
                    "type": "array",
                    "insertionOrder": false,
                    "items": {
                        "$ref": "#/definitions/LogExport"
                    },
                    "maxItems": 16,
                    "minItems": 0
                },

Statefile has

      "instances": [
        {
          "schema_version": 1,
          "attributes": {
            "admin_password_secret_kms_key_id": null,
            ....
            "log_exports": [
              "userlog",
              "connectionlog",
              "useractivitylog"
            ],
            "manage_admin_password": null,
            "namespace": {
              ....
              "log_exports": [
                "connectionlog",
                "useractivitylog",
                "userlog"
              ],

Jul 30 '24 22:07 quixoticmonk

The get-resource response ( on refresh or subsequent plan/apply) shows the ordered list and hence provider is showing a drift.

aws cloudcontrol get-resource --type-name AWS::RedshiftServerless::Namespace --identifier "hs-testing-123" | jq .ResourceDescription.Properties

"{\"AdminUsername\":\"admin\",\"NamespaceName\":\"hs-testing-123\",\"IamRoles\":[],\"SnapshotCopyConfigurations\":[],\"KmsKeyId\":\"AWS_OWNED_KMS_KEY\",\"Namespace\":{\"Status\":\"AVAILABLE\",\"NamespaceName\":\"hs-testing-123\",\"AdminUsername\":\"admin\",\"CreationDate\":\"2024-07-30T22:20:59.326Z\",\"IamRoles\":[],\"NamespaceArn\":\"arn:aws:redshift-serverless:us-east-1:############:namespace/95549688-0f55-48ed-9a2b-744e0b33b4d4\",\"KmsKeyId\":\"AWS_OWNED_KMS_KEY\",\"NamespaceId\":\"95549688-0f55-48ed-9a2b-744e0b33b4d4\",\"LogExports\":[\"connectionlog\",\"useractivitylog\",\"userlog\"]},\"LogExports\":[\"connectionlog\",\"useractivitylog\",\"userlog\"]}"

translates to ..

        "NamespaceId": "95549688-0f55-48ed-9a2b-744e0b33b4d4",
        "LogExports": [
            "connectionlog",
            "useractivitylog",
            "userlog"
        ]
    },
    "LogExports": [
        "connectionlog",
        "useractivitylog",
        "userlog"
    ]

Jul 30 '24 22:07 quixoticmonk

This needs some review as Tags on namespace is an replace operation for the resource from the schema and it would fail with a resource with a custom name.

Opened an internal service ticket to review the schema around log_exports and tags being a replace op.

Jul 30 '24 22:07 quixoticmonk

The secondary issue on the tags being added in the patch operation when anything changes needs to be reviewed.

Jul 30 '24 22:07 quixoticmonk

Hello! Any news or fix time estimations @quixoticmonk? :)

Oct 23 '24 07:10 hszalwinski