[Bug]: r/aws_redshiftserverless_workgroup: Removal of `config_parameter` keys cause perpetual diff

[jar-b]

Terraform Core Version

1.6.0

AWS Provider Version

5.20.1

Affected Resource(s)

• aws_redshiftserverless_workgroup

Expected Behavior

When a query monitoring metric config_parameter with a numeric value (i.e. max_return_row_count) is removed, the update operation should unset it on the remote workgroup.

Actual Behavior

The config_parameter remains in place.

From observing how parameters are removed in the AWS console, it appears the desired config_value should be set to -1 to remove it (versus omitting it from the input altogether). For example, here is the request payload from the console operation where max_return_row_count was removed. All available parameters are sent with a value of -1, except for max_query_execution_time, which has a configured value.

{"configParameters":[{"parameterKey":"max_query_cpu_time","parameterValue":"-1"},{"parameterKey":"max_query_blocks_read","parameterValue":"-1"},{"parameterKey":"max_scan_row_count","parameterValue":"-1"},{"parameterKey":"max_query_queue_time","parameterValue":"-1"},{"parameterKey":"max_query_cpu_usage_percent","parameterValue":"-1"},{"parameterKey":"max_query_temp_blocks_to_disk","parameterValue":"-1"},{"parameterKey":"max_join_row_count","parameterValue":"-1"},{"parameterKey":"max_nested_loop_join_row_count","parameterValue":"-1"},{"parameterKey":"max_return_row_count","parameterValue":"-1"},{"parameterKey":"max_query_execution_time","parameterValue":"100"}],"workgroupName":"jb-test"}

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

# Configure the AWS Provider
provider "aws" {}

resource "aws_redshiftserverless_namespace" "test" {
  namespace_name = "jb-test"
}

resource "aws_redshiftserverless_workgroup" "test" {
  namespace_name = aws_redshiftserverless_namespace.test.id
  workgroup_name = "jb-test"
  base_capacity  = 8


  config_parameter {
    parameter_key   = "auto_mv"
    parameter_value = "true"
  }

  config_parameter {
    parameter_key   = "datestyle"
    parameter_value = "ISO, MDY"
  }

  config_parameter {
    parameter_key   = "enable_case_sensitive_identifier"
    parameter_value = "false"
  }

  config_parameter {
    parameter_key   = "enable_user_activity_logging"
    parameter_value = "true"
  }

  config_parameter {
    parameter_key   = "query_group"
    parameter_value = "default"
  }

  config_parameter {
    parameter_key   = "search_path"
    parameter_value = "$user, public"
  }

  # set this in first apply, then remove it
  config_parameter {
    parameter_key   = "max_query_execution_time"
    parameter_value = "100"
  }
}

Steps to Reproduce

1. Create a workgroup with a numeric config_parameter
2. terraform apply
3. Remove the parameter from Terraform configuration
4. terraform apply
5. terraform plan
6. Observe persistent diff

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[dima-naboka]

There are more issues with how config_parameter{} is implemented

After workgroup is created with one of custom params subsequent TF plan will suggest to overwrite default params with a single custom param

~ config_parameter {
          ~ parameter_key   = "auto_mv" -> "max_query_execution_time"
          ~ parameter_value = "true" -> "300"
        }
      - config_parameter {
          - parameter_key   = "datestyle" -> null
          - parameter_value = "ISO, MDY" -> null
        }
      - config_parameter {
          - parameter_key   = "enable_case_sensitive_identifier" -> null
          - parameter_value = "false" -> null
        }
...

which will fail due to https://docs.aws.amazon.com/redshift-serverless/latest/APIReference/API_UpdateWorkgroup.html ValidationException: Can't update multiple configurations at the same time for workgroup

Unfortunately, you can't manually define default params. Hence, no workaround there expected config_parameter.0.parameter_key to be one of [datestyle enable_user_activity_logging query_group search_path max_query_execution_time], got auto_mv

[Ownmarc]

Hello! Encountered this as I was trying to deploy a Redshift cluster for Zero ETL integration from a MySql rds database. I need to set all the defaults parameters in order to not trigger an update that will cause an error on every deploy. See https://github.com/pulumi/pulumi-aws/issues/4405 for more details.

[OJFord]

It seems all the defaults are read back in, so any unspecified parameters are to be 'changed' back to null:

       - config_parameter {
          - parameter_key   = "auto_mv" -> null
          - parameter_value = "true" -> null
        }
      - config_parameter {
          - parameter_key   = "datestyle" -> null
          - parameter_value = "ISO, MDY" -> null
        }
      - config_parameter {
          - parameter_key   = "enable_user_activity_logging" -> null
          - parameter_value = "true" -> null
        }
      - config_parameter {
          - parameter_key   = "max_query_execution_time" -> null
          - parameter_value = "14400" -> null
        }
      - config_parameter {
          - parameter_key   = "query_group" -> null
          - parameter_value = "default" -> null
        }
      - config_parameter {
          - parameter_key   = "require_ssl" -> null
          - parameter_value = "true" -> null
        }
      - config_parameter {
          - parameter_key   = "search_path" -> null
          - parameter_value = "$user, public" -> null
        }
      - config_parameter {
          - parameter_key   = "use_fips_ssl" -> null
          - parameter_value = "false" -> null
        }
        # (2 unchanged blocks hidden)
    }

(here only enable_case_sensitive_identifier is specified in config)

This is not just an aesthetic issue in the plan, it causes apply to error:

│ Error: updating Redshift Serverless Workgroup (aurora-etl): operation error Redshift Serverless: UpdateWorkgroup, https response error StatusCode: 400, [...] ValidationException: You didn't specify any changes to the configuration parameters.