[rds-controller] can't modify DBParameterGroup after failure

[gecube]

1. create a DBParameterGroup with incorrect parameters.

Then got:

status:
  ackResourceMetadata:
    arn: 'arn:aws:rds:eu-west-2:474417630776:pg:****-1'
    ownerAccountID: '474417630776'
    region: eu-west-2
  conditions:
    - message: 'unknown parameter: character_set_system'
      status: 'True'
      type: ACK.Terminal
    - lastTransitionTime: '2025-05-16T14:00:44Z'
      message: Resource not synced
      reason: resource is in terminal condition
      status: 'False'
      type: ACK.ResourceSynced

2. Change the DBParameterGroup. Find that there are no changes.

[github-actions[bot]]

Hello @gecube 👋 Thank you for opening an issue in ACK! A maintainer will triage this issue soon.

We encourage community contributions, so if you're interested in tackling this yourself or suggesting a solution, please check out our Contribution and Code of Conduct guidelines.

You can find more information about ACK on our website.

[gecube]

Anyway, I checked that if I delete it from the cluster - it immediately is deleted from the cloud (so the controller is working).

[gecube]

after removal and recreation of the resource - everything is fine:

status:
  ackResourceMetadata:
    arn: 'arn:aws:rds:eu-west-2:474417630776:pg:****'
    ownerAccountID: '474417630776'
    region: eu-west-2
  conditions:
    - lastTransitionTime: '2025-05-16T14:11:40Z'
      message: Resource synced successfully
      reason: ''
      status: 'True'
      type: ACK.ResourceSynced
  parameterOverrideStatuses:
    - applyMethod: immediate
      applyType: dynamic
      parameterName: character_set_client
      parameterValue: utf8
    - applyMethod: immediate
      applyType: dynamic
      parameterName: character_set_connection
      parameterValue: utf8
    - applyMethod: immediate
      applyType: dynamic
      parameterName: character_set_database
      parameterValue: utf8
    - applyMethod: immediate
      applyType: dynamic
      parameterName: character_set_results
      parameterValue: utf8
    - applyMethod: immediate
      applyType: dynamic
      parameterName: character_set_server
      parameterValue: utf8
    - applyMethod: immediate
      applyType: dynamic
      parameterName: collation_connection
      parameterValue: utf8_general_ci
    - applyMethod: immediate
      applyType: dynamic
      parameterName: collation_server
      parameterValue: utf8_general_ci
    - applyMethod: immediate
      applyType: dynamic
      parameterName: log_bin_trust_function_creators
      parameterValue: '1'

[rushmash91]

Hi @gecube , Thank you for opening the issue. Can you share the DBParameterGroup manifest you are using as well? Did you have to change a value from original to not encounter the Terminal error?

[gecube]

@rushmash91 the current is:

apiVersion: rds.services.k8s.aws/v1alpha1
kind: DBParameterGroup
metadata:
  name: mysql57-triggers
spec:
  description: "MySQL with triggers"
  family: "mysql5.7"
  name: *****
  parameterOverrides:
    log_bin_trust_function_creators: "1"
    collation_connection: utf8_general_ci
    collation_server: utf8_general_ci
    character_set_connection: utf8
    character_set_database: utf8
    character_set_results: utf8
    character_set_client: utf8
    character_set_server: utf8
  tags:
    - key: env
      value: uat

the previous one was something like:

apiVersion: rds.services.k8s.aws/v1alpha1
kind: DBParameterGroup
metadata:
  name: mysql57-triggers
spec:
  description: "MySQL with triggers"
  family: "mysql5.7"
  name: *****
  parameterOverrides:
    log_bin_trust_function_creators: "1"
    collation_connection: utf8_general_ci
    collation_server: utf8_general_ci
    collation_database: utf8_general_ci
    character_set_connection: utf8
    character_set_database: utf8
    character_set_results: utf8
    character_set_client: utf8
    character_set_server: utf8
    character_set_system: utf8
  tags:
    - key: env
      value: uat

and the very first one was:

apiVersion: rds.services.k8s.aws/v1alpha1
kind: DBParameterGroup
metadata:
  name: mysql57-triggers
spec:
  description: "MySQL with triggers"
  family: "mysql5.7"
  name: *****
  parameterOverrides:
    log_bin_trust_function_creators: "1"
    collation_connection: utf8_general_ci
  tags:
    - key: env
      value: uat

[rushmash91]

character_set_system: utf8 is a internal server variable. Its reflects the server’s compiled-in defaults and can not be overridden. Hence, the unknown parameter error.

https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_character_set_system

[tomreece]

character_set_system: utf8 is a internal server variable. Its reflects the server’s compiled-in defaults and can not be overridden. Hence, the unknown parameter error.

https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_character_set_system

Sure, but that doesn't explain why the resource fails to sync after the issue is corrected.

I am experiencing the same issue. If I add an incorrect parameterOverride that results in an error, then I fix that by removing it from the spec, I would expect it to sync the DBParameterGroup successfully now that the bad parameter is removed. But it's stuck in a bad state complaining about a value that doesn't exist in the spec any more. The only way I've been able to fix it is to manually delete the parameter group in the AWS UI and then let ACK recreate it. But I don't want to have to do that.

Start with:

apiVersion: rds.services.k8s.aws/v1alpha1
kind: DBParameterGroup
metadata:
  name: my-db-pg
spec:
  name: my-db-pg
  description: My db parameter group.
  family: aurora-mysql8.0
  parameterOverrides:
    innodb_sort_buffer_size: "99999999999999"

Get sync error:

"InvalidParameterValue: Value: 99999999999999 is outside of range: 65536-67108864
        for parameter: innodb_sort_buffer_size\n\tstatus code: 400"

Fix issue:

apiVersion: rds.services.k8s.aws/v1alpha1
kind: DBParameterGroup
metadata:
  name: my-db-pg
spec:
  name: my-db-pg
  description: My db parameter group.
  family: aurora-mysql8.0
  parameterOverrides:
    innodb_sort_buffer_size: "67108864"

Expected behavior - DBParameterGroup should sync successfully Actual behavior - DBParameterGroup never syncs. Same error is shown even though spec has changed.