community icon indicating copy to clipboard operation
community copied to clipboard

RDS stale value on `DBInstance` updating `dbParameterGroups` `parameterApplyStatus`

Open dweebo opened this issue 1 year ago • 4 comments

Describe the bug Sometimes our DBInstance instance have a stale value in status.dbParameterGroups.parameterApplyStatus.

I'm not sure but it seems like the rds operator stops reconciling the DBInstance resource once it thinks it is "available". I stop seeing logs for the instances that I see with this issue.

Steps to reproduce Create a DBInstance from snapshot (dbSnapshotIdentifier) and with a custom dbParameterGroupName. Wait for RDS instance to be available and for the db parameter group to be set/applied. Check the DBInstance.status.dbParameterGroups.parameterApplyStatus. 3 out of 10 times in my testing the value was not updated on the DBInstance custom resources, even after waiting for an hour after the RDS instances were fully running.

Expected outcome I expect the RDS operator to keep reconciling until after the db parameter group is applied and the value is set to in-sync.

Environment

  • Kubernetes version 1.25

  • Using EKS (yes/no), if so version? No

  • AWS service targeted (S3, RDS, etc.) RDS

dweebo avatar Nov 28 '23 18:11 dweebo

Hi @dweebo - are you configuring the reconcile frequency of the controller? by default it's 10hours.

a-hilaly avatar Jan 19 '24 20:01 a-hilaly

Hi @dweebo - are you configuring the reconcile frequency of the controller? by default it's 10hours.

No I don't believe we are changing that. I didn't want to unnecessarily be reconciling but I see that could be a workaround.

dweebo avatar Jan 24 '24 18:01 dweebo

Issues go stale after 180d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 60d of inactivity and eventually close. If this issue is safe to close now please do so with /close. Provide feedback via https://github.com/aws-controllers-k8s/community. /lifecycle stale

ack-bot avatar Jul 22 '24 20:07 ack-bot