aws-controllers-k8s
RDS DBInstance reconcile on `masterUserPassword` changes
Good day Sirs,
I tried to create a RDS instance:
apiVersion: rds.services.k8s.aws/v1alpha1
kind: DBInstance
metadata:
name: dbserver-5-dev
spec:
dbSnapshotIdentifier: "****"
allocatedStorage: 20
dbInstanceClass: db.r5.large
dbInstanceIdentifier: dbserver-5-dev
engine: mysql
engineVersion: "5.7"
masterUsername: "root"
masterUserPassword:
namespace: infra-dev
name: dbserver-5-db-password
key: password
dbSubnetGroupRef:
from:
name: rds-subnet
publiclyAccessible: false
It was created. Then I tried to add a security group, but I see that it was not added to the instance. I see that the instance is stuck:
conditions:
- lastTransitionTime: '2024-05-23T10:41:28Z'
status: 'True'
type: ACK.ReferencesResolved
- message: "InvalidParameterValue: The parameter MasterUserPassword is not a valid password because it is longer than 41 characters.\n\tstatus code: 400, request id: 819086ef-4462-4359-8aa9-f050b3ac3fb6"
status: 'True'
type: ACK.Terminal
- lastTransitionTime: '2024-05-23T10:41:29Z'
message: Resource not synced
reason: resource is in terminal condition
status: 'False'
type: ACK.ResourceSynced
I recreated the secret and checked that it is precisely 41 symbol. Unfortunately, no changes happened.
I see errors like
{"level":"error","ts":"2024-05-23T10:41:26.972Z","msg":"Reconciler error","controller":"dbinstance","controllerGroup":"rds.services.k8s.aws","controllerKind":"DBInstance","DBInstance":{"name":"dbserver-5-dev","namespace":"infra-dev"},"namespace":"infra-dev","name":"dbserver-5-dev","reconcileID":"d778898e-0331-40e3-a844-6dfd4a2e77a4","error":"the referenced resource is not synced yet. resource:SecurityGroup, namespace:infra-dev, name:limit-rds-to-subnet","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
in the logs of controller.
Finally I got:
{"level":"info","ts":"2024-05-23T10:41:28.837Z","logger":"ackrt","msg":"desired resource state has changed","kind":"DBInstance","namespace":"infra-dev","name":"dbserver-5-dev","account":"178394743802","role":"arn:aws:iam::178394743802:role/ack-ec2-controller","region":"eu-west-2","is_adopted":false,"generation":2,"diff":[{"Path":{"Parts":["Spec","MasterUserPassword"]},"A":"","B":"infra-dev/dbserver-5-db-password.password"},{"Path":{"Parts":["Spec","AutoMinorVersionUpgrade"]},"A":null,"B":true},{"Path":{"Parts":["Spec","BackupRetentionPeriod"]},"A":null,"B":1},{"Path":{"Parts":["Spec","CACertificateIdentifier"]},"A":null,"B":"rds-ca-rsa2048-g1"},{"Path":{"Parts":["Spec","CopyTagsToSnapshot"]},"A":null,"B":false},{"Path":{"Parts":["Spec","DeletionProtection"]},"A":null,"B":false},{"Path":{"Parts":["Spec","LicenseModel"]},"A":null,"B":"general-public-license"},{"Path":{"Parts":["Spec","MonitoringInterval"]},"A":null,"B":0},{"Path":{"Parts":["Spec","MultiAZ"]},"A":null,"B":false},{"Path":{"Parts":["Spec","PerformanceInsightsEnabled"]},"A":null,"B":false},{"Path":{"Parts":["Spec","PreferredBackupWindow"]},"A":null,"B":"05:01-05:31"},{"Path":{"Parts":["Spec","PreferredMaintenanceWindow"]},"A":null,"B":"fri:23:40-sat:00:10"},{"Path":{"Parts":["Spec","StorageEncrypted"]},"A":null,"B":false},{"Path":{"Parts":["Spec","StorageThroughput"]},"A":null,"B":0},{"Path":{"Parts":["Spec","StorageType"]},"A":null,"B":"gp2"},{"Path":{"Parts":["Spec","VPCSecurityGroupIDs"]},"A":["sg-06b842635c85a4c5e"],"B":["sg-0e87e0dd9f6d43f31"]}]}
But again - no changes to infra are made.
@gecube The rds-controller doesn't react on secret data changes. However it should react if the change the secret reference :)
@a-hilaly Hi! Does it documented somewhere? I thought that I will be able to get the rotation of passwords out of box in such a way. Probably you know ESO - https://external-secrets.io/v0.8.5/api/generator/password/
It could be used for many purposes like getting passwords from external systems like Vault and putting them into k8s as a secrets. But I am using it for generating random password. The use case is the next. I am spinning up the brand new environment. I don't like to reuse passwords between envs. So I create new password for DB:
apiVersion: generators.external-secrets.io/v1alpha1
kind: Password
metadata:
name: dbserver-5-db-password
spec:
length: 41
digits: 5
symbols: 5
symbolCharacters: "-"
noUpper: false
allowRepeat: true
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: dbserver-5-db-password
spec:
target:
name: dbserver-5-db-password
dataFrom:
- sourceRef:
generatorRef:
apiVersion: generators.external-secrets.io/v1alpha1
kind: Password
name: dbserver-5-db-password
Then I have a password in a secret named dbserver-5-db-password and I can use it anywhere. But what is more important - it could be refreshed... let's say every week. And I would be happy if ACK could watch the secret and reflect the changes inside of it.
Issues go stale after 180d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 60d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/lifecycle stale
Issues go stale after 180d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 60d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/lifecycle stale
Issues go stale after 180d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 60d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/lifecycle stale