percona
Non-existing statefulsets are being deleted over and over
Report
I noticed that the percona mongodb operator fills my kube API logs with deletes of non-existant stateful sets every couple of seconds. This clutters the logs and spills resources on the API server.
In my case, where I run 2 mongos and an arbiter, I get a delete for sts's:
- clustername-cfg
- clustername-mongos
- clustername-rs0-nv
every 6 seconds. See the below screenshot.
More about the problem
Checking the codepaths, I see that it might be intended behaviour (the delete is done, and a k8serror.IsNotFound(err) is actually in place to ignore this error, see https://github.com/percona/percona-server-mongodb-operator/blob/main/pkg/controller/perconaservermongodb/psmdb_controller.go#L995 for one example).
This seems wrong, I'd assume that after a single attempt, the logic can be stopped to try and delete a non-existing object.
Steps to reproduce
- deploy the example crd, just adjust to have 1 arbiter and 2 mongos.
- make sure to disable the sharding
- observe the kube API logs once everything is deployed
Versions
- Kubernetes - GKE 1.31
- Operator - latest
- Database - latest
Anything else?
No response
@wonko yes, this is intended behavior but I agree that it's suboptimal. I created https://perconadev.atlassian.net/browse/K8SPSMDB-1455 to address this.
