kubeblocks
kubeblocks copied to clipboard
apecloud
[BUG] cluster delete error configmaps mongo-jucowe-mongodb-mongodb-config not found
Describe the bug cluster status is always deleting：error configmaps mongo-jucowe-mongodb-mongodb-config not found
kbcli version
Kubernetes: v1.26.6-gke.1700
KubeBlocks: 0.8.2-beta.1
kbcli: 0.8.1
ref： https://github.com/apecloud/kubeblocks/actions/runs/7694417877/job/20966398748
To Reproduce Steps to reproduce the behavior:
- create cluster
- delete cluster
- See error
kubectl get all -l app.kubernetes.io/instance=mongo-jucowe
NAME COMPLETIONS DURATION AGE
job.batch/dp-backup-0-mongo-jucowe-20240129162701-fcc14002 0/1 10h 10h
NAME TYPE CLUSTER STATUS PROGRESS AGE
opsrequest.apps.kubeblocks.io/backup-default-mongo-jucowe-20240130000259 Backup mongo-jucowe Failed -/- 11h
NAME CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS AGE
cluster.apps.kubeblocks.io/mongo-jucowe mongodb mongodb-6.0 WipeOut Deleting 11h
describe cluster
kubectl describe cluster mongo-jucowe
Name: mongo-jucowe
Namespace: default
Labels: app.kubernetes.io/instance=mongo-jucowe
clusterdefinition.kubeblocks.io/name=mongodb
clusterversion.kubeblocks.io/name=mongodb-6.0
Annotations: <none>
API Version: apps.kubeblocks.io/v1alpha1
Kind: Cluster
Metadata:
Creation Timestamp: 2024-01-29T15:20:45Z
Deletion Grace Period Seconds: 0
Deletion Timestamp: 2024-01-30T02:37:56Z
Finalizers:
cluster.kubeblocks.io/finalizer
Generation: 9
Spec:
Affinity:
Pod Anti Affinity: Preferred
Tenancy: SharedNode
Cluster Definition Ref: mongodb
Cluster Version Ref: mongodb-6.0
Component Specs:
Class Def Ref:
Class:
Component Def Ref: mongodb
Enabled Logs:
running
Monitor: true
Name: mongodb
No Create PDB: false
Replicas: 2
Resources:
Limits:
Cpu: 200m
Memory: 644245094400m
Requests:
Cpu: 200m
Memory: 644245094400m
Rsm Transform Policy: ToSts
Service Account Name: kb-mongo-jucowe
Volume Claim Templates:
Name: data
Spec:
Access Modes:
ReadWriteOnce
Resources:
Requests:
Storage: 5Gi
Monitor:
Resources:
Cpu: 0
Memory: 0
Storage:
Size: 0
Termination Policy: WipeOut
Status:
Cluster Def Generation: 3
Components:
Mongodb:
Phase: Running
Pods Ready: true
Pods Ready Time: 2024-01-29T16:31:09Z
Conditions:
Last Transition Time: 2024-01-29T15:20:45Z
Message: The operator has started the provisioning of Cluster: mongo-jucowe
Observed Generation: 8
Reason: PreCheckSucceed
Status: True
Type: ProvisioningStarted
Last Transition Time: 2024-01-30T02:37:57Z
Message: Operation cannot be fulfilled on replicatedstatemachines.workloads.kubeblocks.io "mongo-jucowe-mongodb": the object has been modified; please apply your changes to the latest version and try again
Reason: ApplyResourcesFailed
Status: False
Type: ApplyResources
Last Transition Time: 2024-01-29T16:31:09Z
Message: all pods of components are ready, waiting for the probe detection successful
Reason: AllReplicasReady
Status: True
Type: ReplicasReady
Last Transition Time: 2024-01-29T16:31:09Z
Message: Cluster: mongo-jucowe is ready, current phase is Running
Reason: ClusterReady
Status: True
Type: Ready
Observed Generation: 8
Phase: Deleting
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ApplyResourcesSucceed 41m (x10 over 11h) cluster-controller Successfully applied for resources
Normal DeletingCR 4m21s (x1381 over 29m) cluster-controller Deleting : mongo-jucowe
logs kubeblocks
2024-01-30T02:38:14.793Z ERROR Reconciler error {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "Configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}, "namespace": "default", "name": "mongo-jucowe-mongodb", "reconcileID": "85e13bad-2aa4-44a1-8709-fd2c02a3e82e", "error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found", "errorCauses": [{"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}, {"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}]}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:265
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:226
2024-01-30T02:38:35.298Z INFO ConfigurationReconcile failed to run configuration reconcile task. {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "Configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}, "namespace": "default", "name": "mongo-jucowe-mongodb", "reconcileID": "30c542e6-5d45-4624-8bd2-42a463847109", "configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}}
2024-01-30T02:38:35.298Z ERROR Reconciler error {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "Configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}, "namespace": "default", "name": "mongo-jucowe-mongodb", "reconcileID": "30c542e6-5d45-4624-8bd2-42a463847109", "error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found", "errorCauses": [{"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}, {"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}]}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:265
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:226
2024-01-30T02:39:16.288Z INFO ConfigurationReconcile failed to run configuration reconcile task. {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "Configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}, "namespace": "default", "name": "mongo-jucowe-mongodb", "reconcileID": "3d6761ba-83e3-4ecc-b1e5-192683334081", "configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}}
2024-01-30T02:39:16.288Z ERROR Reconciler error {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "Configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}, "namespace": "default", "name": "mongo-jucowe-mongodb", "reconcileID": "3d6761ba-83e3-4ecc-b1e5-192683334081", "error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found", "errorCauses": [{"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}, {"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}]}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:265
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:226
2024-01-30T02:40:38.232Z INFO ConfigurationReconcile failed to run configuration reconcile task. {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "Configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}, "namespace": "default", "name": "mongo-jucowe-mongodb", "reconcileID": "74b08f55-4370-4bec-aa41-ac824a3b64ff", "configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}}
2024-01-30T02:40:38.233Z ERROR Reconciler error {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "Configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}, "namespace": "default", "name": "mongo-jucowe-mongodb", "reconcileID": "74b08f55-4370-4bec-aa41-ac824a3b64ff", "error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found", "errorCauses": [{"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}, {"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}]}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:265
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:226
2024-01-30T02:43:22.097Z INFO ConfigurationReconcile failed to run configuration reconcile task. {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "Configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}, "namespace": "default", "name": "mongo-jucowe-mongodb", "reconcileID": "cdb5e1f5-33de-4bf6-afc7-e98b0c274722", "configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}}
2024-01-30T02:43:22.097Z ERROR Reconciler error {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "Configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}, "namespace": "default", "name": "mongo-jucowe-mongodb", "reconcileID": "cdb5e1f5-33de-4bf6-afc7-e98b0c274722", "error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found", "errorCauses": [{"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}, {"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}]}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:265
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:226
2024-01-30T02:48:49.806Z INFO ConfigurationReconcile failed to run configuration reconcile task. {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "Configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}, "namespace": "default", "name": "mongo-jucowe-mongodb", "reconcileID": "18e90c1b-9450-47b1-8390-8dbb9f1d710b", "configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}}
2024-01-30T02:48:49.806Z ERROR Reconciler error {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "Configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}, "namespace": "default", "name": "mongo-jucowe-mongodb", "reconcileID": "18e90c1b-9450-47b1-8390-8dbb9f1d710b", "error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found", "errorCauses": [{"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}, {"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}]}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:265
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:226
2024-01-30T02:59:45.195Z INFO ConfigurationReconcile failed to run configuration reconcile task. {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "Configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}, "namespace": "default", "name": "mongo-jucowe-mongodb", "reconcileID": "c3f66466-80cf-46af-a951-b4473466bd0c", "configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}}
2024-01-30T02:59:45.195Z ERROR Reconciler error {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "Configuration": {"name":"mongo-jucowe-mongodb","namespace":"default"}, "namespace": "default", "name": "mongo-jucowe-mongodb", "reconcileID": "c3f66466-80cf-46af-a951-b4473466bd0c", "error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found", "errorCauses": [{"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}, {"error": "configmaps \"mongo-jucowe-mongodb-mongodb-config\" not found"}]}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:265
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:226
Expected behavior A clear and concise description of what you expected to happen.
Screenshots If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
Additional context Add any other context about the problem here.
This issue has been marked as stale because it has been open for 30 days with no activity
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}