kubeblocks
kubeblocks copied to clipboard
Published
20 hours ago •
apecloud
apecloud
[BUG] es cluster upgrade KB from 0.8 to 0.9 ops failed: cv spec.componentSpecs[*].componentDefRef [elasticsearch] not found in cd
Describe the bug
- elasticsearch error: Warning PreCheckFailed 16m cluster-controller the referenced ClusterVersion is unavailable: elasticsearch-8.8.2
status:
clusterDefGeneration: 3
message: spec.componentSpecs[*].componentDefRef [elasticsearch] not found in ClusterDefinition.spec.componentDefs[*].name
observedGeneration: 2
phase: Unavailable
- starrocks ce error: Warning PreCheckFailed 9m7s cluster-controller the referenced ClusterVersion is not up to date: starrocks-3.1.1
status:
clusterDefGeneration: 1
message: spec.componentSpecs[*].componentDefRef [fe be] not found in ClusterDefinition.spec.componentDefs[*].name
observedGeneration: 2
phase: Unavailable
To Reproduce Steps to reproduce the behavior:
- install KB 0.8
curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v0.8.4-beta.1
kbcli kubeblocks install --create-namespace --version 0.8.4-beta.11 --set image.registry=docker.io --set dataProtection.image.registry=docker.io --set addonChartsImage.registry=docker.io --set dataProtection.image.datasafed.tag=0.1.0 --namespace kb-tfctfv
- create cluster
helm upgrade --install --namespace kb-tfctfv kb-addon-elasticsearch kubeblocks-addons/elasticsearch --version 0.8.0
apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
name: esearch-tfctfv
namespace: ns-tfctfv
annotations:
kubeblocks.io/extra-env: '***"elasticsearch-roles":"master,data,ingest,transform"***'
spec:
clusterDefinitionRef: elasticsearch
clusterVersionRef: elasticsearch-8.8.2
terminationPolicy: Halt
componentSpecs:
- name: elasticsearch
componentDefRef: elasticsearch
componentDef: elasticsearch
replicas: 1
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: 500m
memory: 1Gi
volumeClaimTemplates:
- name: data
spec:
storageClassName:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
- upgrade kb to 0.9
curl -fsSL https://kubeblocks.io/installer/install_cli.sh | bash -s v0.9.0-beta.5
kbcli kubeblocks upgrade --auto-approve --set upgradeAddons=true --version 0.9.0-beta.48 --set image.registry=docker.io --set dataProtection.image.registry=docker.io --set addonChartsImage.registry=docker.io --set dataProtection.image.datasafed.tag=0.2.0 --namespace kb-tfctfv
helm upgrade --install --namespace kb-tfctfv kb-addon-elasticsearch kubeblocks-addons/elasticsearch --version 0.9.0
- stop cluster
kbcli cluster stop esearch-tfctfv --auto-approve --force=true --namespace ns-tfctfv
- See error ops is always Running describe ops
kubectl describe opsrequest esearch-tfctfv-stop-x6ws2 --namespace ns-tfctfv
Name: esearch-tfctfv-stop-x6ws2
Namespace: ns-tfctfv
Labels: app.kubernetes.io/instance=esearch-tfctfv
app.kubernetes.io/managed-by=kubeblocks
ops.kubeblocks.io/ops-type=Stop
Annotations: <none>
API Version: apps.kubeblocks.io/v1alpha1
Kind: OpsRequest
Metadata:
Creation Timestamp: 2024-07-03T15:54:05Z
Finalizers:
opsrequest.kubeblocks.io/finalizer
Generate Name: esearch-tfctfv-stop-
Generation: 2
Owner References:
API Version: apps.kubeblocks.io/v1alpha1
Kind: Cluster
Name: esearch-tfctfv
UID: 9630cf3c-1c77-40b9-93f5-3175d157f5f9
Resource Version: 170767
UID: 29a7496d-b54c-4340-a259-c85e073a286b
Spec:
Cluster Name: esearch-tfctfv
Force: true
Pre Condition Deadline Seconds: 0
Type: Stop
Status:
Cluster Generation: 9
Components:
Elasticsearch:
Phase: Running
Progress Details:
Group: elasticsearch/Delete
Message: wait to delete pod: Pod/esearch-tfctfv-elasticsearch-0 in Component: elasticsearch
Object Key: Pod/esearch-tfctfv-elasticsearch-0
Status: Pending
Group: elasticsearch/Delete
Message: wait to delete pod: Pod/esearch-tfctfv-elasticsearch-1 in Component: elasticsearch
Object Key: Pod/esearch-tfctfv-elasticsearch-1
Status: Pending
Group: elasticsearch/Delete
Message: wait to delete pod: Pod/esearch-tfctfv-elasticsearch-2 in Component: elasticsearch
Object Key: Pod/esearch-tfctfv-elasticsearch-2
Status: Pending
Conditions:
Last Transition Time: 2024-07-03T15:54:07Z
Message: wait for the controller to process the OpsRequest: esearch-tfctfv-stop-x6ws2 in Cluster: esearch-tfctfv
Reason: WaitForProgressing
Status: True
Type: WaitForProgressing
Last Transition Time: 2024-07-03T15:54:09Z
Message: OpsRequest: esearch-tfctfv-stop-x6ws2 is validated
Reason: ValidateOpsRequestPassed
Status: True
Type: Validated
Last Transition Time: 2024-07-03T15:54:09Z
Message: Start to stop the Cluster: esearch-tfctfv
Reason: StopStarted
Status: True
Type: Stopping
Last Configuration:
Components:
Elasticsearch:
Replicas: 3
Phase: Running
Progress: 0/3
Start Timestamp: 2024-07-03T15:54:09Z
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal WaitForProgressing 16m ops-request-controller wait for the controller to process the OpsRequest: esearch-tfctfv-stop-x6ws2 in Cluster: esearch-tfctfv
Normal ValidateOpsRequestPassed 16m ops-request-controller OpsRequest: esearch-tfctfv-stop-x6ws2 is validated
Normal StopStarted 16m ops-request-controller Start to stop the Cluster: esearch-tfctfv
describe cluster error: the referenced ClusterVersion is unavailable: elasticsearch-8.8.2
kubectl describe cluster esearch-tfctfv --namespace ns-tfctfv
Name: esearch-tfctfv
Namespace: ns-tfctfv
Labels: app.kubernetes.io/instance=esearch-tfctfv
clusterdefinition.kubeblocks.io/name=elasticsearch
clusterversion.kubeblocks.io/name=elasticsearch-8.8.2
Annotations: kubeblocks.io/extra-env: ***"elasticsearch-roles":"master,data,ingest,transform"***
kubeblocks.io/ops-request: [***"name":"esearch-tfctfv-stop-x6ws2","type":"Stop"***]
kubeblocks.io/reconcile: 2024-07-03T14:33:56.222015238Z
kubeblocks.io/snapshot-for-start: ***"elasticsearch":3***
API Version: apps.kubeblocks.io/v1alpha1
Kind: Cluster
Metadata:
Creation Timestamp: 2024-07-03T14:20:27Z
Finalizers:
cluster.kubeblocks.io/finalizer
Generation: 9
Resource Version: 170638
UID: 9630cf3c-1c77-40b9-93f5-3175d157f5f9
Spec:
Cluster Definition Ref: elasticsearch
Cluster Version Ref: elasticsearch-8.8.2
Component Specs:
Class Def Ref:
Class:
Component Def: elasticsearch
Component Def Ref: elasticsearch
Monitor: false
Name: elasticsearch
Replicas: 0
Resources:
Limits:
Cpu: 600m
Memory: 1536Mi
Requests:
Cpu: 600m
Memory: 1536Mi
Service Version: 8.8.2
Volume Claim Templates:
Name: data
Spec:
Access Modes:
ReadWriteOnce
Resources:
Requests:
Storage: 20Gi
Resources:
Cpu: 0
Memory: 0
Storage:
Size: 0
Termination Policy: WipeOut
Status:
Cluster Def Generation: 2
Components:
Elasticsearch:
Phase: Running
Pods Ready: true
Pods Ready Time: 2024-07-03T14:39:48Z
Conditions:
Last Transition Time: 2024-07-03T15:54:08Z
Message: the referenced ClusterVersion is unavailable: elasticsearch-8.8.2
Reason: PreCheckFailed
Status: False
Type: ProvisioningStarted
Last Transition Time: 2024-07-03T14:20:30Z
Message: Successfully applied for resources
Observed Generation: 8
Reason: ApplyResourcesSucceed
Status: True
Type: ApplyResources
Last Transition Time: 2024-07-03T14:39:48Z
Message: all pods of components are ready, waiting for the probe detection successful
Reason: AllReplicasReady
Status: True
Type: ReplicasReady
Last Transition Time: 2024-07-03T14:39:48Z
Message: Cluster: esearch-tfctfv is ready, current phase is Running
Reason: ClusterReady
Status: True
Type: Ready
Observed Generation: 8
Phase: Running
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal PreCheckSucceed 34m cluster-controller The operator has started the provisioning of Cluster: esearch-tfctfv
Normal ApplyResourcesSucceed 34m cluster-controller Successfully applied for resources
Warning PreCheckFailed 16m cluster-controller the referenced ClusterVersion is unavailable: elasticsearch-8.8.2
get ec cv yaml
kubectl get clusterVersions elasticsearch-8.8.2 -o yaml --namespace kb-tfctfv `
Warning: The ClusterVersion CRD has been deprecated since 0.9.0
apiVersion: apps.kubeblocks.io/v1alpha1
kind: ClusterVersion
metadata:
annotations:
meta.helm.sh/release-name: kb-addon-elasticsearch
meta.helm.sh/release-namespace: kb-tfctfv
creationTimestamp: "2024-07-03T14:20:00Z"
finalizers:
- clusterversion.kubeblocks.io/finalizer
generation: 2
labels:
app.kubernetes.io/instance: kb-addon-elasticsearch
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: elasticsearch
app.kubernetes.io/version: 8.8.2
clusterdefinition.kubeblocks.io/name: elasticsearch
helm.sh/chart: elasticsearch-0.9.0
name: elasticsearch-8.8.2
resourceVersion: "158466"
uid: 3a4a728e-d2f9-46e9-b95a-d8e9ecf1844a
spec:
clusterDefinitionRef: elasticsearch
componentVersions:
- componentDefRef: elasticsearch
versionsContext:
containers:
- image: docker.io/elasticsearch:8.8.2
imagePullPolicy: IfNotPresent
name: elasticsearch
resources: ***
- image: docker.io/elasticsearch:8.8.2
imagePullPolicy: IfNotPresent
name: elasticsearch-master-graceful-termination-handler
resources: ***
- image: docker.io/bitnami/elasticsearch-exporter:1.5.0-debian-11-r102
imagePullPolicy: IfNotPresent
name: metrics
resources: ***
initContainers:
- image: docker.io/busybox:1.35
imagePullPolicy: IfNotPresent
name: sysctl
resources: ***
- image: docker.io/apecloud/elasticsearch-plugins:8.8.2
imagePullPolicy: IfNotPresent
name: prepare-plugins
resources: ***
status:
clusterDefGeneration: 3
message: spec.componentSpecs[*].componentDefRef [elasticsearch] not found in ClusterDefinition.spec.componentDefs[*].name
observedGeneration: 2
phase: Unavailable
get sr cv yaml
kubectl get clusterVersions starrocks-3.1.1 -o yaml --namespace kb-tfctfv
Warning: The ClusterVersion CRD has been deprecated since 0.9.0
apiVersion: apps.kubeblocks.io/v1alpha1
kind: ClusterVersion
metadata:
annotations:
meta.helm.sh/release-name: kb-addon-starrocks
meta.helm.sh/release-namespace: kb-tfctfv
creationTimestamp: "2024-07-03T14:19:38Z"
deletionGracePeriodSeconds: 0
deletionTimestamp: "2024-07-03T15:42:16Z"
finalizers:
- clusterversion.kubeblocks.io/finalizer
generation: 3
labels:
app.kubernetes.io/instance: kb-addon-starrocks
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: starrocks
app.kubernetes.io/version: 3.1.1
clusterdefinition.kubeblocks.io/name: starrocks
helm.sh/chart: starrocks-0.8.0
name: starrocks-3.1.1
resourceVersion: "160115"
uid: f18bc3d1-f225-4da5-bcd2-8f56c4a12629
spec:
clusterDefinitionRef: starrocks
componentVersions:
- componentDefRef: fe
versionsContext:
containers:
- image: docker.io/starrocks/fe-ubuntu:2.5.4
imagePullPolicy: IfNotPresent
name: fe
resources: ***
- componentDefRef: be
versionsContext:
containers:
- image: docker.io/starrocks/be-ubuntu:2.5.4
imagePullPolicy: IfNotPresent
name: be
resources: ***
status:
clusterDefGeneration: 1
message: spec.componentSpecs[*].componentDefRef [fe be] not found in ClusterDefinition.spec.componentDefs[*].name
observedGeneration: 2
phase: Unavailable
Expected behavior A clear and concise description of what you expected to happen.
Screenshots If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
Additional context Add any other context about the problem here.
