kubeblocks
kubeblocks copied to clipboard
apecloud
[BUG] create milvus cluster-with-dep KubeBlocks pod crash
Describe the bug A clear and concise description of what the bug is.
kbcli version
Kubernetes: v1.30.4-vke.4
KubeBlocks: 0.9.6-beta.3
kbcli: 0.9.5
helm get notes -n kb-system kb-addon-milvus
NOTES:
1. Get the application URL by running these commands:
Release Information:
Commit ID: "94e493b6a14a05e4a1a162bbdfeba3415d11d808"
Commit Time: "2025-09-03 10:54:56 +0800"
Release Branch: "v0.9.6-beta.3"
Release Time: "2025-11-10 16:06:45 +0800"
Enterprise: "false"
Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "Component": {"name":"milvus-upsyzf-datanode","namespace":"default"}, "namespace": "default", "name": "milvus-upsyzf-datanode", "reconcileID": "19c401bb-20e5-44a0-ba9d-70d1aaf9fcd1"}
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x130 pc=0x1bdde0c]
goroutine 950 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:116 +0x1e5
panic({0x28966a0?, 0x4ba1b50?})
/usr/local/go/src/runtime/panic.go:914 +0x21f
github.com/apecloud/kubeblocks/pkg/controller/component.resolveServiceRefHostRef.func1({0x2c8b020?, 0x0?})
/src/pkg/controller/component/vars.go:945 +0x2c
github.com/apecloud/kubeblocks/pkg/controller/component.resolveClusterObjectVars({0x2d06b71, 0xa}, {{0x0, 0x0}, {0xc001913920, 0x13}, 0xc002e074d4, 0x0}, 0xc0024ce760, 0xc002c6ed18, ...)
/src/pkg/controller/component/vars.go:1343 +0x478
github.com/apecloud/kubeblocks/pkg/controller/component.resolveServiceRefVarRefLow({0x1?, 0xc23cdfac244aa55e?}, {0xc0024d9c20?, 0x1?}, 0x0?, {{{0x0, 0x0}, {0xc001913920, 0x13}, 0xc002e074d4, ...}, ...}, ...)
/src/pkg/controller/component/vars.go:1147 +0xd2
github.com/apecloud/kubeblocks/pkg/controller/component.resolveServiceRefHostRef({0x33564e0?, 0xc002c4b020?}, {0x333a7d0?, 0xc003cf3a70?}, 0xc00024bea0?, {0xc001913908?, 0x0?}, {{{0x0, 0x0}, {0xc001913920, ...}, ...}, ...})
/src/pkg/controller/component/vars.go:953 +0xc6
github.com/apecloud/kubeblocks/pkg/controller/component.resolveServiceRefVarRef({0x33564e0?, 0xc002c4b020?}, {0x333a7d0?, 0xc003cf3a70?}, 0xc002c6f0b8?, {0xc001913908?, 0x3339ab0?}, {{{0x0, 0x0}, {0xc001913920, ...}, ...}, ...})
/src/pkg/controller/component/vars.go:923 +0xfc
github.com/apecloud/kubeblocks/pkg/controller/component.resolveClusterObjectVarRef({0x33564e0?, 0xc002c4b020?}, {0x333a7d0?, 0xc003cf3a70?}, 0xc003d8a1e0?, {0xc001913908?, 0xc003c9d0e0?}, {0x0, 0x0, 0x0, ...}, ...)
/src/pkg/controller/component/vars.go:497 +0x1be
github.com/apecloud/kubeblocks/pkg/controller/component.resolveClusterObjectRefVars({0x33564e0, 0xc002c4b020}, {0x333a7d0, 0xc003cf3a70}, 0xc001944e00, {0xc000f0d200?, 0xc, 0x0?})
/src/pkg/controller/component/vars.go:465 +0x5ac
github.com/apecloud/kubeblocks/pkg/controller/component.resolveBuiltinNObjectRefVars({0x33564e0, 0xc002c4b020}, {0x333a7d0, 0xc003cf3a70}, 0x0?, {0xc000f0d200, 0xc, 0xc})
/src/pkg/controller/component/vars.go:159 +0xc5
github.com/apecloud/kubeblocks/pkg/controller/component.resolveNewTemplateNEnvVars({0x33564e0?, 0xc002c4b020?}, {0x333a7d0?, 0xc003cf3a70?}, 0xc0006a6be0?, {0xc000f0d200?, 0xc002c6f668?, 0x410645?})
/src/pkg/controller/component/vars.go:136 +0x47
github.com/apecloud/kubeblocks/pkg/controller/component.resolveTemplateNEnvVars({0x33564e0?, 0xc002c4b020?}, {0x333a7d0?, 0xc003cf3a70?}, 0x0?, {0xc000f0d200?, 0x0?, 0x0?}, 0x0)
/src/pkg/controller/component/vars.go:107 +0x57
github.com/apecloud/kubeblocks/pkg/controller/component.ResolveTemplateNEnvVars(...)
/src/pkg/controller/component/vars.go:63
github.com/apecloud/kubeblocks/controllers/apps.(*componentVarsTransformer).Transform(0xc0024cf6c0?, {0x3356070?, 0xc000b821b0}, 0xc0024cf700)
/src/controllers/apps/transformer_component_vars.go:74 +0x32e
github.com/apecloud/kubeblocks/pkg/controller/graph.TransformerChain.ApplyTo({0xc000923e40?, 0x16, 0xc002c5a650?}, {0x3356070, 0xc000b821b0}, 0x417852?)
/src/pkg/controller/graph/transformer.go:57 +0x9b
github.com/apecloud/kubeblocks/controllers/apps.(*componentPlanBuilder).Build(0xc000a56410)
/src/controllers/apps/component_plan_builder.go:117 +0xb6
github.com/apecloud/kubeblocks/controllers/apps.(*ComponentReconciler).Reconcile(0xc000723290, {0x33564e0, 0xc002c4ad20}, {{{0xc000c33440, 0x7}, {0xc000ba0858, 0x16}}})
/src/controllers/apps/component_controller.go:188 +0xa0e
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x335c5e8?, {0x33564e0?, 0xc002c4ad20?}, {{{0xc000c33440?, 0xb?}, {0xc000ba0858?, 0x0?}}})
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119 +0xb7
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0003e92c0, {0x3356518, 0xc000256690}, {0x29d5f40?, 0xc00033a0a0?})
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316 +0x3cc
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0003e92c0, {0x3356518, 0xc000256690})
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266 +0x1af
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227 +0x79
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 118
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:223 +0x565
To Reproduce Steps to reproduce the behavior:
- create monio cluster
apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
name: miniom-upsyzf
namespace: default
spec:
terminationPolicy: WipeOut
componentSpecs:
- componentDef: minio
name: minio
replicas: 2
env:
- name: MINIO_BUCKETS
value: kbcli-test
name: minio
resources:
limits:
cpu: 100m
memory: 0.5Gi
requests:
cpu: 100m
memory: 0.5Gi
volumeClaimTemplates:
- name: data
spec:
storageClassName:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
- create milvus cluster-with-dep
apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
name: milvus-upsyzf
namespace: default
spec:
clusterDefinitionRef: milvus
topology: cluster-with-dep
terminationPolicy: WipeOut
componentSpecs:
- name: etcd
serviceVersion: 3.6.1
replicas: 1
disableExporter: true
resources:
limits:
cpu: 500m
memory: 0.5Gi
requests:
cpu: 500m
memory: 0.5Gi
volumeClaimTemplates:
- name: data
spec:
storageClassName:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
- name: kafka
serviceVersion: 3.3.2
replicas: 1
disableExporter: true
resources:
limits:
cpu: 500m
memory: 0.5Gi
requests:
cpu: 500m
memory: 0.5Gi
volumeClaimTemplates:
- name: data
spec:
storageClassName:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
- name: proxy
serviceVersion: 2.3.2
replicas: 1
disableExporter: true
resources:
limits:
cpu: 500m
memory: 0.5Gi
requests:
cpu: 500m
memory: 0.5Gi
env:
- name: MINIO_BUCKET
value: kbcli-test
- name: MINIO_ROOT_PATH
value:
- name: MINIO_USE_PATH_STYLE
value: "true"
serviceRefs:
- name: milvus-object-storage
namespace: default
clusterServiceSelector:
cluster: miniom-upsyzf
service:
component: minio
service: default
port: api
credential:
component: minio
name: root
- name: mixcoord
serviceVersion: 2.3.2
replicas: 1
disableExporter: true
resources:
limits:
cpu: 500m
memory: 0.5Gi
requests:
cpu: 500m
memory: 0.5Gi
env:
- name: MINIO_BUCKET
value: kbcli-test
- name: MINIO_ROOT_PATH
value:
- name: MINIO_USE_PATH_STYLE
value: "true"
serviceRefs:
- name: milvus-object-storage
namespace: default
clusterServiceSelector:
cluster: miniom-upsyzf
service:
component: minio
service: default
port: api
credential:
component: minio
name: root
- name: datanode
serviceVersion: 2.3.2
replicas: 1
disableExporter: true
resources:
limits:
cpu: 500m
memory: 0.5Gi
requests:
cpu: 500m
memory: 0.5Gi
env:
- name: MINIO_BUCKET
value: kbcli-test
- name: MINIO_ROOT_PATH
value:
- name: MINIO_USE_PATH_STYLE
value: "true"
serviceRefs:
- name: milvus-object-storage
namespace: default
clusterServiceSelector:
cluster: miniom-upsyzf
service:
component: minio
service: default
port: api
credential:
component: minio
name: root
- name: indexnode
serviceVersion: 2.3.2
replicas: 1
disableExporter: true
resources:
limits:
cpu: 500m
memory: 0.5Gi
requests:
cpu: 500m
memory: 0.5Gi
env:
- name: MINIO_BUCKET
value: kbcli-test
- name: MINIO_ROOT_PATH
value:
- name: MINIO_USE_PATH_STYLE
value: "true"
serviceRefs:
- name: milvus-object-storage
namespace: default
clusterServiceSelector:
cluster: miniom-upsyzf
service:
component: minio
service: default
port: api
credential:
component: minio
name: root
- name: querynode
serviceVersion: 2.3.2
replicas: 1
disableExporter: true
resources:
limits:
cpu: 500m
memory: 0.5Gi
requests:
cpu: 500m
memory: 0.5Gi
env:
- name: MINIO_BUCKET
value: kbcli-test
- name: MINIO_ROOT_PATH
value:
- name: MINIO_USE_PATH_STYLE
value: "true"
serviceRefs:
- name: milvus-object-storage
namespace: default
clusterServiceSelector:
cluster: miniom-upsyzf
service:
component: minio
service: default
port: api
credential:
component: minio
name: root
- See error
kubectl get cluster
NAME CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS AGE
milvus-upsyzf milvus WipeOut Updating 61s
miniom-upsyzf WipeOut Running 3m48s
kubectl get pod
NAME READY STATUS RESTARTS AGE
milvus-upsyzf-etcd-0 2/2 Running 0 77s
milvus-upsyzf-kafka-0 2/2 Running 0 77s
miniom-upsyzf-minio-0 2/2 Running 0 4m4s
miniom-upsyzf-minio-1 2/2 Running 0 3m35s
kubectl get cmp
NAME DEFINITION SERVICE-VERSION STATUS AGE
milvus-upsyzf-datanode milvus-datanode-0.9.2 2.3.2 25s
milvus-upsyzf-etcd etcd 3.6.1 Running 51s
milvus-upsyzf-indexnode milvus-indexnode-0.9.2 2.3.2 25s
milvus-upsyzf-kafka kafka-combine 3.3.2 Running 51s
milvus-upsyzf-proxy milvus-proxy-0.9.2 2.3.2 24s
milvus-upsyzf-querynode milvus-querynode-0.9.2 2.3.2 24s
miniom-upsyzf-minio minio 2024.6.29 Running 3m38s
describe cmp
kubectl describe cmp milvus-upsyzf-datanode
Name: milvus-upsyzf-datanode
Namespace: default
Labels: app.kubernetes.io/instance=milvus-upsyzf
app.kubernetes.io/managed-by=kubeblocks
apps.kubeblocks.io/cluster-uid=b9c85699-e55d-4693-8120-9386c5605886
apps.kubeblocks.io/component-name=datanode
clusterdefinition.kubeblocks.io/name=milvus
clusterversion.kubeblocks.io/name=
componentdefinition.kubeblocks.io/name=milvus-datanode-0.9.2
Annotations: kubeblocks.io/generation: 2
API Version: apps.kubeblocks.io/v1alpha1
Kind: Component
Metadata:
Creation Timestamp: 2025-11-11T09:50:14Z
Finalizers:
cluster.kubeblocks.io/finalizer
component.kubeblocks.io/finalizer
Generation: 1
Owner References:
API Version: apps.kubeblocks.io/v1alpha1
Block Owner Deletion: true
Controller: true
Kind: Cluster
Name: milvus-upsyzf
UID: b9c85699-e55d-4693-8120-9386c5605886
Resource Version: 115358401
UID: be909273-9683-4ae5-b819-573d6a70f9a3
Spec:
Comp Def: milvus-datanode-0.9.2
Disable Exporter: true
Env:
Name: MINIO_BUCKET
Value: kbcli-test
Name: MINIO_ROOT_PATH
Name: MINIO_USE_PATH_STYLE
Value: true
Replicas: 1
Resources:
Limits:
Cpu: 500m
Memory: 512Mi
Requests:
Cpu: 500m
Memory: 512Mi
Scheduling Policy:
Affinity:
Node Affinity:
Preferred During Scheduling Ignored During Execution:
Preference:
Match Expressions:
Key: kb-data
Operator: In
Values:
true
Weight: 100
Tolerations:
Effect: NoSchedule
Key: kb-data
Operator: Equal
Value: true
Service Refs:
Cluster Service Selector:
Cluster: miniom-upsyzf
Credential:
Component: minio
Name: root
Service:
Component: minio
Port: api
Service: default
Name: milvus-object-storage
Namespace: default
Service Version: 2.3.2
Events: <none>
KB pod CrashLoopBackOff
kubectl get pod -n kb-system
NAME READY STATUS RESTARTS AGE
kubeblocks-6d8746b8d9-wsgjm 0/1 CrashLoopBackOff 5 (44s ago) 36m
logs KB pod
kubectl logs -n kb-system kubeblocks-6d8746b8d9-wsgjm --previous
Defaulted container "manager" out of: manager, tools (init), datascript (init)
2025-11-11T09:51:09.609Z INFO setup config file: /etc/kubeblocks/config.yaml
2025-11-11T09:51:09.609Z INFO setup config settings: map[apps:true cache_sync_timeout:300 cert_dir:/tmp/k8s-webhook-server/serving-certs client_burst:256 client_qps:128 cm_namespace:kb-system cm_recon_retry_duration_ms:1000 component_replicas_annotation:true config_manager_grpc_port:9901 config_manager_log_level:info data_plane_affinity:{"nodeAffinity":{"preferredDuringSchedulingIgnoredDuringExecution":[{"preference":{"matchExpressions":[{"key":"kb-data","operator":"In","values":["true"]}]},"weight":100}]}} data_plane_tolerations:[{"effect":"NoSchedule","key":"kb-data","operator":"Equal","value":"true"}] default_storage_class: dual_operators_mode:false enable_debug_sysaccounts:false enabled_runtime_metrics:false enablerbacmanager:true experimental:false extensions:true health_probe_bind_address::8081 host_port_cm_name:kubeblocks-host-ports host_port_exclude_ranges:6443,10250,10257,10259,2379-2380,30000-32767 host_port_include_ranges:1025-65536 ignore_config_template_default_mode:false ignore_pod_vertical_scaling:false in_place_pod_vertical_scaling:false kb_max_lag:10 kb_ttl:15 kubeblocks_addon_helm_install_options:--atomic --cleanup-on-fail --wait --insecure-skip-tls-verify kubeblocks_addon_helm_uninstall_options:[] kubeblocks_addon_sa_name:kubeblocks-addon-installer kubeblocks_datascript_clients_image:apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-datascript:0.9.6-beta.3 kubeblocks_reconcile_workers:32 kubeblocks_serviceaccount_name:kubeblocks kubeblocks_tools_image:apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.9.6-beta.3 kubeconfig: kubernetes_cluster_domain:cluster.local leader_elect:true leader_elect_id:001c317f leader_elect_lease_duration:30 leader_elect_renew_deadline:25 leader_elect_retry_period:5 lorry_grpc_port:50001 lorry_http_port:3501 lorry_log_level:info managed_namespaces: max_plain_revision_count:1024 maxconcurrentreconciles_addon:16 metrics_bind_address::8080 multi_cluster_contexts: multi_cluster_contexts_disabled: multi_cluster_kubeconfig: no_rsm_env:false pod_min_ready_seconds:10 user_agent:kubeblocks volumesnapshot_api_beta:false workloads:true zap_devel:false zap_encoder:console zap_log_level: zap_stacktrace_level: zap_time_encoding:iso8601]
2025-11-11T09:51:09.609Z INFO setup golang runtime metrics. {"featureGate": false}
2025-11-11T09:51:09.630Z INFO setup starting manager
2025-11-11T09:51:09.630Z INFO controller-runtime.metrics Starting metrics server
2025-11-11T09:51:09.630Z INFO starting server {"kind": "health probe", "addr": "[::]:8081"}
2025-11-11T09:51:09.630Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8080", "secure": false}
I1111 09:51:09.630337 1 leaderelection.go:250] attempting to acquire leader lease kb-system/001c317f.kubeblocks.io...
I1111 09:51:43.913520 1 leaderelection.go:260] successfully acquired lease kb-system/001c317f.kubeblocks.io
2025-11-11T09:51:43.913Z INFO Starting EventSource {"controller": "cluster", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Cluster", "source": "kind source: *v1alpha1.Cluster"}
2025-11-11T09:51:43.913Z INFO Starting EventSource {"controller": "cluster", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Cluster", "source": "kind source: *v1alpha1.Component"}
2025-11-11T09:51:43.913Z INFO Starting EventSource {"controller": "opsdefinition", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsDefinition", "source": "kind source: *v1alpha1.OpsDefinition"}
2025-11-11T09:51:43.913Z INFO Starting EventSource {"controller": "cluster", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Cluster", "source": "kind source: *v1.Service"}
2025-11-11T09:51:43.913Z INFO Starting Controller {"controller": "opsdefinition", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsDefinition"}
2025-11-11T09:51:43.913Z INFO Starting EventSource {"controller": "cluster", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Cluster", "source": "kind source: *v1.Secret"}
2025-11-11T09:51:43.913Z INFO Starting EventSource {"controller": "cluster", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Cluster", "source": "kind source: *v1alpha1.BackupPolicy"}
2025-11-11T09:51:43.913Z INFO Starting EventSource {"controller": "cluster", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Cluster", "source": "kind source: *v1alpha1.BackupSchedule"}
2025-11-11T09:51:43.913Z INFO Starting Controller {"controller": "cluster", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Cluster"}
2025-11-11T09:51:43.913Z INFO Starting EventSource {"controller": "clusterversion", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ClusterVersion", "source": "kind source: *v1alpha1.ClusterVersion"}
2025-11-11T09:51:43.913Z INFO Starting EventSource {"controller": "clusterdefinition", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ClusterDefinition", "source": "kind source: *v1alpha1.ClusterDefinition"}
2025-11-11T09:51:43.913Z INFO Starting Controller {"controller": "clusterversion", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ClusterVersion"}
2025-11-11T09:51:43.913Z INFO Starting Controller {"controller": "clusterdefinition", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ClusterDefinition"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "configconstraint", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ConfigConstraint", "source": "kind source: *v1beta1.ConfigConstraint"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "configconstraint", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ConfigConstraint", "source": "kind source: *v1.ConfigMap"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "configconstraint", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ConfigConstraint"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "cluster", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Cluster", "source": "kind source: *v1alpha1.Cluster"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "opsrequest", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsRequest", "source": "kind source: *v1alpha1.OpsRequest"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "cluster", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Cluster", "source": "kind source: *v1.Secret"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "cluster", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Cluster", "source": "kind source: *v1.Job"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "cluster", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Cluster"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "opsrequest", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsRequest", "source": "kind source: *v1.Job"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "opsrequest", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsRequest", "source": "kind source: *v1alpha1.Restore"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "opsrequest", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsRequest", "source": "kind source: *v1alpha1.Cluster"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "opsrequest", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsRequest", "source": "kind source: *v1alpha1.Component"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "event", "controllerGroup": "", "controllerKind": "Event", "source": "kind source: *v1.Event"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "opsrequest", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsRequest", "source": "kind source: *v1alpha1.InstanceSet"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "event", "controllerGroup": "", "controllerKind": "Event"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "opsrequest", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsRequest", "source": "kind source: *v1alpha1.Backup"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "opsrequest", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsRequest", "source": "kind source: *v1.PersistentVolumeClaim"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "opsrequest", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsRequest", "source": "kind source: *v1.Pod"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "source": "kind source: *v1alpha1.Component"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "opsrequest", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsRequest"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "source": "kind source: *v1alpha1.InstanceSet"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "source": "kind source: *v1.Service"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "source": "kind source: *v1.Secret"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "source": "kind source: *v1.ConfigMap"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "source": "kind source: *v1alpha1.Backup"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "source": "kind source: *v1alpha1.Restore"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "source": "kind source: *v1.Job"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "source": "kind source: *v1.ClusterRoleBinding"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "source": "kind source: *v1.RoleBinding"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "source": "kind source: *v1.ServiceAccount"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "source": "kind source: *v1.PersistentVolumeClaim"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "source": "kind source: *v1alpha1.Configuration"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "source": "kind source: *v1alpha1.Configuration"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "source": "kind source: *v1.ConfigMap"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "configmap", "controllerGroup": "", "controllerKind": "ConfigMap", "source": "kind source: *v1.ConfigMap"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "configmap", "controllerGroup": "", "controllerKind": "ConfigMap"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "componentversion", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ComponentVersion", "source": "kind source: *v1alpha1.ComponentVersion"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "componentversion", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ComponentVersion", "source": "kind source: *v1alpha1.ComponentDefinition"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "componentversion", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ComponentVersion"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "componentdefinition", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ComponentDefinition", "source": "kind source: *v1alpha1.ComponentDefinition"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "componentdefinition", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ComponentDefinition"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "instanceset", "controllerGroup": "workloads.kubeblocks.io", "controllerKind": "InstanceSet", "source": "kind source: *v1alpha1.InstanceSet"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "instanceset", "controllerGroup": "workloads.kubeblocks.io", "controllerKind": "InstanceSet", "source": "kind source: *v1.Job"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "componentclassdefinition", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ComponentClassDefinition", "source": "kind source: *v1alpha1.ComponentClassDefinition"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "componentclassdefinition", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ComponentClassDefinition"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "backuppolicytemplate", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "BackupPolicyTemplate", "source": "kind source: *v1alpha1.BackupPolicyTemplate"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "instanceset", "controllerGroup": "workloads.kubeblocks.io", "controllerKind": "InstanceSet", "source": "kind source: *v1.Service"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "backuppolicytemplate", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "BackupPolicyTemplate", "source": "kind source: *v1alpha1.ComponentDefinition"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "backuppolicytemplate", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "BackupPolicyTemplate"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "instanceset", "controllerGroup": "workloads.kubeblocks.io", "controllerKind": "InstanceSet", "source": "kind source: *v1.ConfigMap"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "instanceset", "controllerGroup": "workloads.kubeblocks.io", "controllerKind": "InstanceSet", "source": "kind source: *v1.Pod"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "addon", "controllerGroup": "extensions.kubeblocks.io", "controllerKind": "Addon", "source": "kind source: *v1alpha1.Addon"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "servicedescriptor", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ServiceDescriptor", "source": "kind source: *v1alpha1.ServiceDescriptor"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "servicedescriptor", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ServiceDescriptor"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "addon", "controllerGroup": "extensions.kubeblocks.io", "controllerKind": "Addon", "source": "kind source: *v1.Job"}
2025-11-11T09:51:43.914Z INFO Starting EventSource {"controller": "instanceset", "controllerGroup": "workloads.kubeblocks.io", "controllerKind": "InstanceSet", "source": "kind source: *v1.PersistentVolumeClaim"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "addon", "controllerGroup": "extensions.kubeblocks.io", "controllerKind": "Addon"}
2025-11-11T09:51:43.914Z INFO Starting Controller {"controller": "instanceset", "controllerGroup": "workloads.kubeblocks.io", "controllerKind": "InstanceSet"}
W1111 09:51:43.925246 1 warnings.go:70] Due to the lack of practical use cases, this API is deprecated from KB 0.9.0.
W1111 09:51:44.007527 1 warnings.go:70] Due to the lack of practical use cases, this API is deprecated from KB 0.9.0.
W1111 09:51:44.009564 1 warnings.go:70] The ClusterVersion CRD has been deprecated since 0.9.0
W1111 09:51:44.011541 1 warnings.go:70] The ClusterVersion CRD has been deprecated since 0.9.0
2025-11-11T09:51:44.426Z INFO Starting workers {"controller": "configconstraint", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ConfigConstraint", "worker count": 1}
2025-11-11T09:51:44.507Z INFO Starting workers {"controller": "configmap", "controllerGroup": "", "controllerKind": "ConfigMap", "worker count": 8}
2025-11-11T09:51:44.507Z INFO Starting workers {"controller": "componentdefinition", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ComponentDefinition", "worker count": 1}
2025-11-11T09:51:44.507Z INFO Starting workers {"controller": "configuration", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Configuration", "worker count": 8}
2025-11-11T09:51:44.507Z INFO Starting workers {"controller": "addon", "controllerGroup": "extensions.kubeblocks.io", "controllerKind": "Addon", "worker count": 16}
2025-11-11T09:51:44.507Z INFO Starting workers {"controller": "backuppolicytemplate", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "BackupPolicyTemplate", "worker count": 1}
2025-11-11T09:51:44.507Z INFO Starting workers {"controller": "servicedescriptor", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ServiceDescriptor", "worker count": 1}
2025-11-11T09:51:44.507Z INFO Starting workers {"controller": "componentclassdefinition", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ComponentClassDefinition", "worker count": 1}
2025-11-11T09:51:44.507Z INFO Starting workers {"controller": "clusterversion", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ClusterVersion", "worker count": 1}
2025-11-11T09:51:44.508Z INFO Starting workers {"controller": "instanceset", "controllerGroup": "workloads.kubeblocks.io", "controllerKind": "InstanceSet", "worker count": 32}
2025-11-11T09:51:44.508Z INFO Starting workers {"controller": "opsdefinition", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsDefinition", "worker count": 1}
2025-11-11T09:51:44.508Z INFO Starting workers {"controller": "componentversion", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ComponentVersion", "worker count": 1}
2025-11-11T09:51:44.508Z INFO Starting workers {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "worker count": 32}
2025-11-11T09:51:44.508Z INFO Starting workers {"controller": "event", "controllerGroup": "", "controllerKind": "Event", "worker count": 8}
2025-11-11T09:51:44.508Z INFO Starting workers {"controller": "cluster", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Cluster", "worker count": 1}
2025-11-11T09:51:44.509Z INFO Starting workers {"controller": "opsrequest", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "OpsRequest", "worker count": 16}
2025-11-11T09:51:44.509Z INFO Starting workers {"controller": "cluster", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Cluster", "worker count": 32}
2025-11-11T09:51:44.509Z INFO Starting workers {"controller": "clusterdefinition", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "ClusterDefinition", "worker count": 1}
W1111 09:51:44.515766 1 warnings.go:70] Due to the lack of practical use cases, this API is deprecated from KB 0.9.0.
W1111 09:51:44.516689 1 warnings.go:70] Due to the lack of practical use cases, this API is deprecated from KB 0.9.0.
2025-11-11T09:51:44.607Z INFO KubeAPIWarningLogger The ClusterVersion CRD has been deprecated since 0.9.0
2025-11-11T09:51:44.611Z INFO ReconfigureRequestReconcile not found configSpec[minio-configuration] in the component[minio]. {"controller": "configmap", "controllerGroup": "", "controllerKind": "ConfigMap", "ConfigMap": {"name":"miniom-upsyzf-minio-minio-configuration","namespace":"default"}, "namespace": "default", "name": "miniom-upsyzf-minio-minio-configuration", "reconcileID": "d837e944-6713-492c-87e8-6edbc525a938", "ConfigMap": {"name":"miniom-upsyzf-minio-minio-configuration","namespace":"default"}, "ClusterName": "miniom-upsyzf", "ComponentName": "minio"}
2025-11-11T09:51:44.613Z INFO Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference {"controller": "component", "controllerGroup": "apps.kubeblocks.io", "controllerKind": "Component", "Component": {"name":"milvus-upsyzf-datanode","namespace":"default"}, "namespace": "default", "name": "milvus-upsyzf-datanode", "reconcileID": "19c401bb-20e5-44a0-ba9d-70d1aaf9fcd1"}
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x130 pc=0x1bdde0c]
goroutine 950 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:116 +0x1e5
panic({0x28966a0?, 0x4ba1b50?})
/usr/local/go/src/runtime/panic.go:914 +0x21f
github.com/apecloud/kubeblocks/pkg/controller/component.resolveServiceRefHostRef.func1({0x2c8b020?, 0x0?})
/src/pkg/controller/component/vars.go:945 +0x2c
github.com/apecloud/kubeblocks/pkg/controller/component.resolveClusterObjectVars({0x2d06b71, 0xa}, {{0x0, 0x0}, {0xc001913920, 0x13}, 0xc002e074d4, 0x0}, 0xc0024ce760, 0xc002c6ed18, ...)
/src/pkg/controller/component/vars.go:1343 +0x478
github.com/apecloud/kubeblocks/pkg/controller/component.resolveServiceRefVarRefLow({0x1?, 0xc23cdfac244aa55e?}, {0xc0024d9c20?, 0x1?}, 0x0?, {{{0x0, 0x0}, {0xc001913920, 0x13}, 0xc002e074d4, ...}, ...}, ...)
/src/pkg/controller/component/vars.go:1147 +0xd2
github.com/apecloud/kubeblocks/pkg/controller/component.resolveServiceRefHostRef({0x33564e0?, 0xc002c4b020?}, {0x333a7d0?, 0xc003cf3a70?}, 0xc00024bea0?, {0xc001913908?, 0x0?}, {{{0x0, 0x0}, {0xc001913920, ...}, ...}, ...})
/src/pkg/controller/component/vars.go:953 +0xc6
github.com/apecloud/kubeblocks/pkg/controller/component.resolveServiceRefVarRef({0x33564e0?, 0xc002c4b020?}, {0x333a7d0?, 0xc003cf3a70?}, 0xc002c6f0b8?, {0xc001913908?, 0x3339ab0?}, {{{0x0, 0x0}, {0xc001913920, ...}, ...}, ...})
/src/pkg/controller/component/vars.go:923 +0xfc
github.com/apecloud/kubeblocks/pkg/controller/component.resolveClusterObjectVarRef({0x33564e0?, 0xc002c4b020?}, {0x333a7d0?, 0xc003cf3a70?}, 0xc003d8a1e0?, {0xc001913908?, 0xc003c9d0e0?}, {0x0, 0x0, 0x0, ...}, ...)
/src/pkg/controller/component/vars.go:497 +0x1be
github.com/apecloud/kubeblocks/pkg/controller/component.resolveClusterObjectRefVars({0x33564e0, 0xc002c4b020}, {0x333a7d0, 0xc003cf3a70}, 0xc001944e00, {0xc000f0d200?, 0xc, 0x0?})
/src/pkg/controller/component/vars.go:465 +0x5ac
github.com/apecloud/kubeblocks/pkg/controller/component.resolveBuiltinNObjectRefVars({0x33564e0, 0xc002c4b020}, {0x333a7d0, 0xc003cf3a70}, 0x0?, {0xc000f0d200, 0xc, 0xc})
/src/pkg/controller/component/vars.go:159 +0xc5
github.com/apecloud/kubeblocks/pkg/controller/component.resolveNewTemplateNEnvVars({0x33564e0?, 0xc002c4b020?}, {0x333a7d0?, 0xc003cf3a70?}, 0xc0006a6be0?, {0xc000f0d200?, 0xc002c6f668?, 0x410645?})
/src/pkg/controller/component/vars.go:136 +0x47
github.com/apecloud/kubeblocks/pkg/controller/component.resolveTemplateNEnvVars({0x33564e0?, 0xc002c4b020?}, {0x333a7d0?, 0xc003cf3a70?}, 0x0?, {0xc000f0d200?, 0x0?, 0x0?}, 0x0)
/src/pkg/controller/component/vars.go:107 +0x57
github.com/apecloud/kubeblocks/pkg/controller/component.ResolveTemplateNEnvVars(...)
/src/pkg/controller/component/vars.go:63
github.com/apecloud/kubeblocks/controllers/apps.(*componentVarsTransformer).Transform(0xc0024cf6c0?, {0x3356070?, 0xc000b821b0}, 0xc0024cf700)
/src/controllers/apps/transformer_component_vars.go:74 +0x32e
github.com/apecloud/kubeblocks/pkg/controller/graph.TransformerChain.ApplyTo({0xc000923e40?, 0x16, 0xc002c5a650?}, {0x3356070, 0xc000b821b0}, 0x417852?)
/src/pkg/controller/graph/transformer.go:57 +0x9b
github.com/apecloud/kubeblocks/controllers/apps.(*componentPlanBuilder).Build(0xc000a56410)
/src/controllers/apps/component_plan_builder.go:117 +0xb6
github.com/apecloud/kubeblocks/controllers/apps.(*ComponentReconciler).Reconcile(0xc000723290, {0x33564e0, 0xc002c4ad20}, {{{0xc000c33440, 0x7}, {0xc000ba0858, 0x16}}})
/src/controllers/apps/component_controller.go:188 +0xa0e
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x335c5e8?, {0x33564e0?, 0xc002c4ad20?}, {{{0xc000c33440?, 0xb?}, {0xc000ba0858?, 0x0?}}})
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119 +0xb7
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0003e92c0, {0x3356518, 0xc000256690}, {0x29d5f40?, 0xc00033a0a0?})
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316 +0x3cc
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0003e92c0, {0x3356518, 0xc000256690})
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266 +0x1af
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227 +0x79
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 118
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:223 +0x565
Expected behavior A clear and concise description of what you expected to happen.
Screenshots If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
Additional context Add any other context about the problem here.
