kubeblocks
kubeblocks copied to clipboard
apecloud
[BUG] minio/zookeeprr upgrade to v1 pod pull msoap/shell2http:1.16.0 timeout
Describe the bug A clear and concise description of what the bug is.
kbcli version
Kubernetes: v1.30.4-vke.4
KubeBlocks: 1.0.0-beta.41,0.9.4-beta.10
kbcli: 1.0.0-beta.20
To Reproduce Steps to reproduce the behavior:
- install minio addon 0.9.0
helm list -n kb-system
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
kb-addon-minio kb-system 2 2025-04-02 14:24:55.812432 +0800 CST deployed minio-0.9.0 master
- create cluster in kb 0.9.4-beta.10
apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
name: minio-cluster
namespace: default
spec:
terminationPolicy: WipeOut
componentSpecs:
- componentDef: minio
name: minio
replicas: 2
env:
- name: MINIO_BUCKETS
name: minio
resources:
limits:
cpu: 100m
memory: 0.5Gi
requests:
cpu: 100m
memory: 0.5Gi
volumeClaimTemplates:
- name: data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
- install kb 1.0.0-beta.41
- install minio addon 1.0.0-alpha.0
helm list -n kb-system-new
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
kb-addon-minio kb-system-new 1 2025-04-03 10:27:07.772072 +0800 CST deployed minio-1.0.0-alpha.0 master
- upgrade minio to v1
echo yes|kbcli cluster upgrade-to-v1 minio-cluster
Cluster minio-cluster will be converted to v1 with output as yaml.
Please type 'Yes/yes' to confirm your operation: yes
Cluster minio-cluster has converted successfully, you can view the spec:
kubectl get clusters.apps.kubeblocks.io minio-cluster -n default -oyaml
- See error
kubectl get cluster minio-cluster
NAME CLUSTER-DEFINITION TERMINATION-POLICY STATUS AGE
minio-cluster WipeOut Running 20h
➜ ~
➜ ~ kubectl get pod -l app.kubernetes.io/instance=minio-cluster
NAME READY STATUS RESTARTS AGE
minio-cluster-minio-0 2/2 Running 0 20h
minio-cluster-minio-1 0/3 Init:ImagePullBackOff 0 46m
describe pod
kubectl describe pod minio-cluster-minio-0
Name: minio-cluster-minio-0
Namespace: default
Priority: 0
Service Account: kb-minio-cluster
Node: 172.31.0.9/172.31.0.9
Start Time: Wed, 02 Apr 2025 14:27:37 +0800
Labels: app.kubernetes.io/component=minio
app.kubernetes.io/instance=minio-cluster
app.kubernetes.io/managed-by=kubeblocks
app.kubernetes.io/name=minio
app.kubernetes.io/version=minio
apps.kubeblocks.io/cluster-uid=bb8e0c4a-5fbf-465d-a010-e7d763171033
apps.kubeblocks.io/component-name=minio
apps.kubeblocks.io/pod-name=minio-cluster-minio-0
componentdefinition.kubeblocks.io/name=minio
controller-revision-hash=85446d69fc
kubeblocks.io/role=readwrite
workloads.kubeblocks.io/access-mode=ReadWrite
workloads.kubeblocks.io/instance=minio-cluster-minio
workloads.kubeblocks.io/managed-by=InstanceSet
Annotations: apps.kubeblocks.io/component-replicas: 2
apps.kubeblocks.io/last-role-snapshot-version: 1743575646775318
vke.volcengine.com/cello-pod-evict-policy: allow
Status: Running
IP: 172.31.0.15
IPs:
IP: 172.31.0.15
Controlled By: InstanceSet/minio-cluster-minio
Init Containers:
init:
Container ID: containerd://12e0ef93dd52a2054c60c50920eb2e8a2552986a41b193d9df755dd77dc541bc
Image: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.2
Image ID: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools@sha256:51b497cdae889a32d3e90f4b276bb3968c84c8f95cfeeed2dadd24d005bc5ffd
Port: <none>
Host Port: <none>
Command:
/bin/sh
-ce
name="$KB_CLUSTER_COMP_NAME"-minio-configuration
namespace="$KB_NAMESPACE"
key="MINIO_REPLICAS_HISTORY"
cur=$(kubectl get configmaps "$name" -n "$namespace" -o jsonpath="{.data.$key}")
cur=$(echo "$cur" | tr -d '[]')
if [[ -z "$cur" ]]; then
new="[$KB_COMP_REPLICAS]"
else
IFS=',' set -- $cur
array="$@"
last=$(echo $array | awk '{print $NF}')
if [[ "$last" == "$KB_COMP_REPLICAS" ]]; then
new="[$cur]"
else
new="[$cur,$KB_COMP_REPLICAS]"
fi
fi
kubectl patch configmap "$name" -n "$namespace" --type strategic -p "{\"data\":{\"$key\":\"$new\"}}"
echo "ConfigMap $name updated successfully with $key=$new"
State: Terminated
Reason: Completed
Exit Code: 0
Started: Wed, 02 Apr 2025 14:30:11 +0800
Finished: Wed, 02 Apr 2025 14:30:11 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Environment Variables from:
minio-cluster-minio-env ConfigMap Optional: false
Environment:
MINIO_ROOT_USER: <set to the key 'username' in secret 'minio-cluster-minio-account-root'> Optional: false
MINIO_ROOT_PASSWORD: <set to the key 'password' in secret 'minio-cluster-minio-account-root'> Optional: false
KB_POD_NAME: minio-cluster-minio-0 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_POD_FQDN: $(KB_POD_NAME).minio-cluster-minio-headless.$(KB_NAMESPACE).svc
MINIO_BUCKETS:
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ntvqf (ro)
init-lorry:
Container ID: containerd://e43647a348356ea09ace475397cb432fe82535f82d9c2f33f57a40c152b0ca9e
Image: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.9.4-beta.10
Image ID: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools@sha256:d45e681c78d04d02a52fe2d8396329d8b4781f72733ec9c07c3190e711f91888
Port: <none>
Host Port: <none>
Command:
cp
-r
/bin/lorry
/config
/bin/curl
/kubeblocks/
State: Terminated
Reason: Completed
Exit Code: 0
Started: Wed, 02 Apr 2025 14:30:41 +0800
Finished: Wed, 02 Apr 2025 14:30:41 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Environment Variables from:
minio-cluster-minio-env ConfigMap Optional: false
Environment:
MINIO_ROOT_USER: <set to the key 'username' in secret 'minio-cluster-minio-account-root'> Optional: false
MINIO_ROOT_PASSWORD: <set to the key 'password' in secret 'minio-cluster-minio-account-root'> Optional: false
KB_POD_NAME: minio-cluster-minio-0 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_POD_FQDN: $(KB_POD_NAME).minio-cluster-minio-headless.$(KB_NAMESPACE).svc
Mounts:
/kubeblocks from kubeblocks (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ntvqf (ro)
Containers:
minio:
Container ID: containerd://b585c6d1defd17c408f5a752f4435c6e50ebd40737b497a0013f7c1f42eb1cb5
Image: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/minio:RELEASE.2024-06-29T01-20-47Z
Image ID: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/minio@sha256:fce0a90a37bb2887c850b68b61a0ac8ac02ea218b58eaea96bb7ca69eb503e20
Ports: 9000/TCP, 9001/TCP
Host Ports: 0/TCP, 0/TCP
Command:
/bin/sh
-ce
FILE="/minio-config/MINIO_REPLICAS_HISTORY"
if [ ! -f "$FILE" ]; then
echo "minio config don't existed"
exit
fi
# init bucket if set
buckets=$MINIO_BUCKETS
if [ -n "$buckets" ]; then
IFS=',' read -ra BUCKET_ARRAY <<< "$buckets"
for bucket in "${BUCKET_ARRAY[@]}"; do
bucket=$(echo $bucket)
directory="/data/$bucket"
mkdir -p "$directory"
if [ $? -eq 0 ]; then
echo "Successfully init bucket: $directory"
else
echo "Failed to init bucket: $directory"
fi
done
fi
# read the content
content=$(cat "$FILE")
content=$(echo "$content" | tr -d '[]')
IFS=',' read -r -a replicas <<< "$content"
server=""
for ((i=0; i < ${#replicas[@]}; i++)); do
if [ $i -eq 0 ]; then
cur=${replicas[i]}
server+="http://$KB_CLUSTER_COMP_NAME-{0...$((cur-1))}.$KB_CLUSTER_COMP_NAME-headless.$KB_NAMESPACE.svc.cluster.local/data"
else
prev=${replicas[i-1]}
cur=${replicas[i]}
server+=" http://$KB_CLUSTER_COMP_NAME-{$((prev))...$((cur-1))}.$KB_CLUSTER_COMP_NAME-headless.$KB_NAMESPACE.svc.cluster.local/data"
fi
done
echo "the minio server pool is $server"
cmd="/usr/bin/docker-entrypoint.sh minio server "$server" -S /etc/minio/certs/ --address :9000 --console-address :9001"
$cmd
State: Running
Started: Wed, 02 Apr 2025 14:32:40 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 100m
memory: 512Mi
vke.volcengine.com/eni-ip: 1
Requests:
cpu: 100m
memory: 512Mi
vke.volcengine.com/eni-ip: 1
Environment Variables from:
minio-cluster-minio-env ConfigMap Optional: false
minio-cluster-minio-rsm-env ConfigMap Optional: false
Environment:
MINIO_ROOT_USER: <set to the key 'username' in secret 'minio-cluster-minio-account-root'> Optional: false
MINIO_ROOT_PASSWORD: <set to the key 'password' in secret 'minio-cluster-minio-account-root'> Optional: false
KB_POD_NAME: minio-cluster-minio-0 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_POD_FQDN: $(KB_POD_NAME).minio-cluster-minio-headless.$(KB_NAMESPACE).svc
MINIO_ROOT_USER: $(MINIO_ROOT_USER)
MINIO_ROOT_PASSWORD: $(MINIO_ROOT_PASSWORD)
MINIO_BROWSER_LOGIN_ANIMATION: off
MINIO_BUCKETS:
Mounts:
/data from data (rw)
/minio-config from minio-config (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ntvqf (ro)
lorry:
Container ID: containerd://9c6cb11f7e344c91543e0bfb0c89a1c525473bcf26f256aea2b11fa6f8c09975
Image: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/minio:RELEASE.2024-06-29T01-20-47Z
Image ID: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/minio@sha256:fce0a90a37bb2887c850b68b61a0ac8ac02ea218b58eaea96bb7ca69eb503e20
Ports: 3501/TCP, 50001/TCP
Host Ports: 0/TCP, 0/TCP
Command:
/kubeblocks/lorry
--port
3501
--grpcport
50001
--config-path
/kubeblocks/config/lorry/components/
State: Running
Started: Wed, 02 Apr 2025 14:32:41 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Readiness: http-get http://:3501/v1.0/checkrole delay=0s timeout=1s period=10s #success=1 #failure=3
Startup: tcp-socket :3501 delay=0s timeout=1s period=10s #success=1 #failure=3
Environment Variables from:
minio-cluster-minio-env ConfigMap Optional: false
minio-cluster-minio-rsm-env ConfigMap Optional: false
Environment:
MINIO_ROOT_USER: <set to the key 'username' in secret 'minio-cluster-minio-account-root'> Optional: false
MINIO_ROOT_PASSWORD: <set to the key 'password' in secret 'minio-cluster-minio-account-root'> Optional: false
KB_POD_NAME: minio-cluster-minio-0 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_POD_FQDN: $(KB_POD_NAME).minio-cluster-minio-headless.$(KB_NAMESPACE).svc
KB_RSM_ROLE_PROBE_PERIOD: 0
KB_BUILTIN_HANDLER: custom
KB_SERVICE_USER: <set to the key 'username' in secret 'minio-cluster-minio-account-root'> Optional: false
KB_SERVICE_PASSWORD: <set to the key 'password' in secret 'minio-cluster-minio-account-root'> Optional: false
KB_SERVICE_PORT: 9000
KB_DATA_PATH: /data
KB_ACTION_COMMANDS: {"roleProbe":["/bin/sh","-c","if mc config host add minio http://127.0.0.1:9000 $MINIO_ROOT_USER $MINIO_ROOT_PASSWORD \u0026\u003e/dev/null; then\n echo -n \"readwrite\"\nfi\n"]}
MINIO_ROOT_USER: $(MINIO_ROOT_USER)
MINIO_ROOT_PASSWORD: $(MINIO_ROOT_PASSWORD)
MINIO_BROWSER_LOGIN_ANIMATION: off
MINIO_BUCKETS:
KB_RSM_ACTION_SVC_LIST: null
KB_RSM_ROLE_UPDATE_MECHANISM: DirectAPIServerEventUpdate
KB_RSM_ROLE_PROBE_TIMEOUT: 1
KB_CLUSTER_NAME: (v1:metadata.labels['app.kubernetes.io/instance'])
KB_COMP_NAME: (v1:metadata.labels['apps.kubeblocks.io/component-name'])
KB_SERVICE_CHARACTER_TYPE: custom
Mounts:
/data from data (rw)
/kubeblocks from kubeblocks (rw)
/minio-config from minio-config (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ntvqf (ro)
Conditions:
Type Status
PodReadyToStartContainers True
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
minio-config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: minio-cluster-minio-minio-configuration
Optional: false
data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: data-minio-cluster-minio-0
ReadOnly: false
kubeblocks:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
kube-api-access-ntvqf:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: kb-data=true:NoSchedule
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events: <none>
➜ ~ kubectl describe pod minio-cluster-minio-1
Name: minio-cluster-minio-1
Namespace: default
Priority: 0
Service Account: kb-minio-cluster
Node: 172.31.0.2/172.31.0.2
Start Time: Thu, 03 Apr 2025 10:38:10 +0800
Labels: app.kubernetes.io/component=minio
app.kubernetes.io/instance=minio-cluster
app.kubernetes.io/managed-by=kubeblocks
app.kubernetes.io/name=minio
app.kubernetes.io/version=minio
apps.kubeblocks.io/cluster-uid=bb8e0c4a-5fbf-465d-a010-e7d763171033
apps.kubeblocks.io/component-name=minio
apps.kubeblocks.io/pod-name=minio-cluster-minio-1
componentdefinition.kubeblocks.io/name=minio
controller-revision-hash=848c898874
workloads.kubeblocks.io/instance=minio-cluster-minio
workloads.kubeblocks.io/managed-by=InstanceSet
Annotations: apps.kubeblocks.io/component-replicas: 2
vke.volcengine.com/cello-pod-evict-policy: allow
Status: Pending
IP: 172.31.0.71
IPs:
IP: 172.31.0.71
Controlled By: InstanceSet/minio-cluster-minio
Init Containers:
init:
Container ID: containerd://0a505bf16401575504bbf8d9df86c6288a2ee947451102c11a70246107b1f3fe
Image: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.2
Image ID: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools@sha256:51b497cdae889a32d3e90f4b276bb3968c84c8f95cfeeed2dadd24d005bc5ffd
Port: <none>
Host Port: <none>
Command:
/bin/sh
-ce
name="$KB_CLUSTER_COMP_NAME"-minio-configuration
namespace="$KB_NAMESPACE"
key="MINIO_REPLICAS_HISTORY"
cur=$(kubectl get configmaps "$name" -n "$namespace" -o jsonpath="{.data.$key}")
cur=$(echo "$cur" | tr -d '[]')
if [[ -z "$cur" ]]; then
new="[$KB_COMP_REPLICAS]"
else
IFS=',' set -- $cur
array="$@"
last=$(echo $array | awk '{print $NF}')
if [[ "$last" == "$KB_COMP_REPLICAS" ]]; then
new="[$cur]"
else
new="[$cur,$KB_COMP_REPLICAS]"
fi
fi
kubectl patch configmap "$name" -n "$namespace" --type strategic -p "{\"data\":{\"$key\":\"$new\"}}"
echo "ConfigMap $name updated successfully with $key=$new"
State: Terminated
Reason: Completed
Exit Code: 0
Started: Thu, 03 Apr 2025 10:38:20 +0800
Finished: Thu, 03 Apr 2025 10:38:20 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Environment Variables from:
minio-cluster-minio-env ConfigMap Optional: false
Environment:
MINIO_ROOT_USER: <set to the key 'username' in secret 'minio-cluster-minio-account-root'> Optional: false
MINIO_ROOT_PASSWORD: <set to the key 'password' in secret 'minio-cluster-minio-account-root'> Optional: false
KB_POD_NAME: minio-cluster-minio-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_POD_FQDN: $(KB_POD_NAME).minio-cluster-minio-headless.$(KB_NAMESPACE).svc
MINIO_BUCKETS:
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-h6tpj (ro)
init-lorry:
Container ID: containerd://8e3f997a0a875c50b256637a4357eaaebd57b6edd836c00f43175e31cb958b47
Image: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.9.4-beta.10
Image ID: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools@sha256:d45e681c78d04d02a52fe2d8396329d8b4781f72733ec9c07c3190e711f91888
Port: <none>
Host Port: <none>
Command:
cp
-r
/bin/lorry
/config
/bin/curl
/kubeblocks/
State: Terminated
Reason: Completed
Exit Code: 0
Started: Thu, 03 Apr 2025 10:38:21 +0800
Finished: Thu, 03 Apr 2025 10:38:21 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Environment Variables from:
minio-cluster-minio-env ConfigMap Optional: false
Environment:
MINIO_ROOT_USER: <set to the key 'username' in secret 'minio-cluster-minio-account-root'> Optional: false
MINIO_ROOT_PASSWORD: <set to the key 'password' in secret 'minio-cluster-minio-account-root'> Optional: false
KB_POD_NAME: minio-cluster-minio-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_POD_FQDN: $(KB_POD_NAME).minio-cluster-minio-headless.$(KB_NAMESPACE).svc
Mounts:
/kubeblocks from kubeblocks (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-h6tpj (ro)
role-agent-installer:
Container ID:
Image: msoap/shell2http:1.16.0
Image ID:
Port: <none>
Host Port: <none>
Command:
cp
/app/shell2http
/role-probe/agent
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/role-probe from role-agent (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-h6tpj (ro)
Containers:
minio:
Container ID:
Image: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/minio:RELEASE.2024-06-29T01-20-47Z
Image ID:
Ports: 9000/TCP, 9001/TCP
Host Ports: 0/TCP, 0/TCP
Command:
/bin/sh
-ce
FILE="/minio-config/MINIO_REPLICAS_HISTORY"
if [ ! -f "$FILE" ]; then
echo "minio config don't existed"
exit
fi
# init bucket if set
buckets=$MINIO_BUCKETS
if [ -n "$buckets" ]; then
IFS=',' read -ra BUCKET_ARRAY <<< "$buckets"
for bucket in "${BUCKET_ARRAY[@]}"; do
bucket=$(echo $bucket)
directory="/data/$bucket"
mkdir -p "$directory"
if [ $? -eq 0 ]; then
echo "Successfully init bucket: $directory"
else
echo "Failed to init bucket: $directory"
fi
done
fi
# read the content
content=$(cat "$FILE")
content=$(echo "$content" | tr -d '[]')
IFS=',' read -r -a replicas <<< "$content"
server=""
for ((i=0; i < ${#replicas[@]}; i++)); do
if [ $i -eq 0 ]; then
cur=${replicas[i]}
server+="http://$KB_CLUSTER_COMP_NAME-{0...$((cur-1))}.$KB_CLUSTER_COMP_NAME-headless.$KB_NAMESPACE.svc.cluster.local/data"
else
prev=${replicas[i-1]}
cur=${replicas[i]}
server+=" http://$KB_CLUSTER_COMP_NAME-{$((prev))...$((cur-1))}.$KB_CLUSTER_COMP_NAME-headless.$KB_NAMESPACE.svc.cluster.local/data"
fi
done
echo "the minio server pool is $server"
cmd="/usr/bin/docker-entrypoint.sh minio server "$server" -S /etc/minio/certs/ --address :9000 --console-address :9001"
$cmd
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Limits:
cpu: 100m
memory: 512Mi
vke.volcengine.com/eni-ip: 1
Requests:
cpu: 100m
memory: 512Mi
vke.volcengine.com/eni-ip: 1
Environment Variables from:
minio-cluster-minio-env ConfigMap Optional: false
minio-cluster-minio-rsm-env ConfigMap Optional: false
Environment:
MINIO_ROOT_USER: <set to the key 'username' in secret 'minio-cluster-minio-account-root'> Optional: false
MINIO_ROOT_PASSWORD: <set to the key 'password' in secret 'minio-cluster-minio-account-root'> Optional: false
KB_POD_NAME: minio-cluster-minio-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_POD_FQDN: $(KB_POD_NAME).minio-cluster-minio-headless.$(KB_NAMESPACE).svc
MINIO_ROOT_USER: $(MINIO_ROOT_USER)
MINIO_ROOT_PASSWORD: $(MINIO_ROOT_PASSWORD)
MINIO_BROWSER_LOGIN_ANIMATION: off
MINIO_BUCKETS:
Mounts:
/data from data (rw)
/minio-config from minio-config (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-h6tpj (ro)
lorry:
Container ID:
Image: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/minio:RELEASE.2024-06-29T01-20-47Z
Image ID:
Ports: 3501/TCP, 50001/TCP
Host Ports: 0/TCP, 0/TCP
Command:
/kubeblocks/lorry
--port
3501
--grpcport
50001
--config-path
/kubeblocks/config/lorry/components/
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Readiness: http-get http://:3501/v1.0/checkrole delay=0s timeout=1s period=10s #success=1 #failure=3
Startup: tcp-socket :3501 delay=0s timeout=1s period=10s #success=1 #failure=3
Environment Variables from:
minio-cluster-minio-env ConfigMap Optional: false
minio-cluster-minio-rsm-env ConfigMap Optional: false
Environment:
MINIO_ROOT_USER: <set to the key 'username' in secret 'minio-cluster-minio-account-root'> Optional: false
MINIO_ROOT_PASSWORD: <set to the key 'password' in secret 'minio-cluster-minio-account-root'> Optional: false
KB_POD_NAME: minio-cluster-minio-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_POD_FQDN: $(KB_POD_NAME).minio-cluster-minio-headless.$(KB_NAMESPACE).svc
KB_RSM_ROLE_PROBE_PERIOD: 0
KB_BUILTIN_HANDLER: custom
KB_SERVICE_USER: <set to the key 'username' in secret 'minio-cluster-minio-account-root'> Optional: false
KB_SERVICE_PASSWORD: <set to the key 'password' in secret 'minio-cluster-minio-account-root'> Optional: false
KB_SERVICE_PORT: 9000
KB_DATA_PATH: /data
KB_ACTION_COMMANDS: {"roleProbe":["/bin/sh","-c","if mc config host add minio http://127.0.0.1:9000 $MINIO_ROOT_USER $MINIO_ROOT_PASSWORD \u0026\u003e/dev/null; then\n echo -n \"readwrite\"\nfi\n"]}
MINIO_ROOT_USER: $(MINIO_ROOT_USER)
MINIO_ROOT_PASSWORD: $(MINIO_ROOT_PASSWORD)
MINIO_BROWSER_LOGIN_ANIMATION: off
MINIO_BUCKETS:
KB_RSM_ACTION_SVC_LIST: [36501]
KB_RSM_ROLE_UPDATE_MECHANISM: DirectAPIServerEventUpdate
KB_RSM_ROLE_PROBE_TIMEOUT: 1
KB_CLUSTER_NAME: (v1:metadata.labels['app.kubernetes.io/instance'])
KB_COMP_NAME: (v1:metadata.labels['apps.kubeblocks.io/component-name'])
KB_SERVICE_CHARACTER_TYPE: custom
Mounts:
/data from data (rw)
/kubeblocks from kubeblocks (rw)
/minio-config from minio-config (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-h6tpj (ro)
action-0:
Container ID:
Image: busybox:1.35
Image ID:
Port: <none>
Host Port: <none>
Command:
/role-probe/agent
-port
36501
-export-all-vars
-form
/role
/bin/sh -c if mc config host add minio http://127.0.0.1:9000 $MINIO_ROOT_USER $MINIO_ROOT_PASSWORD &>/dev/null; then
echo -n "readwrite"
fi
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
KB_RSM_USERNAME: <set to the key 'username' in secret 'minio-cluster-minio-account-root'> Optional: false
KB_RSM_PASSWORD: <set to the key 'password' in secret 'minio-cluster-minio-account-root'> Optional: false
Mounts:
/role-probe from role-agent (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-h6tpj (ro)
Conditions:
Type Status
PodReadyToStartContainers True
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
minio-config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: minio-cluster-minio-minio-configuration
Optional: false
data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: data-minio-cluster-minio-1
ReadOnly: false
kubeblocks:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
role-agent:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
kube-api-access-h6tpj:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: kb-data=true:NoSchedule
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 46m default-scheduler Successfully assigned default/minio-cluster-minio-1 to 172.31.0.2
Normal Pulled 46m kubelet Successfully pulled image "apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.9.4-beta.10" in 192ms (192ms including waiting). Image size: 140696452 bytes.
Normal Pulled 46m kubelet Successfully pulled image "apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.2" in 144ms (144ms including waiting). Image size: 130768516 bytes.
Normal Created 46m kubelet Created container init
Normal Started 46m kubelet Started container init
Normal Pulling 46m kubelet Pulling image "apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.9.4-beta.10"
Normal Pulling 46m kubelet Pulling image "apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.2"
Normal Started 46m kubelet Started container init-lorry
Normal Created 46m kubelet Created container init-lorry
Warning Failed 45m (x2 over 46m) kubelet Failed to pull image "msoap/shell2http:1.16.0": failed to pull and unpack image "docker.io/msoap/shell2http:1.16.0": failed to resolve reference "docker.io/msoap/shell2http:1.16.0": failed to do request: Head "https://registry-1.docker.io/v2/msoap/shell2http/manifests/1.16.0": dial tcp 199.96.61.1:443: connect: connection timed out
Normal Pulling 45m (x3 over 46m) kubelet Pulling image "msoap/shell2http:1.16.0"
Warning Failed 44m (x3 over 46m) kubelet Error: ErrImagePull
Warning Failed 44m kubelet Failed to pull image "msoap/shell2http:1.16.0": failed to pull and unpack image "docker.io/msoap/shell2http:1.16.0": failed to resolve reference "docker.io/msoap/shell2http:1.16.0": failed to do request: Head "https://registry-1.docker.io/v2/msoap/shell2http/manifests/1.16.0": dial tcp 31.13.94.49:443: connect: connection timed out
Warning Failed 44m (x4 over 45m) kubelet Error: ImagePullBackOff
Normal BackOff 75s (x184 over 45m) kubelet Back-off pulling image "msoap/shell2http:1.16.0"
Expected behavior A clear and concise description of what you expected to happen.
Screenshots If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
Additional context Add any other context about the problem here.
kbcli version
Kubernetes: v1.30.4-vke.4
KubeBlocks: 1.0.0-beta.41,0.9.4-beta.10
kbcli: 1.0.0-beta.20
helm list -n kb-system
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
kb-addon-zookeeper kb-system 2 2025-04-02 14:25:25.011235 +0800 CST deployed zookeeper-0.9.0 3.7.2
create cluster
apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
name: zkeeper-cluster
namespace: default
spec:
terminationPolicy: WipeOut
componentSpecs:
- name: zookeeper
componentDef: zookeeper
replicas: 3
serviceVersion: 3.6.4
disableExporter: true
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: 500m
memory: 1Gi
services:
env:
- name: ZOOKEEPER_IMAGE_VERSION
value: "3.6.4"
volumeClaimTemplates:
- name: data
spec:
storageClassName:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
- name: log
spec:
storageClassName:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
kubectl get cluster zkeeper-cluster
NAME CLUSTER-DEFINITION TERMINATION-POLICY STATUS AGE
zkeeper-cluster WipeOut Running 21h
kubectl get pod -l app.kubernetes.io/instance=zkeeper-cluster
NAME READY STATUS RESTARTS AGE
zkeeper-cluster-zookeeper-0 2/2 Running 0 21h
zkeeper-cluster-zookeeper-1 0/3 Init:ImagePullBackOff 0 46m
zkeeper-cluster-zookeeper-2 2/2 Running 0 21h
kubectl describe pod zkeeper-cluster-zookeeper-1
Name: zkeeper-cluster-zookeeper-1
Namespace: default
Priority: 0
Service Account: kb-zkeeper-cluster
Node: 172.31.0.75/172.31.0.75
Start Time: Thu, 03 Apr 2025 10:41:20 +0800
Labels: app.kubernetes.io/component=zookeeper
app.kubernetes.io/instance=zkeeper-cluster
app.kubernetes.io/managed-by=kubeblocks
app.kubernetes.io/name=zookeeper
app.kubernetes.io/version=zookeeper
apps.kubeblocks.io/cluster-uid=5af0ed70-2548-4899-bd51-5a5fd562aece
apps.kubeblocks.io/component-name=zookeeper
apps.kubeblocks.io/pod-name=zkeeper-cluster-zookeeper-1
componentdefinition.kubeblocks.io/name=zookeeper
controller-revision-hash=754bbdbd6c
workloads.kubeblocks.io/instance=zkeeper-cluster-zookeeper
workloads.kubeblocks.io/managed-by=InstanceSet
Annotations: apps.kubeblocks.io/component-replicas: 3
vke.volcengine.com/cello-pod-evict-policy: allow
Status: Pending
IP: 172.31.0.40
IPs:
IP: 172.31.0.40
Controlled By: InstanceSet/zkeeper-cluster-zookeeper
Init Containers:
init-lorry:
Container ID: containerd://e228ae8a2b1209b084943e5cdd083c7bd9f912aa1dea2a572eb4f23955a477fd
Image: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.9.4-beta.10
Image ID: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools@sha256:d45e681c78d04d02a52fe2d8396329d8b4781f72733ec9c07c3190e711f91888
Port: <none>
Host Port: <none>
Command:
cp
-r
/bin/lorry
/config
/bin/curl
/kubeblocks/
State: Terminated
Reason: Completed
Exit Code: 0
Started: Thu, 03 Apr 2025 10:41:29 +0800
Finished: Thu, 03 Apr 2025 10:41:29 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Environment Variables from:
zkeeper-cluster-zookeeper-env ConfigMap Optional: false
Environment:
KB_POD_NAME: zkeeper-cluster-zookeeper-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_POD_FQDN: $(KB_POD_NAME).zkeeper-cluster-zookeeper-headless.$(KB_NAMESPACE).svc
Mounts:
/kubeblocks from kubeblocks (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vnpvg (ro)
role-agent-installer:
Container ID:
Image: msoap/shell2http:1.16.0
Image ID:
Port: <none>
Host Port: <none>
Command:
cp
/app/shell2http
/role-probe/agent
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/role-probe from role-agent (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vnpvg (ro)
Containers:
zookeeper:
Container ID:
Image: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/zookeeper:3.6.4
Image ID:
Ports: 2181/TCP, 2888/TCP, 3888/TCP, 8080/TCP, 7000/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP
Command:
/bin/bash
-c
/kubeblocks/scripts/startup.sh
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Limits:
cpu: 500m
memory: 1Gi
vke.volcengine.com/eni-ip: 1
Requests:
cpu: 500m
memory: 1Gi
vke.volcengine.com/eni-ip: 1
Liveness: exec [/bin/bash -c . "$ZOOBINDIR"/zkEnv.sh > /dev/null
java -cp "$CLASSPATH" $CLIENT_JVMFLAGS $JVMFLAGS org.apache.zookeeper.client.FourLetterWordMain \
localhost 2181 ruok | grep imok
] delay=30s timeout=5s period=10s #success=1 #failure=6
Readiness: exec [bash -c . "$ZOOBINDIR"/zkEnv.sh > /dev/null
java -cp "$CLASSPATH" $CLIENT_JVMFLAGS $JVMFLAGS org.apache.zookeeper.client.FourLetterWordMain \
localhost 2181 ruok | grep imok
] delay=5s timeout=5s period=10s #success=1 #failure=6
Environment Variables from:
zkeeper-cluster-zookeeper-env ConfigMap Optional: false
zkeeper-cluster-zookeeper-rsm-env ConfigMap Optional: false
Environment:
KB_POD_NAME: zkeeper-cluster-zookeeper-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_POD_FQDN: $(KB_POD_NAME).zkeeper-cluster-zookeeper-headless.$(KB_NAMESPACE).svc
ZOOBINDIR: /opt/bitnami/zookeeper/bin
ZOOBIN: /opt/bitnami/zookeeper/bin
ZOO_ENABLE_AUTH: yes
SERVICE_PORT: 7000
ZOO_LOG4J_PROP: INFO,ROLLINGFILE,TRACEFILE,CONSOLE
CURRENT_POD_NAME: zkeeper-cluster-zookeeper-1 (v1:metadata.name)
ZOOKEEPER_IMAGE_VERSION: 3.6.4
Mounts:
/bitnami/zookeeper/data from data (rw)
/bitnami/zookeeper/log from snapshot-log (rw)
/kubeblocks/scripts from scripts (rw)
/opt/bitnami/zookeeper/conf/log4j.properties from config-log (rw,path="log4j.properties")
/opt/bitnami/zookeeper/conf/logback.xml from config-log (rw,path="logback.xml")
/opt/bitnami/zookeeper/conf/zoo.cfg from config (rw,path="zoo.cfg")
/opt/bitnami/zookeeper/logs from log (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vnpvg (ro)
lorry:
Container ID:
Image: apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/zookeeper:3.6.4
Image ID:
Ports: 3501/TCP, 50001/TCP
Host Ports: 0/TCP, 0/TCP
Command:
/kubeblocks/lorry
--port
3501
--grpcport
50001
--config-path
/kubeblocks/config/lorry/components/
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Readiness: http-get http://:3501/v1.0/checkrole delay=0s timeout=1s period=10s #success=1 #failure=3
Startup: tcp-socket :3501 delay=0s timeout=1s period=10s #success=1 #failure=3
Environment Variables from:
zkeeper-cluster-zookeeper-env ConfigMap Optional: false
zkeeper-cluster-zookeeper-rsm-env ConfigMap Optional: false
Environment:
KB_POD_NAME: zkeeper-cluster-zookeeper-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_POD_FQDN: $(KB_POD_NAME).zkeeper-cluster-zookeeper-headless.$(KB_NAMESPACE).svc
KB_RSM_ROLE_PROBE_PERIOD: 0
KB_BUILTIN_HANDLER: custom
KB_SERVICE_PORT: 2181
KB_DATA_PATH: /bitnami/zookeeper/data
KB_ACTION_COMMANDS: {"roleProbe":["/bin/bash","-c",". \"$ZOOBINDIR\"/zkEnv.sh \u003e /dev/null\nSTAT=$(java -cp \"$CLASSPATH\" $CLIENT_JVMFLAGS $JVMFLAGS org.apache.zookeeper.client.FourLetterWordMain \\\nlocalhost 2181 srvr 2\u003e /dev/null | grep Mode)\nmode=$(echo \"$STAT\" | awk -F': ' '{print $2}' | awk '{RS=\"\"; ORS=\"\"; print}' )\nif [[ \"$mode\" == \"standalone\" ]]; then\n echo -n \"leader\"\nelse\n echo -n ${mode}\nfi\n"]}
ZOOBINDIR: /opt/bitnami/zookeeper/bin
ZOOBIN: /opt/bitnami/zookeeper/bin
ZOO_ENABLE_AUTH: yes
SERVICE_PORT: 7000
ZOO_LOG4J_PROP: INFO,ROLLINGFILE,TRACEFILE,CONSOLE
CURRENT_POD_NAME: zkeeper-cluster-zookeeper-1 (v1:metadata.name)
ZOOKEEPER_IMAGE_VERSION: 3.6.4
KB_RSM_ACTION_SVC_LIST: [36501]
KB_RSM_ROLE_UPDATE_MECHANISM: DirectAPIServerEventUpdate
KB_RSM_ROLE_PROBE_TIMEOUT: 1
KB_CLUSTER_NAME: (v1:metadata.labels['app.kubernetes.io/instance'])
KB_COMP_NAME: (v1:metadata.labels['apps.kubeblocks.io/component-name'])
KB_SERVICE_CHARACTER_TYPE: custom
Mounts:
/bitnami/zookeeper/data from data (rw)
/bitnami/zookeeper/log from snapshot-log (rw)
/kubeblocks from kubeblocks (rw)
/kubeblocks/scripts from scripts (rw)
/opt/bitnami/zookeeper/conf/log4j.properties from config-log (rw,path="log4j.properties")
/opt/bitnami/zookeeper/conf/logback.xml from config-log (rw,path="logback.xml")
/opt/bitnami/zookeeper/conf/zoo.cfg from config (rw,path="zoo.cfg")
/opt/bitnami/zookeeper/logs from log (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vnpvg (ro)
action-0:
Container ID:
Image: busybox:1.35
Image ID:
Port: <none>
Host Port: <none>
Command:
/role-probe/agent
-port
36501
-export-all-vars
-form
/role
/bin/bash -c . "$ZOOBINDIR"/zkEnv.sh > /dev/null
STAT=$(java -cp "$CLASSPATH" $CLIENT_JVMFLAGS $JVMFLAGS org.apache.zookeeper.client.FourLetterWordMain \
localhost 2181 srvr 2> /dev/null | grep Mode)
mode=$(echo "$STAT" | awk -F': ' '{print $2}' | awk '{RS=""; ORS=""; print}' )
if [[ "$mode" == "standalone" ]]; then
echo -n "leader"
else
echo -n ${mode}
fi
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/role-probe from role-agent (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vnpvg (ro)
Conditions:
Type Status
PodReadyToStartContainers True
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
snapshot-log:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: zkeeper-cluster-zookeeper-zookeeper-config
Optional: false
config-log:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: zkeeper-cluster-zookeeper-zookeeper-log
Optional: false
scripts:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: zkeeper-cluster-zookeeper-zookeeper-scripts
Optional: false
data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: data-zkeeper-cluster-zookeeper-1
ReadOnly: false
log:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: log-zkeeper-cluster-zookeeper-1
ReadOnly: false
kubeblocks:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
role-agent:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
kube-api-access-vnpvg:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: kb-data=true:NoSchedule
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 47m default-scheduler Successfully assigned default/zkeeper-cluster-zookeeper-1 to 172.31.0.75
Normal Pulling 47m kubelet Pulling image "apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.9.4-beta.10"
Normal Pulled 47m kubelet Successfully pulled image "apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.9.4-beta.10" in 168ms (168ms including waiting). Image size: 140696452 bytes.
Normal Created 47m kubelet Created container init-lorry
Normal Started 47m kubelet Started container init-lorry
Warning Failed 46m kubelet Failed to pull image "msoap/shell2http:1.16.0": failed to pull and unpack image "docker.io/msoap/shell2http:1.16.0": failed to resolve reference "docker.io/msoap/shell2http:1.16.0": failed to do request: Head "https://registry-1.docker.io/v2/msoap/shell2http/manifests/1.16.0": dial tcp 104.244.45.246:443: connect: connection timed out
Warning Failed 45m kubelet Failed to pull image "msoap/shell2http:1.16.0": failed to pull and unpack image "docker.io/msoap/shell2http:1.16.0": failed to resolve reference "docker.io/msoap/shell2http:1.16.0": failed to do request: Head "https://registry-1.docker.io/v2/msoap/shell2http/manifests/1.16.0": dial tcp 104.244.43.128:443: connect: connection timed out
Warning Failed 45m (x4 over 46m) kubelet Error: ImagePullBackOff
Normal Pulling 44m (x4 over 47m) kubelet Pulling image "msoap/shell2http:1.16.0"
Warning Failed 44m (x4 over 46m) kubelet Error: ErrImagePull
Warning Failed 44m (x2 over 46m) kubelet Failed to pull image "msoap/shell2http:1.16.0": failed to pull and unpack image "docker.io/msoap/shell2http:1.16.0": failed to resolve reference "docker.io/msoap/shell2http:1.16.0": failed to do request: Head "https://registry-1.docker.io/v2/msoap/shell2http/manifests/1.16.0": dial tcp 156.233.67.243:443: connect: connection timed out
Normal BackOff 7m2s (x157 over 46m) kubelet Back-off pulling image "msoap/shell2http:1.16.0"
Warning Failed 2m (x6 over 23m) kubelet (combined from similar events): Failed to pull image "msoap/shell2http:1.16.0": failed to pull and unpack image "docker.io/msoap/shell2http:1.16.0": failed to resolve reference "docker.io/msoap/shell2http:1.16.0": failed to do request: Head "https://registry-1.docker.io/v2/msoap/shell2http/manifests/1.16.0": dial tcp 128.242.240.180:443: connect: connection timed out
has been fixed
echo yes|kbcli cluster upgrade-to-v1 minio-cluster
┌──────────────────────────────────────────────────────────────┐ ┌────────────────────────────────────────────────────────────┐
│apiVersion: apps.kubeblocks.io/v1alpha1 │ │apiVersion: apps.kubeblocks.io/v1 │
│kind: Cluster │ │kind: Cluster │
│metadata: │ │metadata: │
│ annotations: │ │ annotations: │
│ kubeblocks.io/crd-api-version: apps.kubeblocks.io/v1alpha1│ │ kubeblocks.io/crd-api-version: apps.kubeblocks.io/v1 │
│ kubeblocks.io/reconcile: "2025-04-28T07:41:01.52010029Z" │ │ kubeblocks.io/reconcile: "2025-04-28T07:41:01.52010029Z"│
│ creationTimestamp: "2025-04-28T07:23:41Z" │ │ creationTimestamp: "2025-04-28T07:23:41Z" │
│ finalizers: │ │ finalizers: │
│ - cluster.kubeblocks.io/finalizer │ │ - cluster.kubeblocks.io/finalizer │
│ generation: 2 │ │ generation: 2 │
│ name: minio-cluster │ │ name: minio-cluster │
│ namespace: default │ │ namespace: default │
│ resourceVersion: "36329" │ │ resourceVersion: "36329" │
│ uid: af7e959a-a08e-4345-b551-b40d4aab589e │ │ uid: af7e959a-a08e-4345-b551-b40d4aab589e │
│spec: │ │spec: │
│ componentSpecs: │ │ componentSpecs: │
│ - componentDef: minio │ │ - componentDef: minio-1.0.0-alpha.0 │
│ env: │ │ env: │
│ - name: MINIO_BUCKETS │ │ - name: MINIO_BUCKETS │
│ name: minio │ │ name: minio │
│ replicas: 2 │ │ replicas: 2 │
│ resources: │ │ resources: │
│ limits: │ │ limits: │
│ cpu: 100m │ │ cpu: 100m │
│ memory: 512Mi │ │ memory: 512Mi │
│ requests: │ │ requests: │
│ cpu: 100m │ │ cpu: 100m │
│ memory: 512Mi │ │ memory: 512Mi │
│ serviceVersion: 0.9.0 │ │ serviceVersion: 2024.6.29 │
│ volumeClaimTemplates: │ │ volumeClaimTemplates: │
│ - name: data │ │ - name: data │
│ spec: │ │ spec: │
│ accessModes: │ │ accessModes: │
│ - ReadWriteOnce │ │ - ReadWriteOnce │
│ resources: │ │ resources: │
│ requests: │ │ requests: │
│ storage: 20Gi │ │ storage: 20Gi │
│ resources: │ │ terminationPolicy: WipeOut │
│ cpu: "0" │ │status: {} │
│ memory: "0" │ │ │
│ storage: │ └────────────────────────────────────────────────────────────┘
│ size: "0" │
│ terminationPolicy: WipeOut │
│status: {} │
│ │
└──────────────────────────────────────────────────────────────┘
Cluster minio-cluster will be converted to v1 with output as yaml.
Please type 'Yes/yes' to confirm your operation: yes
minio-cluster-minio
Cluster minio-cluster has converted successfully, you can view the spec:
kubectl get clusters.apps.kubeblocks.io minio-cluster -n default -oyaml
kubectl get cluster minio-cluster
NAME CLUSTER-DEFINITION TERMINATION-POLICY STATUS AGE
minio-cluster WipeOut Running 72m
➜ ~
➜ ~ kubectl get pod -l app.kubernetes.io/instance=minio-cluster
NAME READY STATUS RESTARTS AGE
minio-cluster-minio-0 2/2 Running 0 12m
minio-cluster-minio-1 2/2 Running 0 14m
➜ ~
➜ ~ kubectl get cluster minio-cluster -oyaml
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
annotations:
kubeblocks.io/crd-api-version: apps.kubeblocks.io/v1
kubeblocks.io/reconcile: "2025-04-28T07:41:01.52010029Z"
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"apps.kubeblocks.io/v1alpha1","kind":"Cluster","metadata":{"annotations":{},"name":"minio-cluster","namespace":"default"},"spec":{"componentSpecs":[{"componentDef":"minio","env":[{"name":"MINIO_BUCKETS"}],"name":"minio","replicas":2,"resources":{"limits":{"cpu":"100m","memory":"0.5Gi"},"requests":{"cpu":"100m","memory":"0.5Gi"}},"volumeClaimTemplates":[{"name":"data","spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"20Gi"}}}}]}],"terminationPolicy":"WipeOut"}}
creationTimestamp: "2025-04-28T07:23:41Z"
finalizers:
- cluster.kubeblocks.io/finalizer
generation: 3
name: minio-cluster
namespace: default
resourceVersion: "63406"
uid: af7e959a-a08e-4345-b551-b40d4aab589e
spec:
componentSpecs:
- componentDef: minio-1.0.0-alpha.0
env:
- name: MINIO_BUCKETS
name: minio
replicas: 2
resources:
limits:
cpu: 100m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
serviceVersion: 2024.6.29
volumeClaimTemplates:
- name: data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
terminationPolicy: WipeOut
status:
components:
minio:
message:
InstanceSet/minio-cluster-minio: '["minio-cluster-minio-0"]'
phase: Running
conditions:
- lastTransitionTime: "2025-04-28T08:07:36Z"
message: 'The operator has started the provisioning of Cluster: minio-cluster'
observedGeneration: 3
reason: PreCheckSucceed
status: "True"
type: ProvisioningStarted
- lastTransitionTime: "2025-04-28T07:23:48Z"
message: Successfully applied for resources
observedGeneration: 3
reason: ApplyResourcesSucceed
status: "True"
type: ApplyResources
- lastTransitionTime: "2025-04-28T07:35:56Z"
message: all pods of components are ready, waiting for the probe detection successful
reason: AllReplicasReady
status: "True"
type: ReplicasReady
- lastTransitionTime: "2025-04-28T07:35:56Z"
message: cluster minio-cluster is ready
reason: ClusterReady
status: "True"
type: Ready
observedGeneration: 3
phase: Running