kubeblocks
kubeblocks copied to clipboard
apecloud
[BUG] pulsar cluster pulsar-broker component crash upgrade kb from 0.7.* to 0.8.2 on Azure AKS
Describe the bug A clear and concise description of what the bug is.
To Reproduce Steps to reproduce the behavior:
- create pulasr cluster
apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
name: pulsar-upkb781
namespace: default
spec:
clusterDefinitionRef: pulsar
clusterVersionRef: pulsar-2.11.2
terminationPolicy: WipeOut
affinity:
topologyKeys:
- kubernetes.io/hostname
componentSpecs:
- name: pulsar-broker
componentDefRef: pulsar-broker
monitor: false
replicas: 2
resources:
requests:
cpu: 500m
memory: 0.5Gi
limits:
cpu: 500m
memory: 0.5Gi
- name: bookies
componentDefRef: bookies
monitor: false
replicas: 3
resources:
requests:
cpu: 500m
memory: 0.5Gi
limits:
cpu: 500m
memory: 0.5Gi
volumeClaimTemplates:
- name: ledgers
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
- name: journal
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
- name: zookeeper
componentDefRef: zookeeper
monitor: false
replicas: 3
resources:
requests:
cpu: 100m
memory: 0.5Gi
limits:
cpu: 100m
memory: 0.5Gi
volumeClaimTemplates:
- name: data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
- name: pulsar-proxy
componentDefRef: pulsar-proxy
monitor: false
replicas: 1
resources:
requests:
cpu: 100m
memory: 0.5Gi
limits:
cpu: 100m
memory: 0.5Gi
- upgrade from 0.7.* to 0.8.2
- See error
kubectl get pod -l app.kubernetes.io/instance=pulsar-upkb781
NAME READY STATUS RESTARTS AGE
pulsar-upkb781-bookies-0 2/2 Running 0 15m
pulsar-upkb781-bookies-1 2/2 Running 0 16m
pulsar-upkb781-bookies-2 2/2 Running 0 17m
pulsar-upkb781-pulsar-broker-0 3/3 Running 0 41m
pulsar-upkb781-pulsar-broker-1 2/3 CrashLoopBackOff 8 (34s ago) 17m
pulsar-upkb781-pulsar-proxy-0 2/2 Running 0 45m
pulsar-upkb781-zookeeper-0 2/2 Running 0 36m
pulsar-upkb781-zookeeper-1 2/2 Running 0 36m
pulsar-upkb781-zookeeper-2 2/2 Running 0 37m
describe pod
kubectl describe pod pulsar-upkb781-pulsar-broker-1
Name: pulsar-upkb781-pulsar-broker-1
Namespace: default
Priority: 0
Node: aks-testpool-55647713-vmss00001f/10.224.0.9
Start Time: Tue, 19 Mar 2024 10:56:39 +0800
Labels: app.kubernetes.io/component=pulsar-broker
app.kubernetes.io/instance=pulsar-upkb781
app.kubernetes.io/managed-by=kubeblocks
app.kubernetes.io/name=pulsar
app.kubernetes.io/version=pulsar-2.11.2
apps.kubeblocks.io/cluster-uid=deae6f57-6315-4e5a-846d-0c49bf0b3302
apps.kubeblocks.io/component-name=pulsar-broker
apps.kubeblocks.io/workload-type=Stateful
clusterdefinition.kubeblocks.io/name=pulsar
clusterversion.kubeblocks.io/name=pulsar-2.11.2
controller-revision-hash=pulsar-upkb781-pulsar-broker-5cf985b684
statefulset.kubernetes.io/pod-name=pulsar-upkb781-pulsar-broker-1
Annotations: apps.kubeblocks.io/component-replicas: 2
config.kubeblocks.io/restart-broker-config: 69c4f95446
kubeblocks.io/restart: 2024-03-19T02:28:03Z
Status: Running
IP: 10.244.22.69
IPs:
IP: 10.244.22.69
Controlled By: StatefulSet/pulsar-upkb781-pulsar-broker
Init Containers:
init-broker-cluster:
Container ID: containerd://7ac169b37c5f2e26081078b850f46fbee2ea1b15bf9c7bb89c8d0de6810b4c61
Image: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsar:2.11.2
Image ID: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsar@sha256:d394d32706c927c54b28a2e0f051ac7ee8ac25bdb69a3fb23d7de6510e4daa1a
Port: <none>
Host Port: <none>
Command:
/kb-scripts/init-broker.sh
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 19 Mar 2024 10:56:40 +0800
Finished: Tue, 19 Mar 2024 10:56:40 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Environment Variables from:
pulsar-upkb781-pulsar-broker-env ConfigMap Optional: false
pulsar-upkb781-pulsar-broker-broker-env-envfrom ConfigMap Optional: false
Environment:
KB_POD_NAME: pulsar-upkb781-pulsar-broker-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_CLUSTER_NAME: pulsar-upkb781
KB_COMP_NAME: pulsar-broker
KB_CLUSTER_COMP_NAME: pulsar-upkb781-pulsar-broker
KB_CLUSTER_UID_POSTFIX_8: bf0b3302
KB_POD_FQDN: $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
brokerServiceUrl: pulsar://$(KB_CLUSTER_COMP_NAME).$(KB_NAMESPACE).svc.cluster.local:6650
clusterName: $(KB_NAMESPACE)-$(KB_CLUSTER_COMP_NAME)
webServiceUrl: http://$(KB_CLUSTER_COMP_NAME).$(KB_NAMESPACE).svc.cluster.local:80
Mounts:
/kb-scripts from scripts (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-pmzt2 (ro)
init-sysctl:
Container ID: containerd://858b84d0b60546fed350097b7f1e7fe8e6e21f7cc953ba51fe0b7652dbfb1e64
Image: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsar:2.11.2
Image ID: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsar@sha256:d394d32706c927c54b28a2e0f051ac7ee8ac25bdb69a3fb23d7de6510e4daa1a
Port: <none>
Host Port: <none>
Command:
/kb-scripts/init-broker-sysctl.sh
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 19 Mar 2024 10:56:41 +0800
Finished: Tue, 19 Mar 2024 10:56:41 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Environment Variables from:
pulsar-upkb781-pulsar-broker-env ConfigMap Optional: false
Environment:
KB_POD_NAME: pulsar-upkb781-pulsar-broker-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_CLUSTER_NAME: pulsar-upkb781
KB_COMP_NAME: pulsar-broker
KB_CLUSTER_COMP_NAME: pulsar-upkb781-pulsar-broker
KB_CLUSTER_UID_POSTFIX_8: bf0b3302
KB_POD_FQDN: $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
Mounts:
/kb-scripts from scripts (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-pmzt2 (ro)
init-pulsar-tools:
Container ID: containerd://f6cb45ff5e4c85fd86e5742e03dce89bb0918fe5425676e85a294f9771e5f561
Image: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsartools:2.11
Image ID: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsartools@sha256:ca94a8331febfe3a51b79ffe3c366a554e708c9a607fb9f6c0539cc28914a49a
Port: <none>
Host Port: <none>
Command:
cp
/bin/pulsar-admin
/kb/tools/pulsar-admin
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 19 Mar 2024 10:56:43 +0800
Finished: Tue, 19 Mar 2024 10:56:43 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Environment Variables from:
pulsar-upkb781-pulsar-broker-env ConfigMap Optional: false
Environment:
KB_POD_NAME: pulsar-upkb781-pulsar-broker-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_CLUSTER_NAME: pulsar-upkb781
KB_COMP_NAME: pulsar-broker
KB_CLUSTER_COMP_NAME: pulsar-upkb781-pulsar-broker
KB_CLUSTER_UID_POSTFIX_8: bf0b3302
KB_POD_FQDN: $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
TOOLS_SCRIPTS_PATH: /opt/kb-tools/reload/broker-config
Mounts:
/kb/tools from kb-tools (rw)
/opt/config-manager from config-manager-config (rw)
/opt/kb-tools/reload/broker-config from cm-script-broker-config (rw)
/opt/pulsar/conf from pulsar-config (rw)
/opt/pulsar/none from broker-env (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-pmzt2 (ro)
init-pulsar-client-config:
Container ID: containerd://c6409cd5ec4fb26b8a8a91c165918e57bb7e087475fbd7d765b18c7fa0500e50
Image: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsar:2.11.2
Image ID: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsar@sha256:d394d32706c927c54b28a2e0f051ac7ee8ac25bdb69a3fb23d7de6510e4daa1a
Port: <none>
Host Port: <none>
Command:
$(TOOLS_SCRIPTS_PATH)/install-pulsar-client-config.sh
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 19 Mar 2024 10:56:44 +0800
Finished: Tue, 19 Mar 2024 10:56:44 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Environment Variables from:
pulsar-upkb781-pulsar-broker-env ConfigMap Optional: false
pulsar-upkb781-pulsar-broker-broker-env-envfrom ConfigMap Optional: false
Environment:
KB_POD_NAME: pulsar-upkb781-pulsar-broker-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_CLUSTER_NAME: pulsar-upkb781
KB_COMP_NAME: pulsar-broker
KB_CLUSTER_COMP_NAME: pulsar-upkb781-pulsar-broker
KB_CLUSTER_UID_POSTFIX_8: bf0b3302
KB_POD_FQDN: $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
TOOLS_SCRIPTS_PATH: /opt/kb-tools/reload/broker-config
Mounts:
/kb/tools from kb-tools (rw)
/opt/config-manager from config-manager-config (rw)
/opt/kb-tools/reload/broker-config from cm-script-broker-config (rw)
/opt/pulsar/conf from pulsar-config (rw)
/opt/pulsar/none from broker-env (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-pmzt2 (ro)
Containers:
broker:
Container ID: containerd://41e7ad68558757c7ae4b77911886ee25f9338f4742ae175f3b81abc679728dd1
Image: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsar:2.11.2
Image ID: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsar@sha256:d394d32706c927c54b28a2e0f051ac7ee8ac25bdb69a3fb23d7de6510e4daa1a
Ports: 6650/TCP, 8080/TCP, 9092/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP
Command:
sh
-c
Args:
set -x
/kb-scripts/merge_pulsar_config.py conf/client.conf /opt/pulsar/conf/client.conf && \
/kb-scripts/merge_pulsar_config.py conf/broker.conf /opt/pulsar/conf/broker.conf && \
bin/apply-config-from-env.py conf/broker.conf && \
bin/apply-config-from-env.py conf/client.conf && \
echo 'OK' > status;exec bin/pulsar broker
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Tue, 19 Mar 2024 11:13:17 +0800
Finished: Tue, 19 Mar 2024 11:13:21 +0800
Ready: False
Restart Count: 8
Limits:
cpu: 600m
memory: 644245094400m
Requests:
cpu: 600m
memory: 644245094400m
Liveness: http-get http://:http/status.html delay=5s timeout=100s period=10s #success=1 #failure=30
Readiness: http-get http://:http/status.html delay=30s timeout=100s period=10s #success=1 #failure=3
Startup: http-get http://:http/status.html delay=5s timeout=100s period=10s #success=1 #failure=30
Environment Variables from:
pulsar-upkb781-pulsar-broker-env ConfigMap Optional: false
pulsar-upkb781-pulsar-broker-broker-env-envfrom ConfigMap Optional: false
pulsar-upkb781-pulsar-broker-rsm-env ConfigMap Optional: false
Environment:
KB_POD_NAME: pulsar-upkb781-pulsar-broker-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_CLUSTER_NAME: pulsar-upkb781
KB_COMP_NAME: pulsar-broker
KB_CLUSTER_COMP_NAME: pulsar-upkb781-pulsar-broker
KB_CLUSTER_UID_POSTFIX_8: bf0b3302
KB_POD_FQDN: $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
SERVICE_PORT: 8080
POD_NAME: $(KB_POD_NAME)
PULSAR_PREFIX_brokerServicePort: 6650
PULSAR_PREFIX_internalListenerName: cluster
PULSAR_PREFIX_advertisedListeners: cluster:pulsar://$(POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc.cluster.local:6650
PULSAR_PREFIX_allowAutoTopicCreationType: partitioned
PULSAR_PREFIX_kafkaListeners: PLAINTEXT://0.0.0.0:9092
PULSAR_PREFIX_kafkaAdvertisedListeners: PLAINTEXT://$(POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc.cluster.local:9092
brokerServiceUrl: pulsar://$(KB_CLUSTER_COMP_NAME).$(KB_NAMESPACE).svc.cluster.local:6650
clusterName: $(KB_NAMESPACE)-$(KB_CLUSTER_COMP_NAME)
webServiceUrl: http://$(KB_CLUSTER_COMP_NAME).$(KB_NAMESPACE).svc.cluster.local:80
Mounts:
/kb-scripts from scripts (rw)
/kb/tools from kb-tools (rw)
/opt/pulsar/conf from pulsar-config (rw)
/opt/pulsar/none from broker-env (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-pmzt2 (ro)
metrics:
Container ID: containerd://c4325c184572415712ba7cfc3775299753d37ab3054dc327fd1ab983a4d2ab2c
Image: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto:0.1.2-beta.2
Image ID: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto@sha256:a85d7d80d33de188ab39880d8d6013b663ebade9186371ce09a06c29dfa443ac
Port: 1234/TCP
Host Port: 0/TCP
Command:
/bin/agamotto
--config=/opt/agamotto/agamotto-config.yaml
--feature-gates=-pkg.translator.prometheus.NormalizeName
State: Running
Started: Tue, 19 Mar 2024 10:56:45 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Environment Variables from:
pulsar-upkb781-pulsar-broker-env ConfigMap Optional: false
pulsar-upkb781-pulsar-broker-rsm-env ConfigMap Optional: false
Environment:
KB_POD_NAME: pulsar-upkb781-pulsar-broker-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_CLUSTER_NAME: pulsar-upkb781
KB_COMP_NAME: pulsar-broker
KB_CLUSTER_COMP_NAME: pulsar-upkb781-pulsar-broker
KB_CLUSTER_UID_POSTFIX_8: bf0b3302
KB_POD_FQDN: $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
JOB_NAME: broker
SERVICE_PORT: 8080
POD_NAME: $(KB_POD_NAME)
POD_NAMESPACE: $(KB_NAMESPACE)
LOG_LEVEL: info
Mounts:
/opt/agamotto from agamotto-configuration (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-pmzt2 (ro)
config-manager:
Container ID: containerd://ba422c1cee838b712334ecb71eb92be9ee37318ea8de82c004d73c9c08e3206c
Image: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.2-beta.27
Image ID: infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools@sha256:eb7b34b730892bc80a94b0f5a5d16450370ef84ad4fa24f37f009bedc266465e
Port: <none>
Host Port: <none>
Command:
env
Args:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$(TOOLS_PATH)
/bin/reloader
--log-level
info
--operator-update-enable
--tcp
9901
--config
/opt/config-manager/config-manager.yaml
State: Running
Started: Tue, 19 Mar 2024 10:56:45 +0800
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Environment Variables from:
pulsar-upkb781-pulsar-broker-env ConfigMap Optional: false
pulsar-upkb781-pulsar-broker-rsm-env ConfigMap Optional: false
Environment:
KB_POD_NAME: pulsar-upkb781-pulsar-broker-1 (v1:metadata.name)
KB_POD_UID: (v1:metadata.uid)
KB_NAMESPACE: default (v1:metadata.namespace)
KB_SA_NAME: (v1:spec.serviceAccountName)
KB_NODENAME: (v1:spec.nodeName)
KB_HOST_IP: (v1:status.hostIP)
KB_POD_IP: (v1:status.podIP)
KB_POD_IPS: (v1:status.podIPs)
KB_HOSTIP: (v1:status.hostIP)
KB_PODIP: (v1:status.podIP)
KB_PODIPS: (v1:status.podIPs)
KB_CLUSTER_NAME: pulsar-upkb781
KB_COMP_NAME: pulsar-broker
KB_CLUSTER_COMP_NAME: pulsar-upkb781-pulsar-broker
KB_CLUSTER_UID_POSTFIX_8: bf0b3302
KB_POD_FQDN: $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
CONFIG_MANAGER_POD_IP: (v1:status.podIP)
DB_TYPE: pulsar-broker
TOOLS_PATH: /opt/kb-tools/reload/broker-config:/opt/config-manager:/kb/tools
Mounts:
/kb/tools from kb-tools (rw)
/opt/config-manager from config-manager-config (rw)
/opt/kb-tools/reload/broker-config from cm-script-broker-config (rw)
/opt/pulsar/conf from pulsar-config (rw)
/opt/pulsar/none from broker-env (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-pmzt2 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
agamotto-configuration:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: pulsar-upkb781-pulsar-broker-agamotto-configuration
Optional: false
pulsar-config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: pulsar-upkb781-pulsar-broker-broker-config
Optional: false
broker-env:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: pulsar-upkb781-pulsar-broker-broker-env
Optional: false
scripts:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: pulsar-upkb781-pulsar-broker-pulsar-scripts
Optional: false
cm-script-broker-config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: sidecar-pulsar-tools-script-pulsar-upkb781
Optional: false
config-manager-config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: sidecar-pulsar-upkb781-pulsar-broker-config-manager-config
Optional: false
kb-tools:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
kube-api-access-pmzt2:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: kb-data=true:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 17m default-scheduler Successfully assigned default/pulsar-upkb781-pulsar-broker-1 to aks-testpool-55647713-vmss00001f
Normal Pulled 17m kubelet Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsar:2.11.2" already present on machine
Normal Created 17m kubelet Created container init-broker-cluster
Normal Started 17m kubelet Started container init-broker-cluster
Normal Pulled 17m kubelet Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsar:2.11.2" already present on machine
Normal Created 17m kubelet Created container init-sysctl
Normal Started 17m kubelet Started container init-sysctl
Normal Pulling 17m kubelet Pulling image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsartools:2.11"
Normal Pulled 17m kubelet Successfully pulled image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsartools:2.11" in 1.161215269s (1.161219869s including waiting)
Normal Created 17m kubelet Created container init-pulsar-tools
Normal Started 17m kubelet Started container init-pulsar-tools
Normal Pulled 17m kubelet Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsar:2.11.2" already present on machine
Normal Started 17m kubelet Started container init-pulsar-client-config
Normal Created 17m kubelet Created container init-pulsar-client-config
Normal Pulled 17m kubelet Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.2-beta.27" already present on machine
Normal Created 17m kubelet Created container config-manager
Normal Started 17m kubelet Started container broker
Normal Pulled 17m kubelet Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto:0.1.2-beta.2" already present on machine
Normal Created 17m kubelet Created container metrics
Normal Started 17m kubelet Started container metrics
Normal Started 17m kubelet Started container config-manager
Normal Pulled 17m (x2 over 17m) kubelet Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/pulsar:2.11.2" already present on machine
Normal Created 17m (x2 over 17m) kubelet Created container broker
Warning Unhealthy 17m kubelet Startup probe failed: Get "http://10.244.22.69:8080/status.html": dial tcp 10.244.22.69:8080: connect: connection refused
Warning BackOff 2m26s (x75 over 17m) kubelet Back-off restarting failed container broker in pod pulsar-upkb781-pulsar-broker-1_default(f0a3b07a-c1c8-405d-805c-ac66bf919c24)
logs error pod
kubectl logs pulsar-upkb781-pulsar-broker-1 broker
+ /kb-scripts/merge_pulsar_config.py conf/client.conf /opt/pulsar/conf/client.conf
Traceback (most recent call last):
File "/kb-scripts/merge_pulsar_config.py", line 59, in <module>
merge_config_files(dst_files, src_files)
File "/kb-scripts/merge_pulsar_config.py", line 45, in merge_config_files
lines, keys = load_config_file(src_file)
File "/kb-scripts/merge_pulsar_config.py", line 11, in load_config_file
for line in open(file):
FileNotFoundError: [Errno 2] No such file or directory: '/opt/pulsar/conf/client.conf'
+ exec bin/pulsar broker
VM settings:
Max. Heap Size (Estimated): 186.00M
Using VM: OpenJDK 64-Bit Server VM
2024-03-19T03:13:21,131Z [jdk.internal.loader.ClassLoaders$AppClassLoader@5bc2b487] error Uncaught exception in thread main: Required clusterName is null
java.lang.IllegalArgumentException: Required clusterName is null
at org.apache.pulsar.common.configuration.PulsarConfigurationLoader.isComplete(PulsarConfigurationLoader.java:153)
at org.apache.pulsar.PulsarBrokerStarter.loadConfig(PulsarBrokerStarter.java:72)
at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.<init>(PulsarBrokerStarter.java:169)
at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:331)
Expected behavior pulsar cluster ok.
Screenshots If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
Additional context Add any other context about the problem here.
This issue has been marked as stale because it has been open for 30 days with no activity
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}