某个calico-node 的pod一直在restart

[peterbooy]

查看log结果: ` [root@cl-net /]# kubectl describe pod calico-node-nhfkn -n kube-system Name: calico-node-nhfkn Namespace: kube-system Priority: 2000001000 Priority Class Name: system-node-critical Node: tx-node/172.21.0.13 Start Time: Tue, 28 Jun 2022 14:50:38 +0800 Labels: controller-revision-hash=55dddb9c8d k8s-app=calico-node pod-template-generation=1 Annotations: scheduler.alpha.kubernetes.io/critical-pod: Status: Running IP: 172.21.0.13 IPs: IP: 172.21.0.13 Controlled By: DaemonSet/calico-node Init Containers: upgrade-ipam: Container ID: docker://cba9fdad046b953b95a2eb9d9b1cc9c7b08720aa1e837f0fb3413935a54e2c9c Image: calico/cni:v3.13.1 Image ID: docker-pullable://calico/cni@sha256:c699d5ec4d0799ca5785e9134cfb1f55a1376ebdbb607f5601394736fceef7c8 Port: Host Port: Command: /opt/cni/bin/calico-ipam -upgrade State: Terminated Reason: Completed Exit Code: 0 Started: Tue, 28 Jun 2022 14:50:39 +0800 Finished: Tue, 28 Jun 2022 14:50:39 +0800 Ready: True Restart Count: 0 Environment: KUBERNETES_NODE_NAME: (v1:spec.nodeName) CALICO_NETWORKING_BACKEND: <set to the key 'calico_backend' of config map 'calico-config'> Optional: false Mounts: /host/opt/cni/bin from cni-bin-dir (rw) /var/lib/cni/networks from host-local-net-dir (rw) /var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-6rzvd (ro) install-cni: Container ID: docker://72bf3d6713495bef76842d4b7a79d790cab2d63a38a09a772063ee55db60b914 Image: calico/cni:v3.13.1 Image ID: docker-pullable://calico/cni@sha256:c699d5ec4d0799ca5785e9134cfb1f55a1376ebdbb607f5601394736fceef7c8 Port: Host Port: Command: /install-cni.sh State: Terminated Reason: Completed Exit Code: 0 Started: Tue, 28 Jun 2022 14:50:40 +0800 Finished: Tue, 28 Jun 2022 14:50:40 +0800 Ready: True Restart Count: 0 Environment: CNI_CONF_NAME: 10-calico.conflist CNI_NETWORK_CONFIG: <set to the key 'cni_network_config' of config map 'calico-config'> Optional: false KUBERNETES_NODE_NAME: (v1:spec.nodeName) CNI_MTU: <set to the key 'veth_mtu' of config map 'calico-config'> Optional: false SLEEP: false Mounts: /host/etc/cni/net.d from cni-net-dir (rw) /host/opt/cni/bin from cni-bin-dir (rw) /var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-6rzvd (ro) flexvol-driver: Container ID: docker://6c4722cccfab9214ac28f9a42c75c77bd9c849c4c1ab131b9df05c4abec97866 Image: calico/pod2daemon-flexvol:v3.13.1 Image ID: docker-pullable://calico/pod2daemon-flexvol@sha256:762e1d75e4e031066fe4b67b6f386520923b48067354bb6e9e53215957a4c6b4 Port: Host Port: State: Terminated Reason: Completed Exit Code: 0 Started: Tue, 28 Jun 2022 14:50:41 +0800 Finished: Tue, 28 Jun 2022 14:50:41 +0800 Ready: True Restart Count: 0 Environment: Mounts: /host/driver from flexvol-driver-host (rw) /var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-6rzvd (ro) Containers: calico-node: Container ID: docker://fef1c042b8bb34cd7a3a14c45a13ceae9ddb150a2980a3d2cfa6900074102fca Image: calico/node:v3.13.1 Image ID: docker-pullable://calico/node@sha256:f24c59e93881178bfae85ee1375889fe9399edf1e15b0026713b2870cef079be Port: Host Port: State: Running Started: Tue, 28 Jun 2022 15:39:48 +0800 Last State: Terminated Reason: Error Exit Code: 137 Started: Tue, 28 Jun 2022 15:38:45 +0800 Finished: Tue, 28 Jun 2022 15:39:47 +0800 Ready: False Restart Count: 17 Requests: cpu: 250m Liveness: exec [/bin/calico-node -felix-live -bird-live] delay=10s timeout=1s period=10s #success=1 #failure=6 Readiness: exec [/bin/calico-node -felix-ready -bird-ready] delay=0s timeout=1s period=10s #success=1 #failure=3 Environment: DATASTORE_TYPE: kubernetes WAIT_FOR_DATASTORE: true NODENAME: (v1:spec.nodeName) CALICO_NETWORKING_BACKEND: <set to the key 'calico_backend' of config map 'calico-config'> Optional: false CLUSTER_TYPE: k8s,bgp IP: autodetect CALICO_IPV4POOL_IPIP: Always FELIX_IPINIPMTU: <set to the key 'veth_mtu' of config map 'calico-config'> Optional: false CALICO_DISABLE_FILE_LOGGING: true FELIX_DEFAULTENDPOINTTOHOSTACTION: ACCEPT FELIX_IPV6SUPPORT: false FELIX_LOGSEVERITYSCREEN: info FELIX_HEALTHENABLED: true Mounts: /lib/modules from lib-modules (ro) /run/xtables.lock from xtables-lock (rw) /var/lib/calico from var-lib-calico (rw) /var/run/calico from var-run-calico (rw) /var/run/nodeagent from policysync (rw) /var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-6rzvd (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: lib-modules: Type: HostPath (bare host directory volume) Path: /lib/modules HostPathType: var-run-calico: Type: HostPath (bare host directory volume) Path: /var/run/calico HostPathType: var-lib-calico: Type: HostPath (bare host directory volume) Path: /var/lib/calico HostPathType: xtables-lock: Type: HostPath (bare host directory volume) Path: /run/xtables.lock HostPathType: FileOrCreate cni-bin-dir: Type: HostPath (bare host directory volume) Path: /opt/cni/bin HostPathType: cni-net-dir: Type: HostPath (bare host directory volume) Path: /etc/cni/net.d HostPathType: host-local-net-dir: Type: HostPath (bare host directory volume) Path: /var/lib/cni/networks HostPathType: policysync: Type: HostPath (bare host directory volume) Path: /var/run/nodeagent HostPathType: DirectoryOrCreate flexvol-driver-host: Type: HostPath (bare host directory volume) Path: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds HostPathType: DirectoryOrCreate calico-node-token-6rzvd: Type: Secret (a volume populated by a Secret) SecretName: calico-node-token-6rzvd Optional: false QoS Class: Burstable Node-Selectors: kubernetes.io/os=linux Tolerations: :NoScheduleop=Exists :NoExecuteop=Exists CriticalAddonsOnly op=Exists node.kubernetes.io/disk-pressure:NoSchedule op=Exists node.kubernetes.io/memory-pressure:NoSchedule op=Exists node.kubernetes.io/network-unavailable:NoSchedule op=Exists node.kubernetes.io/not-ready:NoExecute op=Exists node.kubernetes.io/pid-pressure:NoSchedule op=Exists node.kubernetes.io/unreachable:NoExecute op=Exists node.kubernetes.io/unschedulable:NoSchedule op=Exists Events: Type Reason Age From Message

---

Normal Scheduled 49m default-scheduler Successfully assigned kube-system/calico-node-nhfkn to tx-node Normal Pulled 49m kubelet Container image "calico/cni:v3.13.1" already present on machine Normal Started 49m kubelet Started container upgrade-ipam Normal Created 49m kubelet Created container upgrade-ipam Normal Started 49m kubelet Started container install-cni Normal Pulled 49m kubelet Container image "calico/cni:v3.13.1" already present on machine Normal Created 49m kubelet Created container install-cni Normal Started 49m kubelet Started container flexvol-driver Normal Pulled 49m kubelet Container image "calico/pod2daemon-flexvol:v3.13.1" already present on machine Normal Created 49m kubelet Created container flexvol-driver Normal Pulled 49m kubelet Container image "calico/node:v3.13.1" already present on machine Normal Created 49m kubelet Created container calico-node Normal Started 49m kubelet Started container calico-node Warning Unhealthy 48m (x6 over 49m) kubelet Readiness probe failed: calico/node is not ready: felix is not ready: Get http://localhost:9099/readiness: dial tcp 127.0.0.1:9099: connect: connection refused Normal Killing 48m kubelet Container calico-node failed liveness probe, will be restarted Warning Unhealthy 14m (x80 over 49m) kubelet Liveness probe failed: calico/node is not ready: Felix is not live: Get http://localhost:9099/liveness: dial tcp 127.0.0.1:9099: connect: connection refused `

help me

[ihengshuai]

I didn‘t meet the problem,you can describe again