etcd-operator icon indicating copy to clipboard operation
etcd-operator copied to clipboard

Init container hangs indefinitely

Open jicowan opened this issue 5 years ago • 5 comments

After applying the manifest for the example cluster [a 3 node etcd cluster] the init container hangs indefinitely. The last message skip reconciliation: running ([]), pending ([example-etcd-cluster-vnjpsbdfmn])" cluster-name=example-etcd-cluster cluster-namespace=default pkg=cluster keeps repeating. When I look at the logs for the example-etcd-cluster-vnjpsbdfmn pod it says, Error from server (BadRequest): container "etcd" in pod "example-etcd-cluster-vnjpsbdfmn" is waiting to start: PodInitializing. I see no other logs that indicate what the issue might be.

time="2019-04-14T23:21:54Z" level=info msg="creating cluster with Spec:" cluster-name=example-etcd-cluster cluster-namespace=default pkg=cluster
time="2019-04-14T23:21:54Z" level=info msg="{" cluster-name=example-etcd-cluster cluster-namespace=default pkg=cluster
time="2019-04-14T23:21:54Z" level=info msg="    \"size\": 3," cluster-name=example-etcd-cluster cluster-namespace=default pkg=cluster
time="2019-04-14T23:21:54Z" level=info msg="    \"repository\": \"quay.io/coreos/etcd\"," cluster-name=example-etcd-cluster cluster-namespace=default pkg=cluster
time="2019-04-14T23:21:54Z" level=info msg="    \"version\": \"3.2.13\"" cluster-name=example-etcd-cluster cluster-namespace=default pkg=cluster
time="2019-04-14T23:21:54Z" level=info msg="}" cluster-name=example-etcd-cluster cluster-namespace=default pkg=cluster
time="2019-04-14T23:21:54Z" level=info msg="cluster created with seed member (example-etcd-cluster-vnjpsbdfmn)" cluster-name=example-etcd-cluster cluster-namespace=default pkg=cluster
time="2019-04-14T23:21:54Z" level=info msg="start running..." cluster-name=example-etcd-cluster cluster-namespace=default pkg=cluster
time="2019-04-14T23:22:02Z" level=info msg="skip reconciliation: running ([]), pending ([example-etcd-cluster-vnjpsbdfmn])" cluster-name=example-etcd-cluster cluster-namespace=default pkg=cluster

jicowan avatar Apr 14 '19 23:04 jicowan

@jicowan same here. Did you find a solution? Thanks.

brunowego avatar Apr 22 '19 17:04 brunowego

@brunowego Not, yet.

jicowan avatar Apr 22 '19 19:04 jicowan

I've got the same issue

NickCarton avatar May 13 '19 05:05 NickCarton

After change from flannel network to calico, this not happen more. Try switch network.

brunowego avatar May 13 '19 13:05 brunowego

please investigate events in kubectl cluster, especially from etcd pods, there should be an info why pod is still in initializing state. Usually it's related to insufficient resources (too high cpu/memory requests per pod), or incorrectly configured storage (for example pod in in zone A while PV was created in zone B, thus you should create new storageclass with volumeBindingMode: WaitForFirstConsumer).

nvtkaszpir avatar May 14 '19 13:05 nvtkaszpir