kube-arangodb icon indicating copy to clipboard operation
kube-arangodb copied to clipboard

arango-storage-operator failed to start

Open surajnarwade opened this issue 7 years ago • 6 comments

K8s environment

minikube

Output

kubectl get deployment
NAME                                                 DESIRED   CURRENT   READY     AGE
replicaset.apps/arango-storage-operator-85f654fc8b   2         2         0         12m

kubectl get pods
NAME                                           READY     STATUS             RESTARTS   AGE
pod/arango-storage-operator-85f654fc8b-6r5p8   0/1       CrashLoopBackOff   7          13m
pod/arango-storage-operator-85f654fc8b-qpqd9   0/1       CrashLoopBackOff   7          13m

surajnarwade avatar Aug 18 '18 11:08 surajnarwade

it seems that it's rolebinding issue, here are the logs,

kubectl logs pod/arango-storage-operator-85f654fc8b-qpqd9
2018-08-18T11:18:35Z |INFO| Starting arangodb-operator, version 0.2.2 build 0ebc340 operator-id=qpqd9 pod-name=arango-storage-operator-85f654fc8b-qpqd9 pod-namespace=kube-system
2018-08-18T11:18:35Z |INFO| attempting to acquire leader lease  kube-system/arango-storage-operator... component=glog operator-id=qpqd9
2018-08-18T11:18:35Z |INFO| New leader detected component=operator identity=arango-storage-operator-85f654fc8b-6r5p8-arango-storage-operator-85f654fc8b-6r5p8 lock-name=arango-storage-operator operator-id=qpqd9
2018-08-18T11:18:53Z |INFO| successfully acquired lease kube-system/arango-storage-operator component=glog operator-id=qpqd9
2018-08-18T11:18:53Z |INFO| New leader detected component=operator identity=arango-storage-operator-85f654fc8b-qpqd9-arango-storage-operator-85f654fc8b-qpqd9 lock-name=arango-storage-operator operator-id=qpqd9
2018-08-18T11:18:53Z |INFO| Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"arango-storage-operator", UID:"60a1d310-a2d4-11e8-8613-1cbeed35aea7", APIVersion:"v1", ResourceVersion:"7610", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' arango-storage-operator-85f654fc8b-qpqd9-arango-storage-operator-85f654fc8b-qpqd9 became leader operator-id=qpqd9
2018-08-18T11:18:53Z |INFO| Event(v1.ObjectReference{Kind:"Deployment", Namespace:"kube-system", Name:"arango-storage-operator", UID:"5f38b0be-a2d4-11e8-8613-1cbeed35aea7", APIVersion:"apps", ResourceVersion:"7559", FieldPath:""}): type: 'Normal' reason: 'Leader Election Won' Pod arango-storage-operator-85f654fc8b-qpqd9 is running as leader operator-id=qpqd9
2018-08-18T11:18:53Z |ERRO| Failed to update Pod wrt 'role' label component=operator error="pods \"arango-storage-operator-85f654fc8b-qpqd9\" is forbidden: User \"system:serviceaccount:kube-system:arango-storage-operator\" cannot update pods in the namespace \"kube-system\"" lock-name=arango-storage-operator operator-id=qpqd9 pod-name=arango-storage-operator-85f654fc8b-qpqd9
2018-08-18T11:18:53Z |ERRO| Cannot set leader role on Pod. Terminating process component=operator lock-name=arango-storage-operator operator-id=qpqd9

surajnarwade avatar Aug 18 '18 11:08 surajnarwade

This issue has been resolved in master.

ewoutp avatar Aug 20 '18 06:08 ewoutp

@ewoutp , I tried it from master only

surajnarwade avatar Aug 20 '18 08:08 surajnarwade

https://github.com/arangodb/kube-arangodb/blob/master/manifests/arango-storage.yaml#L31 This needs to add update verb to get it working :) if that is the case, shall I send PR for the same ?

surajnarwade avatar Aug 20 '18 08:08 surajnarwade

Sorry my mistake, the update is already in a branch, not yet in master.

No need for a PR, but thx anyway!

ewoutp avatar Aug 20 '18 10:08 ewoutp

If you build from master, the dev version of those manifest will contain the right permissions. The yaml you refer to belongs to 0.2.2.

ewoutp avatar Aug 20 '18 10:08 ewoutp