Prometheus cannot access Resources

Open christianwoehrle opened this issue 7 years ago • 1 comments

I tried Kubernetes Cluster Monitoring but the prometheus dashboard http://localhost:9090/targets didn't shown any data.

Checking the prometheus logs with kubectl logs prometheus-prometheus-0 prometheus -n monitoring shows a lot of erros like

level=error ts=2018-10-17T14:04:45.759210034Z caller=main.go:212 component=k8s_client_runtime err="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:178: Failed to list *v1.Service: services is forbidden: User \"system:serviceaccount:monitoring:prometheus-operator\" cannot list services in the namespace \"kube-system\""
level=error ts=2018-10-17T14:04:45.761409432Z caller=main.go:212 component=k8s_client_runtime err="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:177: Failed to list *v1.Endpoints: endpoints is forbidden: User \"system:serviceaccount:monitoring:prometheus-operator\" cannot list endpoints in the namespace \"kube-system\""

Looks like some verbs have to be added to the ClusterRole prometheus-operator. Anyone with the same error?

Oct 17 '18 14:10 christianwoehrle

@christianwoehrle I downloaded https://raw.githubusercontent.com/prometheus/prometheus/master/documentation/examples/rbac-setup.yml, globally replaced prometheus with prometheus-operator, and applied it, and /targets comes alive.

Oct 17 '18 18:10 geremyCohen