Provide kubernetes role for RBAC

[vdboor]

As of Kubernetes 1.8+, RBAC is enabled by default. You'll need the following setup to allow the program to access the events:

kubectl create sa sentry-kubernetes
kubectl create clusterrole sentry-kubernetes --verb=get,list,watch --resource=events
kubectl create clusterrolebinding sentry-kubernetes --clusterrole=sentry-kubernetes --user=sentry-kubernetes

kubectl run sentry-kubernetes \
  --image bretthoerner/sentry-kubernetes \
  --serviceaccount=sentry-kubernetes \
  --env="DSN=$YOUR_DSN"

When you add --dry-run -o yaml to all commands, you'll get the .yml definition files.

[ghost]

🤔 Still running into RBAC issues after deploying the RBAC roles.

2018-01-24 14:29:13,006 Exception when calling CoreV1Api->list_event_for_all_namespaces: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'Date': 'Wed, 24 Jan 2018 14:29:13 GMT', 'Content-Length': '326'})
HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"events is forbidden: User \\"system:serviceaccount:default:sentry-kubernetes\\" cannot watch events at the cluster scope: Unknown user \\"system:serviceaccount:default:sentry-kubernetes\\"","reason":"Forbidden","details":{"kind":"events"},"code":403}\n'

The serviceaccount was created successfully.

➜  kubectl get sa
NAME                SECRETS   AGE
<..snip..>
sentry-kubernetes   1         4m
<..snip..>

[bretthoerner]

If you -o yaml the sentry-kubernetes what is the output?

[ghost]

› kubectl get deployment sentry-kubernetes -o yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: 2018-01-24T14:27:44Z
  generation: 1
  labels:
    run: sentry-kubernetes
  name: sentry-kubernetes
  namespace: default
  resourceVersion: "580395"
  selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/sentry-kubernetes
  uid: bddc1622-0112-11e8-b7ed-42010a840009
spec:
  replicas: 1
  selector:
    matchLabels:
      run: sentry-kubernetes
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        run: sentry-kubernetes
    spec:
      containers:
      - env:
        - name: DSN
          value: <..snip..>
        image: bretthoerner/sentry-kubernetes
        imagePullPolicy: IfNotPresent
        name: sentry-kubernetes
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: sentry-kubernetes
      serviceAccountName: sentry-kubernetes
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: 2018-01-24T14:27:44Z
    lastUpdateTime: 2018-01-24T14:27:44Z
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  observedGeneration: 1
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

kubectl get sa sentry-kubernetes -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  creationTimestamp: 2018-01-24T14:26:52Z
  name: sentry-kubernetes
  namespace: default
  resourceVersion: "580250"
  selfLink: /api/v1/namespaces/default/serviceaccounts/sentry-kubernetes
  uid: 9f5877b0-0112-11e8-9431-42010a840008
secrets:
- name: sentry-kubernetes-token-nq2f9

[bretthoerner]

Hmm, I guess neither of those list permissions?

I was hoping to verify the --verb=get,list,watch --resource=events part of the role worked.

[ghost]

I've manually verified, and role has the verbs and resource attached, but for some reason the container is not picking it up correctly, or it's not the correct role.

[vdboor]

there you go:

This is what I have in my cluster (extracted with kubectl get -o yaml and cleaned up)

apiVersion: v1
kind: ServiceAccount
metadata:
  name: sentry-kubernetes
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: sentry-kubernetes
rules:
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: sentry-kubernetes
roleRef:
  kind: ClusterRole
  name: sentry-kubernetes
subjects:
- kind: ServiceAccount
  name: sentry-kubernetes

[bretthoerner]

Thanks!

@Chnkr Can you diff with yours?

[vdboor]

It looks like a second set of verbs for the "secrets" resource was missing in my first command line example. That's something I probably found later. Most of it was discovered by analyzing the container logs.

[ghost]

That seems to be the case indeed! Going to give it a go and see if this works 👍

[ghost]

🎉 It seems to be working after making some minor changes to the ClusterRoleBinding

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: sentry-kubernetes
  namespace: default
roleRef:
  kind: ClusterRole
  name: sentry-kubernetes
  apiGroup: rbac.authorization.k8s.io  
subjects:
- kind: ServiceAccount
  name: sentry-kubernetes
  namespace: default  

[bretthoerner]

Awesome, does anyone know what (if any) adjustments need to be made to the original commands @vdboor posted?

kubectl create sa sentry-kubernetes
kubectl create clusterrole sentry-kubernetes --verb=get,list,watch --resource=events
kubectl create clusterrolebinding sentry-kubernetes --clusterrole=sentry-kubernetes --user=sentry-kubernetes

kubectl run sentry-kubernetes \
  --image bretthoerner/sentry-kubernetes \
  --serviceaccount=sentry-kubernetes \
  --env="DSN=$YOUR_DSN"

I'd like to put it in the README if it makes sense.

[vdboor]

One option would be to provide .yml files, and use kubectl apply -f <url to github yml file>.

A second option is to fix the helm chart PR: https://github.com/kubernetes/charts/pull/2708 which gives even easier installation options.

[gianrubio]

I did a new PR using the commits from #2708, https://github.com/kubernetes/charts/pull/3748

[wichert]

FWIW instead of using a cluster role you can also use a normal role and pass in the EVENT_NAMESPACES environment variable to limit monitoring to specific namespaces.

[Fleuri]

Sorry for necroing an old thread, but I'm getting Exception when calling CoreV1Api->list_event_for_all_namespaces: (403) using GKE. Is there some managed cluster quirk I'm missing? If I change ClusterRoleBinding -> subjects -> Kind to ServiceAccount instead of User, I get the same exception but with code 401 Unauthorized.

[shovalaharoni99]

Still running into RBAC issues after deploying the RBAC roles.

2018-01-24 14:29:13,006 Exception when calling CoreV1Api->list_event_for_all_namespaces: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'Date': 'Wed, 24 Jan 2018 14:29:13 GMT', 'Content-Length': '326'})
HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"events is forbidden: User \\"system:serviceaccount:default:sentry-kubernetes\\" cannot watch events at the cluster scope: Unknown user \\"system:serviceaccount:default:sentry-kubernetes\\"","reason":"Forbidden","details":{"kind":"events"},"code":403}\n'

The serviceaccount was created successfully.

➜  kubectl get sa
NAME                SECRETS   AGE
<..snip..>
sentry-kubernetes   1         4m
<..snip..>

I also encountered this problem, did someone managed to solve it ?

[Frankkkkk]

Take a look at the helm chart https://github.com/sentry-kubernetes/charts/tree/develop/sentry-kubernetes/templates it's not that complicated: you only need a clusterrole + clusterrolebinding. Cheers

[shovalaharoni99]

ן already did that, and it still not working. I don't understand why I'm getting this error. @Frankkkkk