Additional permissions are needed to allow use with PodSecurityPolicies

[djschaap]

Commit 1bb67d1 (keel chart 0.9.6) modified the keel ServiceAccount to use a dedicated ClusterRole instead of cluster-admin. This ClusterRole, defined in chart/keel/templates/clusterrole.yaml, requires additional permissions.

To allow keel to monitor AWS ECR repositories, I had to grant create and update verbs on the secrets resource.

To allow keel to actually update the image in my Deployment, I had to grant get on the serviceaccounts and services resources.

  - apiGroups:
      - ""
    resources:
      - serviceaccounts
      - services
    verbs:
      - get

This behavior was seen under Kubernetes 1.20.2 with the latest keel image installed via chart 0.9.8, using the helm3 provider. I was updating a very simple Helm Release (deployment, service, serviceaccount templates) - additional permissions MAY be required if other resource types are included in the Release.

[djschaap]

Update after further testing: keel needs permission to get, create, update, delete, etc. EVERY object that is managed by a Helm chart. With Helm 2, Tiller was essentially cluster-admin, AIUI - for keel to operate properly, it may need a very similar level of elevated access.

Would reverting 1bb67d1 be feasible? Is there a smaller set of permissions that could be granted to the ServiceAccount that would still allow simple deployment and use of keel for "most" situations (where rbac/PSPs are in use)?

keel does do a very nice job of logging the error before it crashes:

time="2021-04-16T15:16:02Z" level=info msg="policy for release myns/da parsed: minor"
time="2021-04-16T15:16:02Z" level=info msg="preparing upgrade for da"
time="2021-04-16T15:16:02Z" level=info msg="reusing the old release's values"
time="2021-04-16T15:16:05Z" level=info msg="performing update for da"
time="2021-04-16T15:16:05Z" level=info msg="creating upgraded release for da"
time="2021-04-16T15:16:06Z" level=info msg="checking 7 resources for changes"
time="2021-04-16T15:16:06Z" level=info msg="Replaced \"da-dummyapp\" with kind PodSecurityPolicy for kind PodSecurityPolicy\n"
time="2021-04-16T15:16:06Z" level=info msg="Looks like there are no changes for ServiceAccount \"da-dummyapp\""
time="2021-04-16T15:16:06Z" level=info msg="Looks like there are no changes for Role \"da-dummyapp\""
time="2021-04-16T15:16:06Z" level=info msg="warning: Upgrade \"da\" failed: could not get information about the resource: rolebindings.rbac.authorization.k8s.io \"da-dummyapp\" is forbidden: User \"system:serviceaccount:kube-system:keel\" cannot get resource \"rolebindings\" in API group \"rbac.authorization.k8s.io\" in the namespace \"myns\""
time="2021-04-16T15:16:06Z" level=fatal msg="helm3: failed to update release from chart" error="could not get information about the resource: rolebindings.rbac.authorization.k8s.io \"da-dummyapp\" is forbidden: User \"system:serviceaccount:kube-system:keel\" cannot get resource \"rolebindings\" in API group \"rbac.authorization.k8s.io\" in the namespace \"myns\""

[joebowbeer]

Close? PSP's have been removed from k8s, IRSA is used now instead of secrets