weave-gitops
weave-gitops copied to clipboard
weaveworks
Okta group impersonate limitation
Describe the bug
Looks like this configuration https://github.com/weaveworks/weave-gitops/tree/main/charts/gitops-server#impersonate doesn't work
rbac:
create: true
impersonationResources: ["groups"]
impersonationResourceNames: ["gitops-reader"]
I have configured login via an OIDC(Okta) provider. In Okta, group created and user added to this group. Okta Application and Authorization Server configured as well. Then I created ClusterRole and ClusterRoleBinding with subject as group name I created in Okta. Without rbac configuration block I mentioned above all works and I can see cluster data in UI, but after adding rbac configuration block UI is empty.
From Logs
2022-09-04T06:44:42.172Z DEBUG gitops.auth-server auth/jwt.go:155 Found principal {"user": "raman", "groups": ["wego-admin-rk"], "tokenLength": 0, "method": "*auth.JWTCookiePrincipalGetter"}
2022-09-04T06:44:42.172Z DEBUG gitops.auth-server auth/jwt.go:155 Found principal {"user": "raman", "groups": ["wego-admin-rk"], "tokenLength": 0, "method": "*auth.JWTCookiePrincipalGetter"}
2022-09-04T06:44:42.194Z ERROR gitops clustersmngr/factory.go:367 failed filtering namespaces {"cluster": "Default", "user": "raman", "error": "user namespace access: users \"raman\" is forbidden: User \"system:serviceaccount:flux-system:ww-gitops-weave-gitops\" cannot impersonate resource \"users\" in API group \"\" at the cluster scope"}
Environment
- Weave-Gitops Version - helm chart version - 3.0.3
- Flux Version - v0.31.2
- Kubernetes version - v1.21.14-eks-18ef993
To Reproduce Steps to reproduce the behavior:
- Configure Okta (User, Group, Application and Authorization Server)
- Create
ClusterRoleandClusterRoleBindingwith subject as group name created in Okta. - Add configuration below:
rbac:
create: true
impersonationResources: ["groups"]
impersonationResourceNames: ["some-group"]
Expected behavior
After adding rbac configuration block impersonate limitation works and in UI we can see cluster data.
Actual Behavior
After adding rbac configuration block with specific group in UI there is no cluster data.
impersonationResourceNames: []
does not throw the error, but still nothing is displayed in the UI
Two issues:
- impersonation for
groupsrequires access toimpersonateonusersresource at cluster scope level (overall bad idea) -
impersonationResourceNames: ["gitops-reader"]does not match group in logs"groups": ["wego-admin-rk"]
Following config requires access to impersonate on users resource at cluster scope level
rbac:
create: true
impersonationResources: ["groups"]
impersonationResourceNames: ["gitops-reader"]
With current configuration it fails with error:
{"cluster": "Default", "user": "raman", "error": "user namespace access: users \"raman\" is forbidden: User \"system:serviceaccount:flux-system:ww-gitops-weave-gitops\" cannot impersonate resource \"users\" in API group \"\" at the cluster scope"}
Here is a fix (adjust names/namespaces accordingly)
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: ww-gitops-weave-gitops-patch
rules:
- apiGroups:
- ""
resources:
- users
verbs:
- impersonate
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: ww-gitops-weave-gitops-patch
subjects:
- kind: ServiceAccount
name: ww-gitops-weave-gitops
namespace: flux-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ww-gitops-weave-gitops-patch
I have the same issue. Configured Okta successfully with Weave server but not seeing any data.
Tried the aforementioned solutions but without success
i am getting this error
2022-12-19T16:39:53.985Z ERROR gitops.core-server server/version.go:105 error getting flux version {"error": "error getting list of objects"} │
│ 2022-12-19T16:39:55.200Z INFO gitops.auth-server auth/jwt.go:77 attempt to read token from auth header │
│ 2022-12-19T16:39:55.201Z INFO gitops.auth-server auth/jwt.go:77 attempt to read token from auth header
and "No data" into Weave UI
versions: Kubernetes: v1.24.7-eks-fb459a0 Flux: 0.33 Weave GitOps: v0.12.0
rbac config:
rbac:
create: true
impersonationResourceNames: ["admin", "AdminGroup"]
impersonationResources: ["users", "groups"]
viewSecretsResourceNames: ["cluster-user-auth", "oidc-auth"]
cluster role and binding
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: wego-admin-cluster-role-okta
rules:
- apiGroups: [""]
resources: ["secrets", "pods" ]
verbs: [ "get", "list" ]
- apiGroups: ["apps"]
resources: [ "deployments", "replicasets"]
verbs: [ "get", "list" ]
- apiGroups: ["kustomize.toolkit.fluxcd.io"]
resources: [ "kustomizations" ]
verbs: [ "get", "list", "patch" ]
- apiGroups: ["helm.toolkit.fluxcd.io"]
resources: [ "helmreleases" ]
verbs: [ "get", "list", "patch" ]
- apiGroups: ["source.toolkit.fluxcd.io"]
resources: [ "buckets", "helmcharts", "gitrepositories", "helmrepositories", "ocirepositories" ]
verbs: [ "get", "list", "patch" ]
- apiGroups: [""]
resources: ["events"]
verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: wego-cluster-admin-okta
subjects:
- kind: Group
name: AdminGroup
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: wego-admin-cluster-role-okta
apiGroup: rbac.authorization.k8s.io
also added and this "patch" cluster role and binding
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: flux-webui-weave-gitops-patch
rules:
- apiGroups:
- ""
resources:
- users
- groups
verbs:
- impersonate
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: flux-webui-weave-gitops-patch
subjects:
- kind: ServiceAccount
name: flux-webui-weave-gitops
namespace: flux-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: flux-webui-weave-gitops-patch
@JamWils do you know if anyone is working on it ? @LukaszRacon did you tested it with K8s version 1.24?
Bare in mind that in k8s version the SA flux-webui-weave-gitops does not have token cuz the token has to be created manually
│ Name: flux-webui-weave-gitops │
│ Namespace: flux-system │
│ Labels: app.kubernetes.io/instance=flux-webui │
│ app.kubernetes.io/managed-by=Helm │
│ app.kubernetes.io/name=weave-gitops │
│ app.kubernetes.io/version=v0.12.0 │
│ helm.sh/chart=weave-gitops-4.0.9 │
│ helm.toolkit.fluxcd.io/name=flux-webui │
│ helm.toolkit.fluxcd.io/namespace=flux-system │
│ Annotations: meta.helm.sh/release-name: flux-webui │
│ meta.helm.sh/release-namespace: flux-system │
│ Image pull secrets: <none> │
│ Mountable secrets: <none> │
│ Tokens: flux-webui-weave-gitops-token │
│ Events: <none> │
│
for example:
---
apiVersion: v1
kind: Secret
metadata:
name: flux-webui-weave-gitops-token
namespace: flux-system
annotations:
kubernetes.io/service-account.name: "flux-webui-weave-gitops"
type: kubernetes.io/flux-webui-weave-gitops-token
I am running a custom build that includes PR #2745 - I am using AzureAD with AKS 1.24.6
Hello. The same issue here - no data in UI when trying to restrict access based on groups in Azure (it shouldn't really matter as OAuth2/OIDC is more or less similarly implemented in any Identity Provider). I had applied all the necessary fixes:
- additional
ClusterRoleandClusterRoleBindingto impersonate all the users - as suggested by @grglzrv - added cli not to request
groupsscope as AzureAD doesn't support it:additionalArgs: - --auth-methods=oidc - --oidc-client-id={{ .clientID }} - --oidc-client-secret={{ .clientSecret }} - --oidc-redirect-url={{ .redirectURL }} - --oidc-issuer-url={{ .issuerURL }} - --custom-oidc-scopes=openid,profile,email
but ended up with the kind of filtering error when the app throws an error on the first group and stops processing the rest, e.g. my user belongs to 5 groups:
2023-02-13T20:44:05.587Z DEBUG gitops middleware/middleware.go:57 request success {"uri": "/v1/objects", "status": 200}
2023-02-13T20:44:06.213Z DEBUG gitops middleware/middleware.go:57 request success {"uri": "/", "status": 200}
2023-02-13T20:44:06.213Z DEBUG gitops middleware/middleware.go:57 request success {"uri": "/", "status": 200}
2023-02-13T20:44:10.690Z INFO gitops.auth-server auth/jwt.go:77 attempt to read token from auth header
2023-02-13T20:44:10.690Z DEBUG gitops.auth-server auth/jwt.go:54 parsing cookie JWT token {"claimsConfig": {"Username":"email","Groups":"groups"}}
2023-02-13T20:44:10.690Z DEBUG gitops.auth-server auth/jwt.go:161 Found principal {"user": "[email protected]", "groups": ["1111-0000-2222-...", "3333-0000-4444-...", "5555-MyTarget-Group-UUID" ], "tokenLength": 0, "method": "*auth.JWTCookiePrincipalGetter"}
2023-02-13T20:44:10.988Z INFO gitops.auth-server auth/jwt.go:77 attempt to read token from auth header
2023-02-13T20:44:10.988Z DEBUG gitops.auth-server auth/jwt.go:54 parsing cookie JWT token {"claimsConfig": {"Username":"email","Groups":"groups"}}
2023-02-13T20:44:10.989Z DEBUG gitops.auth-server auth/jwt.go:161 Found principal ...omitted for brevity
2023-02-13T20:44:11.089Z ERROR gitops clustersmngr/factory.go:566 failed filtering namespaces {"cluster": "Default", "user": "[email protected]", "error": "user namespace access: groups \"1111-0000-2222-...\" is forbidden: User \"system:serviceaccount:flux-system:ww-gitops\" cannot impersonate resource \"groups\" in API group \"\" at the cluster scope"}
Also trying to filter by user throws exactly the same error (and a work around is to create another extra ClusterRole again), so I would say a way to build a final RBAC manifest from Helm chart could be improved.
@grglzrv did you find any solution for the issue ?
Unfortunately no, I gonna try again this week.
@aboutobza-talend you may try this HelmRelease - oidcSecret : https://github.com/FrankelJb/home-ops/blob/main/kubernetes/apps/flux-system/weave-gitops/helmrelease.yaml#L34-L39 and impersonationResourceNames: https://github.com/FrankelJb/home-ops/blob/main/kubernetes/apps/flux-system/weave-gitops/helmrelease.yaml#L60-L62
- additional
ClusterRoleandClusterRoleBindingto impersonate all the users - as suggested by @grglzrv- added cli not to request
groupsscope as AzureAD doesn't support it:additionalArgs: - --auth-methods=oidc - --oidc-client-id={{ .clientID }} - --oidc-client-secret={{ .clientSecret }} - --oidc-redirect-url={{ .redirectURL }} - --oidc-issuer-url={{ .issuerURL }} - --custom-oidc-scopes=openid,profile,email
AzureAD does support groups. Go to the app registration -> Token configuration -> Add groups claim
