Document required cluster permissions to create CRDs

[ckadner]

trafficstars

Describe the bug

When trying to "Launch" a Dataset which attempts to create a PVC, users may get a 403 error, since, by default Kubeflow cannot deploy any CRD resource on the cluster.

To Reproduce

Steps to reproduce the behavior:

1. Go to 'Datasets' and select a Dataset
2. Select the "Launch" tab
3. Fill in a "Namespace", i.e. kubeflow or mlx
4. Click "Submit"

Expected behavior

The Kubeflow Pipeline Run graph should appear.

Required fix for the user

• the user / admin need to patch the cluster:

  kubectl create clusterrolebinding pipeline-runner-extend --clusterrole cluster-admin --serviceaccount=kubeflow:pipeline-runner
  

Required changes in MLX

Either ...

1. Add the clusterrolebinding to all manifests (multi-user excluded?)
2. Add it to each of the MLX deployment READMEs

Also add troubleshooting steps on the Dataset README

@yhwang @Tomcli

[yhwang]

you would see log messages similar to this:

kubernetes.client.rest.ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': '29fea5a9-ee25-40ef-a65b-e67858904c70', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd88dd123-8da1-4d61-9dd7-37f7dd5f43dd', 'Date': 'Tue, 30 Nov 2021 17:31:33 GMT', 'Content-Length': '364'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"datasets.com.ie.ibm.hpsys is forbidden: User \"system:serviceaccount:kubeflow:pipeline-runner\" cannot list resource \"datasets\" in API group \"com.ie.ibm.hpsys\" in the namespace \"kubeflow\"","reason":"Forbidden","details":{"group":"com.ie.ibm.hpsys","kind":"datasets"},"code":403}