Unable to Enable OvDC for Enterprise CSE (PKS)

[mann1mal]

Following the directions from the CSE docs to create a PKS Service Account to use to enable an OvDC for Enterprise PKS:

uaac client add --name cse-admin --scope uaa.none \ 
--authorized_grant_types client_credentials \
--authorities clients.read,clients.write,clients.secret,scim.read,scim.write,pks.clusters.manage 

When using this service account in the pks-config.yaml, enabling the OVDC fails with the following error:

vcd cse ovdc enable pks-ovdc -o PKS-Org -k ent-pks --pks-plan "small" --pks-cluster-domain "pks.example.io"
Usage: vcd cse ovdc enable [OPTIONS] VDC_NAME
Try "vcd cse ovdc enable -h" for help.

Error: PKS error
 status: 403
 body: {"error":"insufficient_scope"

Looking at the cse-server-debug log, we see the following error string:

19-05-28 17:57:09 | pksbroker:589 - create_compute_profile | DEBUG :: Creating compute-profile cp--4c8702a8-24b7-4f4b-8d13-69162a2881eb--pks-ovdc in PKS failed with error:
 (403)
Reason: 
HTTP response headers: HTTPHeaderDict({'Cache-Control': 'no-store', 'Pragma': 'no-cache', 'WWW-Authenticate': 'Bearer error="insufficient_scope", error_description="Insufficient scope for this resource", scope="pks.clusters.admin"', 'X-Content-Type-Options': 'nosniff', 'X-XSS-Protection': '1; mode=block', 'Strict-Transport-Security': 'max-age=31536000 ; includeSubDomains', 'X-Frame-Options': 'DENY', 'Content-Type': 'application/json;charset=UTF-8', 'Transfer-Encoding': 'chunked', 'Date': 'Tue, 28 May 2019 21:57:09 GMT'})
HTTP response body: {"error":"insufficient_scope","error_description":"Insufficient scope for this resource","scope":"pks.clusters.admin"}

19-05-28 17:57:09 | utils:821 - exception_handler_wrapper | ERROR :: Traceback (most recent call last):
  File "/home/cse/environments/my_env/lib64/python3.6/site-packages/container_service_extension/pksbroker.py", line 587, in create_compute_profile
    profile_api.add_compute_profile(body=cp_request)
  File "/home/cse/environments/my_env/lib64/python3.6/site-packages/container_service_extension/pksclient/api/v1beta/profile_api.py", line 55, in add_compute_profile
    (data) = self.add_compute_profile_with_http_info(body, **kwargs)  # noqa: E501
  File "/home/cse/environments/my_env/lib64/python3.6/site-packages/container_service_extension/pksclient/api/v1beta/profile_api.py", line 133, in add_compute_profile_with_http_info
    collection_formats=collection_formats)
  File "/home/cse/environments/my_env/lib64/python3.6/site-packages/container_service_extension/pksclient/client/v1beta/api_client.py", line 322, in call_api
    _preload_content, _request_timeout)
  File "/home/cse/environments/my_env/lib64/python3.6/site-packages/container_service_extension/pksclient/client/v1beta/api_client.py", line 153, in __call_api
    _request_timeout=_request_timeout)
  File "/home/cse/environments/my_env/lib64/python3.6/site-packages/container_service_extension/pksclient/client/v1beta/api_client.py", line 365, in request
    body=body)
  File "/home/cse/environments/my_env/lib64/python3.6/site-packages/container_service_extension/pksclient/client/v1beta/rest.py", line 275, in POST
    body=body)
  File "/home/cse/environments/my_env/lib64/python3.6/site-packages/container_service_extension/pksclient/client/v1beta/rest.py", line 228, in request
    raise ApiException(http_resp=r)
container_service_extension.pksclient.client.v1beta.rest.ApiException: (403)
Reason: 
HTTP response headers: HTTPHeaderDict({'Cache-Control': 'no-store', 'Pragma': 'no-cache', 'WWW-Authenticate': 'Bearer error="insufficient_scope", error_description="Insufficient scope for this resource", scope="pks.clusters.admin"', 'X-Content-Type-Options': 'nosniff', 'X-XSS-Protection': '1; mode=block', 'Strict-Transport-Security': 'max-age=31536000 ; includeSubDomains', 'X-Frame-Options': 'DENY', 'Content-Type': 'application/json;charset=UTF-8', 'Transfer-Encoding': 'chunked', 'Date': 'Tue, 28 May 2019 21:57:09 GMT'})
HTTP response body: {"error":"insufficient_scope","error_description":"Insufficient scope for this resource","scope":"pks.clusters.admin"}


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cse/environments/my_env/lib64/python3.6/site-packages/container_service_extension/utils.py", line 817, in exception_handler_wrapper
    result = func(*args, **kwargs)
  File "/home/cse/environments/my_env/lib64/python3.6/site-packages/container_service_extension/broker_manager.py", line 112, in invoke
    self._create_pks_compute_profile(pks_ctx)
  File "/home/cse/environments/my_env/lib64/python3.6/site-packages/container_service_extension/broker_manager.py", line 536, in _create_pks_compute_profile
    raise ex
  File "/home/cse/environments/my_env/lib64/python3.6/site-packages/container_service_extension/broker_manager.py", line 530, in _create_pks_compute_profile
    pksbroker.create_compute_profile(**compute_profile_params)
  File "/home/cse/environments/my_env/lib64/python3.6/site-packages/container_service_extension/pksbroker.py", line 591, in create_compute_profile
    raise PksServerError(err.status, err.body)
container_service_extension.exceptions.PksServerError: PKS error
 status: 403
 body: {"error":"insufficient_scope","error_description":"Insufficient scope for this resource","scope":"pks.clusters.admin"}

Appears to be an issue with the permissions (or scope) on the service account I created according to the CSE documentation.

After using the default admin user created upon PKS install in the pks-config.yaml, I am able to enable the OvDC as expected:

$ vcd cse ovdc enable cse-ent-ovdc-1 -o cse-ent-org -k ent-pks -p "small" -d "pks.example.io"
metadataUpdate: Updating metadata for Virtual Datacenter pks-ovdc(4c8702a8-24b7-4f4b-8d13-69162a2881eb)

task: 3a6bf21b-93e9-44c9-af6d-635020957b21, Updated metadata for Virtual Datacenter pks-ovdc(4c8702a8-24b7-4f4b-8d13-69162a2881eb), result: success

Not sure if this is user error or doc bug, please let me know if you require any additional information, thank you!

[rocknes]

@mann1mal Is this issue still relevant?

[mann1mal]

@rocknes yes, the issue is still relevant. When following the directions in the CSE documentation to create a new service account to use with PKS, that user does not have sufficient scope to perform the required actions.

[sahithi]

Hi Joe,

Creation of compute profile requires the scope "pks.clusters.admin". This scope is missing from our documentation.

Please refer http://docs-pcf-staging.cfapps.io/pks/1-6/uaa-scopes.html for now until we fix our documentation.

Thanks Sahithi

[mann1mal]

Thanks Sahithi!