[FEATURE] serving-endpoint-id set to root to assign permissions to all endpoints

[spencerimp]

Use-cases

We are deploying our serving endpoints using a service principle for production.

Our model training pipeline will dynamically create a new version of mlflow model and create/update a serving endpoint using the latest model version.

Our developer group does not have the permissions for the resources created by our service principle.

In mlflow model, we assign permission to our developer group by setting the registered_model_id = "root" have a catch-all setup for all models and it worked well.

However, databrick_serving_endpoints does not support serving_endpoints_id = "root". It requires a specific endpoint id.

Attempted Solutions

I have tried to create a file with content like

resource "databricks_permissions" "endpoint_usage" { serving_endpoint_id = "root"

access_control { group_name = "DSML-GROUP" permission_level = "CAN_MANAGE" } }

I ran the Terraform plan and had error messages like

╷ │ Error: cannot create permissions: Cannot change permissions for group 'admins' to None. │ │ with databricks_permissions.endpoint_usage, │ on endpoint_usage.tf line 7, in resource "databricks_permissions" "endpoint_usage": │ 7: resource "databricks_permissions" "endpoint_usage" { │ ╵

Proposal

I traced the source code and this could be a solution.

From permissions/resource_permission.go, I can see something like

func (a PermissionsAPI) Update(objectID string, objectACL AccessControlChangeList) error {
	if objectID == "/authorization/tokens" || objectID == "/registered-models/root" || objectID == "/directories/0" 

Maybe we can add one more, so it becomes

func (a PermissionsAPI) Update(objectID string, objectACL AccessControlChangeList) error {
        if objectID == "/authorization/tokens" || objectID == "/registered-models/root" || objectID == "/directories/0" || objectID == "/serving-endpoints/root" 

By the way, I do not have prior experience with Go, and I could not make the unit tests pass locally even using the original master branch. When I ran

go test

Some test cases passed when some failed.

I am not sure how to test the solution, any idea?

References

[tanmay-db]

Hi @spencerimp, thanks for reaching out. We are prioritising other issues at the moment so won't be able to take this up currently. The change you proposed might have some further implications that we would like to check with the team. Just wanted to confirm, are you blocked currently on this one?

[spencerimp]

@tanmay-db Thanks!

We are blocked by this. Since your team is down-prioritizing this feature, our team will need to think of some workaround then.

Just out of curiosity, if I contact Databricks support via our company account, will it move this ticket forward (without messing up your team)?

[arpitjasa-db]

Hi @spencerimp this feature would be difficult to support without API support from the model serving Databricks team. Reaching out to Databricks support via company account can help escalate this feature and get it prioritized.