Can not create read only external locations [Azure]

[kfollesdal]

Describe the issue

Have

• Databricks with Unity Catalog in Azure
• Have a Azure Storage Account with container

Try to create a read_only external location with a Azure storage account. With commando:

databricks external-locations create demo abfss://[email protected]/ my-storage-cred --read-only

but fails with following error.

Error: The contributor role on the storage account is not set or Managed Identity does not have READ permissions on url abfss://[email protected]/. Please contact your account admin to update the storage credential. No file available under the location to read.

NOTE

If I remove the --read-only flag everything works fine and I can create the external location.

Have tried with different variations of role assignment on storage account with Storage Blob Data Reader and Storage Blob Data Contributor. And with and without read_only storage credential, but get same error as long as i add --read-only flag to databricks external-locations command.

Steps to reproduce the behavior

1. Create a Azure storage account with a container
2. Access Connector for Azure Databricks
3. Give Access Connector for Azure Databricks the role Storage Blob Data Reader. Have also tried with Storage Blob Data Contributor
4. Create a Databricks Storage Credential. have tried with and without read only option

databricks external-locations create demo abfss://[email protected]/ my-storage-cred --read-only

Expected Behavior

Create a read only external location

Actual Behavior

Error: The contributor role on the storage account is not set or Managed Identity does not have READ permissions on url abfss://[email protected]/. Please contact your account admin to update the storage credential. No file available under the location to read.

OS and CLI version

Databricks CLI v0.210.2 NixOS 23.11

Is this a regression?

Have not tested other version of CLI. But have tested on API level and fails with same error.

Debug Logs

databricks external-locations create demo abfss://[email protected]/ my-storage-cred --read-only --log-level=debug
17:05:36  INFO start pid=124176 version=0.210.2 args="databricks, external-locations, create, demo, abfss://[email protected]/, my-storage-cred, --read-only, --log-level=debug"
17:05:36 DEBUG Loading DEFAULT profile from /home/kristoffer/.databrickscfg pid=124176 sdk=true
17:05:36  INFO Generating AAD token for Service Principal (XXX) pid=124176 sdk=true
17:05:37 DEBUG non-retriable error: The contributor role on the storage account is not set or Managed Identity does not have READ permissions on url abfss://[email protected]/. Please contact your account admin to update the storage credential. No file available under the location to read. pid=124176 sdk=true
17:05:37 DEBUG POST /api/2.1/unity-catalog/external-locations
> {
>   "credential_name": "my-storage-cred",
>   "name": "demo",
>   "read_only": true,
>   "url": "abfss://[email protected]/
> }
< HTTP/2.0 403 Forbidden
< {
<   "details": [
<     {
<       "@type": "type.googleapis.com/google.rpc.RequestInfo",
<       "request_id": "ab379720-5f09-4867-81aa-c3539b05c7a9",
<       "serving_data": ""
<     }
<   ],
<   "error_code": "PERMISSION_DENIED",
<   "message": "The contributor role on the storage account is not set or Managed Identity does not have READ pe... (183 more bytes)"
< } pid=124176 sdk=true
Error: The contributor role on the storage account is not set or Managed Identity does not have READ permissions on url abfss://[email protected]/. Please contact your account admin to update the storage credential. No file available under the location to read.
17:05:37 ERROR failed execution pid=124176 exit_code=1 error="The contributor role on the storage account is not set or Managed Identity does not have READ permissions on url abfss://[email protected]/. Please contact your account admin to update the storage credential. No file available under the location to read."