azure account storage-credentials create, error

[evogelpohl]

trafficstars

Describe the issue

Using a jupyter notebook on my macbook pro to build a databricks-cli build script. Other databricks-cli commands work fine using both the DEFAULT and ACCOUNT profiles. This code returns 'Error: " <- no message.

%%bash

storage_cred_name="stgcredsdbxconnector"
azure_dbx_connector_id="/subscriptions/xxxxxxxf/resourceGroups/xxxxxxxx/providers/Microsoft.Databricks/accessConnectors/xxxxxxxxxx"
metastore_id="32f2xxxxxxx94"

json_payload=$(cat <<EOF
{
  "credential_info": {
    "name": "$storage_cred_name",
    "read_only": false,
    "skip_validation": false,
    "azure_managed_identity": {
      "access_connector_id": "$azure_dbx_connector_id"
    }
  }
}
EOF
)

databricks account storage-credentials create $metastore_id --json "$json_payload" -p ACCOUNT

Expected Behavior

Create an account level storage account that I can use to update the metastore to bind storage & the metastore. The metastore is present.

Actual Behavior

Returns "Error: " (no message follows)

Tried in Terminal as well. Same "Error"

OS and CLI version

databricks-cli installed new, today. OSX - current version

Debug Info (re-ran with --log-level debug)

time=2023-11-04T11:10:32.179-04:00 level=INFO source="root.go 55}" msg=start version=0.209.0 args="databricks, account, storage-credentials, create, xxxxxxxxxxca4294, --json, {\n  \"credential_info\": {\n    \"name\": \"stgcredsdbxconnector\",\n    \"comment\": \"metastore creds\",\n    \"read_only\": false,\n    \"azure_managed_identity\": {\n        \"access_connector_id\": \"/subscriptions/exxxxx737bff/resourceGroups/rg_xxxxxxxdatabricks/providers/Microsoft.Databricks/accessConnectors/dbx_msi_xxxxxxxxhl_01\"\n    },\n    \"skip_validation\": false\n  }\n}, -p, ACCOUNT, --log-level, debug"
time=2023-11-04T11:10:32.181-04:00 level=DEBUG source="config_file.go 100}" msg="Loading ACCOUNT profile from [/Users/ericvogelpohl/.databrickscfg](https://file+.vscode-resource.vscode-cdn.net/Users/xxxxxxxxx/.databrickscfg)" sdk=true
time=2023-11-04T11:10:32.516-04:00 level=INFO source="auth_azure_cli.go 129}" msg="Refreshed OAuth token for 2xxxxxxxxxxd from Azure CLI, which expires on 2023-11-04 11:35:04.000000" sdk=true
time=2023-11-04T11:10:32.752-04:00 level=INFO source="auth_azure_cli.go 129}" msg="Refreshed OAuth token for https://management.core.windows.net/ from Azure CLI, which expires on 2023-11-04 11:20:25.000000" sdk=true
time=2023-11-04T11:10:32.752-04:00 level=INFO source="auth_azure_cli.go 87}" msg="Using Azure CLI authentication with AAD tokens" sdk=true
time=2023-11-04T11:10:33.259-04:00 level=DEBUG source="client.go 272}" msg="non-retriable error: " sdk=true
time=2023-11-04T11:10:33.259-04:00 level=DEBUG source="client.go 383}" msg="POST [/api/2.0/accounts/783160d1-a726-4b8a-b6fc-1faef94e7352/metastores/32xxxxxxxxx294/storage-credentials](https://file+.vscode-resource.vscode-cdn.net/api/2.0/accounts/783160d1-a726-4b8a-b6fc-1faef94e7352/metastores/32f28af5-2e94-4c3f-a120-b2438cca4294/storage-credentials)\n> {\n>   \"credential_info\": {\n>     \"azure_managed_identity\": {\n>       \"access_connector_id\": \"/subscriptions/e02xxxxxx7bff/resourceGroups/rg_evogelpohl_databricks/prov... (65 more bytes)\"\n>     },\n>     \"comment\": \"metastore creds\",\n>     \"name\": \"stgcredsdbxconnector\",\n>     \"read_only\": false,\n>     \"skip_validation\": false\n>   }\n> }\n< HTTP/2.0 500 Internal Server Error\n< {\n<   \"details\": [\n<     {\n<       \"@type\": \"type.googleapis.com/google.rpc.RequestInfo\",\n<       \"request_id\": \"a7112be1-5766-4ef5-b93c-cc3872060918\",\n<       \"serving_data\": \"\"\n<     }\n<   ],\n<   \"error_code\": \"INTERNAL_ERROR\",\n<   \"message\": \"\"\n< }" sdk=true
Error: 
time=2023-11-04T11:10:33.259-04:00 level=ERROR source="root.go 114}" msg="failed execution" exit_code=1 error=""

[andrewnester]

@evogelpohl could you try to upgrade to the latest CLI (0.217.1) and run the command again?

[andrewnester]

Confirmed that the issue is not reproducible in 0.219.0 version