Adding secret to Databricks scope doesn't work in Standard Tier

[NowinskiK]

When I tried to add new secret to newly created databricks scope:

Add-DatabricksSecretScope -BearerToken $databricksToken -Region $resourceGroupLocation -ScopeName 'def' -AllUserAccess:$true
Set-DatabricksSecret -BearerToken $databricksToken -Region $resourceGroupLocation -ScopeName 'def' -SecretName 'password' -SecretValue 'some-secret-value'

I've got the following error message, even though "AllUsersAccess" has been declared above.

#{"error_code":"BAD_REQUEST","message":"Premium Tier is disabled in this workspace. Secret scopes can only be created with initial_manage_principal "users"."}

The same works with Databricks CLI.

databricks secrets create-scope --scope xyz --initial-manage-principal users --profile kn
databricks secrets put --scope xyz --key password --profile kn

[simondmorias]

As bizarre as this sounds, for now I think you can just remove that parameter and it will default to all users on the Databricks side as it is a Standard workspace. It's a odd way the API has been implemented, for premium it defaults differently. I will add a check to default it correctly.

[NowinskiK]

Checked. If parameter -AllUserAccess:$true is not provided - I've got the same error.

[simondmorias]

So if you use -AllUserAccess:$True or -AllUserAccess it works fine. The code does nothing to evaluate $true - so why lowercase doesn't work I do not know. Possibly something to do with PS Core as that wouldn't work on linux.

[AlaQabaja]

I am still getting the same error even after setting -AllUserAccess:$True or -AllUserAccess. Can someone please help with this? The error I am getting:

Invoke-RestMethod : {"error_code":"BAD_REQUEST","message":"Premium Tier is disabled in this workspace. Secret scopes can 
only be created with initial_manage_principal \"users\"."}

And this works fine if I use CLI as indicated above.

[simondmorias]

@AlaQabaja what version of the tools and what version of PowerShell and OS are you on?

[PuskasJ]

Hi

I have the same issue as the guys before me: I have a "standard" tier Azure Databricks and i want to add a secret scope and some secrets under that secret scope

Add-DatabricksSecretScope -BearerToken $dataBricksToken -Region $dataBricksRegion -ScopeName 'secret-scope' -AllUserAccess

Set-DatabricksSecret -BearerToken $dataBricksToken -Region $dataBricksRegion -ScopeName 'secret-scope' -SecretName 'adlsName' -SecretValue $adlsName
Set-DatabricksSecret -BearerToken $dataBricksToken -Region $dataBricksRegion -ScopeName 'secret-scope' -SecretName 'adlsKey' -SecretValue $adlsKey

The error is the same:

2019-10-23T09:38:57.6052968Z Invoke-RestMethod : {"error_code":"BAD_REQUEST","message":"Premium Tier is disabled in this workspace. Secret scopes 2019-10-23T09:38:57.6053738Z can only be created with initial_manage_principal "users"."} 2019-10-23T09:38:57.6053866Z At C:\Program 2019-10-23T09:38:57.6053963Z Files\WindowsPowerShell\Modules\azure.databricks.cicd.tools\2.0.43\Public\Add-DatabricksSecretScope.ps1:74 char:9 2019-10-23T09:38:57.6054090Z + Invoke-RestMethod -Method Post -Body $BodyText -Uri "$global: ... 2019-10-23T09:38:57.6054509Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2019-10-23T09:38:57.6054689Z + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebExc 2019-10-23T09:38:57.6054794Z eption 2019-10-23T09:38:57.6054888Z + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

if i upgrade Databricks to premium tier the script works fine.

I guess this might be a Databricks Secrets API issue, but is is worth checking the code of this Powershell module too.

Thanks!

[PuskasJ]

After some further investigation I think the issue is somewhere in this powershell module. Probably the $body['initial_manage_principal'] = "users" is not set correctly in the /Public/Add-DatabricksSecretScope.ps1 file.

First i've checked the Databricks Secrets API using Postman and it worked fine, than i replaced my code (available in the previous post) with the following code: (mainly copied from the ps files of this module)

$SecretScope = 'secret-scope'
$DatabricksURI = "https://$dataBricksRegion.azuredatabricks.net"

#Add-DatabricksSecretScope -BearerToken $dataBricksToken -Region $dataBricksRegion -ScopeName 'secret-scope' -AllUserAccess

$body = @{}
$body['scope'] = $SecretScope
$body['initial_manage_principal'] = "users"
$BodyText = $Body | ConvertTo-Json -Depth 10
$headers = @{
    Authorization="Bearer $dataBricksToken"
}
Try
{
    Invoke-RestMethod -Method Post -Body $BodyText -Uri "$DatabricksURI/api/2.0/secrets/scopes/create" -Headers $Headers
    Write-Verbose "Secret Scope $ScopeName created"
}
 Catch
 {
     $err = $_.ErrorDetails.Message
     if ($err.Contains('already exists'))
     {
         Write-Verbose $err
     }
     else
     {
         throw
     }
 }

#Set-DatabricksSecret -BearerToken $dataBricksToken -Region $dataBricksRegion -ScopeName 'secret-scope' -SecretName 'adlsName' -SecretValue $adlsName
$body = '{ "scope": "' + $SecretScope + '", "key": "' + 'adlsName' + '", "string_value": "' + $adlsName + '"}'
Invoke-RestMethod -Method Post -Body $body -Uri "$global:DatabricksURI/api/2.0/secrets/put" -Headers $Headers

#Set-DatabricksSecret -BearerToken $dataBricksToken -Region $dataBricksRegion -ScopeName 'secret-scope' -SecretName 'adlsKey' -SecretValue $adlsKey
$body = '{ "scope": "' + $SecretScope + '", "key": "' + 'adlsKey' + '", "string_value": "' + $adlsKey + '"}'
Invoke-RestMethod -Method Post -Body $body -Uri "$global:DatabricksURI/api/2.0/secrets/put" -Headers $Headers

[simondmorias]

I'm unable to reproduce this issue still. I have created a Standard workspace and tested the module with the latest Windows PowerShell and PowerShell Core. Both worked as expected (the -AllUserAccess caused initial_manage_principal = users to be set. Can you confirm which version of PowerShell and this module you are running?

$PSVersionTable
Get-Module azure.databricks.cicd.tools -ListAvailable

[PuskasJ]

This is on my machine, but the script should run on Azure DevOps CI/CD (bot on my machine and on DevOps i get the same error)

$PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.17763.771
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17763.771
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1


Get-Module azure.databricks.cicd.tools -ListAvailable


    Directory: C:\Program Files\WindowsPowerShell\Modules


ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Script     2.0.43     azure.databricks.cicd.tools         {Add-DatabricksSecretScope, Export-DatabricksFolder, Impor...