Accessing config/secret values

[guruvonline]

is there any option to access config/secret values from C#? i need to access keys like EventHub connection string, Application Insights keys etc.

i can create secret scope in Databricks and access in my python script as

'dbutils.secrets.get(scope = "", key = "")'

how to do similar thing in C#

[imback82]

Can you please elaborate what you are trying to do?

[guruvonline]

for my c# app i have to keep few config secrets like EventHub connection string(s), Application Insights connection string, connection string for my Configuration store. What is the recommended approach of accessing multiple configuration/Secret values in C#?

as per azure databricks documentation this can be done in scala/python using secret scopes. Is there any recommendation/suggestion of doing it in C#?

[rapoth]

@guruvonline: Have you tried Azure KeyVault?

[suhsteve]

@guruvonline A couple guides that may help. Set and retrieve a secret - .NET and Get Key Vault Secret via REST API

[rapoth]

@guruvonline: Did you get this to work with the guides above?

[guruvonline]

I am using Azure Databricks , which doesn't support managed identity to connect to KeyVault. so i have to keep secret in my code to access Keyvault. For C# app to work to connect to keyvault, i have to keep secret to connect to keyvault in some file on dbfs. So secret to access my secret is not secure.

in pyspark as part of one time cluster configuration, i can create secrete scope, and in my pyspark code i can directly read keyvault values.

is there any recommendation on how to keep secret to access secrestore(keyvault) from c# .net for spark?

[guruvonline]

closed by mistake

[ksiomelo]

+1 Please advise on this issue.

[ravisgoli]

+1

[rapoth]

Unfortunately, dbutils is closed source and there is not much we can do to add support (we are open to contributions, however!). This may change if Databricks decides to open source dbutils or adds support themselves. Have you considered making a feature request to Databricks?

[dbeavon]

+1 here too. I'm ok with using keyvault for secrets but the problem is how to manage the bootstrapping secrets that help you connect to keyvault in the first place.

[dbeavon]

This has been submitted with Databricks.

The Databricks issue tracking (in SF) case number is 00101202

We should get an update from them. I will update this issue once I have more information.

[dbeavon]

This is an update from Databricks Team following the SF number: 00101202

“We can treat this as a feature request I have already created for this. Here is the request number - DB-I-3505”