ProtectKeysWithAzureKeyVault deserves more explanation

[Pyrobolser]

Hello,

Going from development on localhost where everything is simple to Linux App Services, suddenly nothing is working anymore because of this whole "data protection" issue.

I am trying to follow the documentation and understand I need a Blob Storage Account and a Azure Key Vault, I setup all of those, try to generate the <blobUriWithSasToken> but apparently it needs a reference to a "blob". I go from issues to issues until I get a Root element is missing.
Now I have an empty key.xml as a blob and I don't understand what the application actually wants.
Is there an example somewhere that we can follow, the documentation is a little bit light on this side when you don't know all this.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: b63ab082-ad30-b00d-ecd1-ca88eee1659e
• Version Independent ID: 69818122-e47a-3fed-ea37-8009f66e2a5c
• Content: Configure ASP.NET Core Data Protection
• Content Source: aspnetcore/security/data-protection/configuration/overview.md
• Product: aspnet-core
• Technology: aspnetcore-security
• GitHub Login: @Rick-Anderson
• Microsoft Alias: riande

[akempe]

Couldn't agree more. The documentation is woefully inadequate in explaining:

• how to setup the key ring
• how to setup a decent level of restrictive permissions required to the blob storage
• links to articles describing setting up service-to-service permissions / sas tokens effectively and safely
  • the entire sas token vs services-to-service permissions/roles/iam/policies thing is a mess of documentation spread over far too many articles. it's WAY TOO easy to give too much access in the process of trying to just-make-it-work

For those looking, I finally got it to work after sifting through the source code for AzureBlobXmlRepository ... but have a nagging suspicion I've missed the easy route to success here... (?)

I created a template XML file containing:

<?xml version="1.0" encoding="utf-8"?>
<keys></keys>

The name of the root element doesn't seem to be important (hopefully this is correct).

On first run, it seems to have populated the file successfully with the keys. Permissions seemed to have to be created at the Storage Account level with app service identity getting Blob Data Contributor access (not sure if this can be more restrictive?)

The app service identity was granted key wrap/unwrap access to the KeyVault.

HTH someone.

Azure team: can someone confirm this is the correct approach? If it's not, what is the right way? Either way, please can the docs make this more clear, as I'm sure many people are bumping into this continually.

[Rick-Anderson]

Comment hub feedback:

• incomplete information
• Short code snippets without any kind of larger context aren't helpful.
• not easy to understand.
• DataProtection was updated to use IOptions but there's no mention of how to reload configurations for DP (which is the main selling point of that system!)

[akempe]

@Rick-Anderson side question, why can't we just store the keys directly in keyvault? why the layer of indirection here?

[ghost]

Good day, from what i understand,

services.AddDataProtection() .PersistKeysToAzureBlobStorage(new Uri("<blobUriWithSasToken>")) .ProtectKeysWithAzureKeyVault("<keyIdentifier>", "<clientId>", "<clientSecret>");

PersistKeysToAzureBlobStorage saves the identity cookie encryption and decryption keys to azure blob storage.

ProtectKeysWithAzureKeyVault encrypts the blob containing the keys that are created by PersistKeysToAzureBlobStorage.

But what does clientId and clientSecret mean?

[mkArtakMSFT]

Thanks for bringing up this issue. @Pilchie can you please assign this to an engineer who is well familiar with the Data Protection. Let's get the docs updated to address the gap.

[Pilchie]

@haok @blowdart - either of you want to take a look?

[blowdart]

It's getting replaced soon with a new library soon, so it's probably not worth the investment, however @Petermarcu then needs to assign someone to own and update the docs going forward.

[ghost]

Looking forward to new library.

[DaleyKD]

@akempe - Thank you for that. I'm trying to use the new Azure.Extensions.AspNetCore.DataProtection.Keys SDK, and I'm not convinced it requires the Blob to exist, but I'm doing it anyway.

However, I think I may be getting a race condition on startup of my ASP.NET Core app. I know this is a docs issue, and I'm not here to derail. But this definitely needs some TLC to ensure we get it right.

Putting the stack here as a just in case:

info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[58]
      Creating key {bd0a2ab1-9338-477e-98b9-6a725797c851} with creation date 2020-07-10 13:46:43Z, activation date 2020-07-10 13:46:42Z, and expiration date 2020-10-08 13:46:42Z.
fail: Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider[48]
      An error occurred while reading the key ring.
System.Xml.XmlException: Root element is missing.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
   at System.Xml.XmlTextReaderImpl.Read()
   at System.Xml.Linq.XDocument.Load(XmlReader reader, LoadOptions options)
   at System.Xml.Linq.XDocument.Load(XmlReader reader)
   at Azure.Extensions.AspNetCore.DataProtection.Blobs.AzureBlobXmlRepository.CreateDocumentFromBlob(Byte[] blob)
   at Azure.Extensions.AspNetCore.DataProtection.Blobs.AzureBlobXmlRepository.StoreElementAsync(XElement element)
   at Azure.Extensions.AspNetCore.DataProtection.Blobs.AzureBlobXmlRepository.StoreElement(XElement element, String friendlyName)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.IInternalXmlKeyManager.CreateNewKey(Guid keyId, DateTimeOffset creationDate, DateTimeOffset activationDate, DateTimeOffset expirationDate)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.CreateNewKey(DateTimeOffset activationDate, DateTimeOffset expirationDate)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.CreateCacheableKeyRingCore(DateTimeOffset now, IKey keyJustAdded)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.ICacheableKeyRingProvider.GetCacheableKeyRing(DateTimeOffset now)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.GetCurrentKeyRingCore(DateTime utcNow, Boolean forceRefresh)
info: Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationHandler[7]
      Identity.Application was not authenticated. Failure message: Unprotect ticket failed

[akempe]

@DaleyKD no problem! Are you saying you're still getting this error?

System.Xml.XmlException: Root element is missing.

It looks identical to the one I had originally and lead me to initialising the blob with:

<?xml version="1.0" encoding="utf-8"?>
<keys></keys>

If you get it to work with the new SDK, please let us know as I'm sure people are going to run into this while MS resolve their docs situation. For what it's worth, this has been stable in prod for a few months now.

[DaleyKD]

@akempe : Let me clarify, as I wasn't as clear as I had hoped earlier.

• It appears that the new SDK (Azure.Extensions.AspNetCore.DataProtection.Keys, Version=1.0.0.0) does not get mad if the Blob does not exist. I see no error thrown, and the current documentation provided by MSFT (which says we should run the app once without ProtectKeysWithAzureKeyVault, then run it again with it) seems to be obsolete now.
• This error occurs whether or not I initialize the blob with (assuming I'm doing it correctly via C#):

<?xml version="1.0" encoding="utf-8"?><repository></repository>

• In dev, the Root element is missing error seems to occur when:
  • Run the website
  • Login and get a cookie
  • Stop the website
  • Delete the blob .xml
  • Run the website again
• It does not happen if I am logged out and have deleted the .xml

I suspect it's because it's trying to write two or more keys (one or more dealing with the Identity cookie and one or more dealing with the anti-forgery for POST) and therefore, having a race condition

Other things I have tried that have no effect:

• Ensure the initialized blob starts with the BOM \ufeff
• Try <keys></keys> vs. <repository></repository>
• Ensure that the content type for the blob is application/xml;charset=utf-8 vs. application/octet-stream

And finally, even when I run the app a second time, and it has a valid file in there from the previous run, and I do NOT delete it, I still get the error. Everything points back to a race condition.

[nmg196]

I'm also really stuck with this. The docs surrounding this are very unclear as they don't give any examples.

The documentation implies it will create the blob on the first run, but how can it do that if the docs also show you need the full URI to the actual file including a SAS token? That's not physicially possible from what I can see - so is the URL supposed to be for a blob container and not the blob xml file itself?

Looking at the comments it seems like the whole "automatic creation" of the blob doesn't work at all. It doesn't even say what should be in the file if you want to create it manually. The only workaround which worked for me was to initially save it to the local file system, then use the generated local file to seed the Azure blob. Ideally, the API should realise an empty file has been supplied and replace it with one of the correct format to reduce "getting started friction".

As @akempe mentioned, it's also really odd that the article doesn't mention anything about how to set up permissions for the SAS token and Key Vault, given the apparent importance of security when storing keys.

[arqe]

@nmg196 The very same here also. This documentation doesn't make sense and is is unfollowable.

Many of security related docs are frustratingly bad overall. More so that in this particular area you definitely want to know what exactly you are doing and not to trial random to see the magic happening.

[haldiggs]

I am probably the most ignorant person here as to what is supposed to be happening. I am trying to use PersistKeysToAzureBlobStorage and getting nothing but frustrated.

I am using Azure to generate the shared connection strings. I am using the "Blob service SAS URL" and assigning to a string variable named blobSasUrl.

I have created a container and I seem to be successful at creating sub containers in code. I am running this in a .Net 5.0 Core WebApi application.

services.AddDataProtection() .SetApplicationName(Assembly.GetEntryAssembly().GetName().Name) .PersistKeysToAzureBlobStorage(new Uri(blobSasUrl)) .ProtectKeysWithCertificate(clientCertificate);

All I keep getting is this error that doesn't really help me figure out why it does not work. Since I am not the one who generates the Url I have to depend on Azure.

Azure.RequestFailedException: Value for one of the query parameters specified in the request URI is invalid. RequestId:bd9cedff-d01e-00ef-4a26-2b644e000000 Time:2021-04-06T20:52:23.7976519Z Status: 400 (Value for one of the query parameters specified in the request URI is invalid.) ErrorCode: InvalidQueryParameterValue Headers: Server: Microsoft-HTTPAPI/2.0 x-ms-request-id: bd9cedff-d01e-00ef-4a26-2b644e000000 x-ms-client-request-id: eed59e39-e857-4ab4-8f2a-5e146a5cc7f6 x-ms-error-code: InvalidQueryParameterValue Date: Tue, 06 Apr 2021 20:52:23 GMT Content-Length: 351 Content-Type: application/xml

I am hoping this is related to this issue you guys are having with the documentation.

[Mike-E-angelo]

Any update here would be appreciated. Looking at the documentation it states:

If the app uses the older Azure packages (Microsoft.AspNetCore.DataProtection.AzureStorage and Microsoft.AspNetCore.DataProtection.AzureKeyVault), we recommend removing these references and upgrading to the Azure.Extensions.AspNetCore.DataProtection.Blobs and [Azure.Extensions.AspNetCore.DataProtection.Keys] https://www.nuget.org/packages/Azure.Extensions.AspNetCore.DataProtection.Keys)

But looking at these packages on NuGet, the packages recommended to remove are actually newer and updated more frequently than the recommended packages to install in their place.

For example, Azure.Extensions.AspNetCore.DataProtection.Blobs ("newer") is dated 5/14/2021 w/ 6 published versions, while Microsoft.AspNetCore.DataProtection.AzureStorage ("older") was last released 24 days ago with 8 published versions since the last publishing of the "new" package (5/14/2021).

Very confusing.

[haldiggs]

omatic creation" of the blob doesn't work at all. It doesn't even say what should be in the file if you want to create it manually. The only workaround w

good luck getting an answer. there must be documentation somewhere else because there hasn't been any answers to other questions for years

[datwelk]

@Rick-Anderson could we get a reply from Microsoft on this issue please?

[Rick-Anderson]

It's getting replaced soon with a new library soon, so it's probably not worth the investment, however @Petermarcu then needs to assign someone to own and update the docs going forward.

@Petermarcu can you assign someone to own and update the docs going forward.

[bradygaster]

@datwelk @haldiggs @Pyrobolser please see the issue I linked: https://github.com/dotnet/AspNetCore.Docs/issues/26093

If anyone has a moment to try that and let us know if it resolves your issue, we'd be appreciative. Thanks!

Or, if possible, try a fresh deployment using the code from the doc. If it works only after you manually create a container, please confirm.

[haldiggs]

dang, @bradygaster I would love it too, but that code was scrapped a long time back. I will have to rely on others at this time. I will have a new project coming up that could use this. Happy to see it coming alive though

[bradygaster]

I do have an update. PersistKeysToAzureBlobStorage does NOT create your container. It will create the blob if it isn't there.

I'm still researching ProtectKeysWithAzureKeyVault and I'm specifically interested in the error @DaleyKD posted above - Any recollection on how you were able to make that error pop?

[jwisener]

Why is the documentation so fragmented and hard to follow? I too had to dig into the xml code comments of the various libraries to try and understand why the example provided would not work. The only thing I can think is that the people creating the documentation don't understand how it's supposed to work. Or it's a reflection of the communication from the team to the tech writers, aka "you only get just fragments of understanding." Then you just puke it onto some random page. It "mentions permissions", but I am not sure if that is the only permission(s) needed in key-vault. (tbh, I don't trust the documentation).

[ghost]

Just deployed a new app using the code from the documentation and cannot get this working. Is there any guidelines on how to set this up from scratch?

Do you need to create a empty xml-file manually in the blob before running the code the first time?

We use Azure with Managed Identity to handle permissionsso we use that to connect to the keyvault. What do we need to do to get this to work?

//Setup connection details using managed identity
 var blobStorageUri = new Uri($"https://{dpConf.AccountName}.blob.core.windows.net/{dpConf.ContainerName}/{applicationName}");

var tokenCredential = new DefaultAzureCredential(options);

      services.AddDataProtection()
        .SetApplicationName(applicationName)
        .PersistKeysToAzureBlobStorage(blobStorageUri, tokenCredential)
        .ProtectKeysWithAzureKeyVault(new Uri("keyName"), tokenCredential);

[haldiggs]

sorry, we decided to use the file storage method so we could move forward. I'm sure it will be better when we revisit the issue

On Thu, Jun 9, 2022 at 1:40 AM Brady Gaster @.***> wrote:

@datwelk https://github.com/datwelk @haldiggs https://github.com/haldiggs @Pyrobolser https://github.com/Pyrobolser please see the issue I linked: #16422 https://github.com/dotnet/AspNetCore.Docs/issues/16422

If anyone has a moment to try that and let us know if it resolves your issue, we'd be appreciative. Thanks!

— Reply to this email directly, view it on GitHub https://github.com/dotnet/AspNetCore.Docs/issues/16422#issuecomment-1150731212, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM5KUQHZYYQOPGDMIAJN7LVOGGUHANCNFSM4KDLCN3A . You are receiving this because you were mentioned.Message ID: @.***>

-- Hal D. - "Non sibi sed patriae" (Not for self, but for country) Half of communication is listening, and you can't listen with your mouth.

[Rick-Anderson]

@Petermarcu can you assign someone to own and update the docs going forward.

[Rick-Anderson]

Email sent to @Petermarcu

[abratv]

• We need to "generate" the key in keyvault manually
• Make sure set the access policy for "keys" at least "Get", "UnwrapKey", "WrapKey"
• Make sure the container in blob storage exists
• No need to create blob in blob storage, it will be created automatically

[bendrick92]

Has there been any update on the documentation for configuring .NET Core Data Protection w/ Azure blob storage/key vault?

I've been trying to follow this and been getting very frustrated/lost.

EDIT: I took @haldiggs approach and gave up for now. Went with the PersistKeysToFileSystem approach until they can get the Key Vault documentation sorted out. Wasted days trying to troubleshoot all the different variables.

[rwb196884]

This is still completely fucked.

• Went to storage resrouce in azure and creted a new container. Used its menu to 'Generate SAS.
• Went to key vault service and generated a new key, clicked into it and copied its 'key ientifier' URL. Is that what you're supposed to do? Seems consistent with what @abratv says.

AspNetCoreDataProtectionOptions aspNetCoreDataProtectionOptions = new AspNetCoreDataProtectionOptions();
builder.Configuration.GetSection("AspNetCoreDataProtection").Bind(aspNetCoreDataProtectionOptions);
aspNetCoreDataProtectionOptions.BlobUriWithSasToken = "https://whatever.blob.core.windows.net/whatever-data-protection?sp=racwdl&st=2023-08-08T09:23:11Z&se=2123-08-08T17:23:11Z&spr=https&sv=2022-11-02&sr=c&sig=bwixpImR4CVWERyi%2BDtmqGxeCDVX9P0nrzNLaEaWOTQ%3D";
aspNetCoreDataProtectionOptions.KeyIdentifier = "https://whatever-key-vault.vault.azure.net/keys/whatever-data-protection/4a1114f79ad94cceaf6761796668c712";

string keyVaultName = builder.Configuration.GetValue<string>("KeyVaultName");
string trimmedContentRootPath = builder.Environment.ContentRootPath.TrimEnd(Path.DirectorySeparatorChar);
if (!string.IsNullOrEmpty(keyVaultName))
{
    builder.Services.AddDataProtection()
        .SetApplicationName(trimmedContentRootPath)
        .PersistKeysToAzureBlobStorage(new Uri(aspNetCoreDataProtectionOptions.BlobUriWithSasToken))
        .ProtectKeysWithAzureKeyVault(new Uri(aspNetCoreDataProtectionOptions.KeyIdentifier), new DefaultAzureCredential())
        ;
}

but

RequestFailedException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

[jamie-tillman]

This is still completely fucked.

* Went to storage resrouce in azure and creted a new container. Used its menu to 'Generate SAS.

* Went to key vault service and generated a new key, clicked into it and copied its 'key ientifier' URL.
  Is that what you're supposed to do? Seems consistent with what @abratv says.

AspNetCoreDataProtectionOptions aspNetCoreDataProtectionOptions = new AspNetCoreDataProtectionOptions();
builder.Configuration.GetSection("AspNetCoreDataProtection").Bind(aspNetCoreDataProtectionOptions);
aspNetCoreDataProtectionOptions.BlobUriWithSasToken = "https://whatever.blob.core.windows.net/whatever-data-protection?sp=racwdl&st=2023-08-08T09:23:11Z&se=2123-08-08T17:23:11Z&spr=https&sv=2022-11-02&sr=c&sig=bwixpImR4CVWERyi%2BDtmqGxeCDVX9P0nrzNLaEaWOTQ%3D";
aspNetCoreDataProtectionOptions.KeyIdentifier = "https://whatever-key-vault.vault.azure.net/keys/whatever-data-protection/4a1114f79ad94cceaf6761796668c712";

string keyVaultName = builder.Configuration.GetValue<string>("KeyVaultName");
string trimmedContentRootPath = builder.Environment.ContentRootPath.TrimEnd(Path.DirectorySeparatorChar);
if (!string.IsNullOrEmpty(keyVaultName))
{
    builder.Services.AddDataProtection()
        .SetApplicationName(trimmedContentRootPath)
        .PersistKeysToAzureBlobStorage(new Uri(aspNetCoreDataProtectionOptions.BlobUriWithSasToken))
        .ProtectKeysWithAzureKeyVault(new Uri(aspNetCoreDataProtectionOptions.KeyIdentifier), new DefaultAzureCredential())
        ;
}

but

RequestFailedException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

Same issue here in July 2024. This thread having been started more than 4 years ago, it's clear no one cares about making this function usable.