The antiforgery token could not be decrypted - Only for specific user

[weirdyang]

Is there an existing issue for this?

• [X] I have searched the existing issues

Describe the bug

An exception was thrown while deserializing the token.
Microsoft.AspNetCore.Antiforgery.AntiforgeryValidationException: The antiforgery token could not be decrypted.
---> System.Security.Cryptography.CryptographicException: The key {669d513e-a172-4851-b160-04b523abbc1e} was not found in the key ring.
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.UnprotectCore(Byte[] protectedData, Boolean allowOperationsOnRevokedKeys, UnprotectStatus& status)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.DangerousUnprotect(Byte[] protectedData, Boolean ignoreRevocationErrors, Boolean& requiresMigration, Boolean& wasRevoked)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Unprotect(Byte[] protectedData)
   at Microsoft.AspNetCore.Antiforgery.DefaultAntiforgeryTokenSerializer.Deserialize(String serializedToken)
   --- End of inner exception stack trace ---
   at Microsoft.AspNetCore.Antiforgery.DefaultAntiforgeryTokenSerializer.Deserialize(String serializedToken)
   at Microsoft.AspNetCore.Antiforgery.DefaultAntiforgery.GetCookieTokenDoesNotThrow(HttpContext httpContext)

We insert an anti-forgery token into the form using @Html.AntiForgeryToken() and validate the request using the [ValidateAntiForgeryToken], however we occasionally get the error above when one user tries to submit.

How is the key generated? Why does the user affect the generation of the token?

Expected Behavior

When user submit a form, the request token is valid and the user request is accepted.

Steps To Reproduce

1. Create a cshtml page
2. Include an antiforgery token in the form using @Html.AntiForgeryToken()
3. Annotate the controller method with [ValidateAntiForgeryToken]
4. Make a post request with the token

Exceptions (if any)

An exception was thrown while deserializing the token. Microsoft.AspNetCore.Antiforgery.AntiforgeryValidationException: The antiforgery token could not be decrypted.

.NET Version

netcoreapp3.1

Anything else?

We have tried this with different users, different devices and different browsers.

Only this user has the issue, and it is correlated with the logs' timestamp.

I've checked the source code, and was wondering could it be due to special characters in the user name?

https://github.com/dotnet/aspnetcore/blob/main/src/Antiforgery/src/Internal/DefaultAntiforgeryTokenGenerator.cs No response

[amcasey]

I've checked the source code, and was wondering could it be due to special characters in the user name?

I suppose it's not impossible - are you able to share the user's name? Or maybe just the non-ASCII characters in it? Or what sort of non-ASCII characters it contains (Latin with accents? Japanese? emoji?)?

it is correlated with the logs' timestamp

I wasn't sure quite what you meant by this. I think maybe the user is seeing some sort of failure in their browser and when you look for failures in the log at the corresponding time, this is the stack you see?

Some boilerplate questions:

1. What is your hosting environment? IIS? Kestrel? Azure? A container of some sort?
2. Do you have multiple instances running?

[NitinSinghSF]

We are having this same error running aspnet core web api 6.0 on AWS ECS

[ChadEQ]

I had the same issue in asp.net 7.0 on windows/IIS. One instance. And no special characters in the user name.

[weirdyang]

I've checked the source code, and was wondering could it be due to special characters in the user name?

I suppose it's not impossible - are you able to share the user's name? Or maybe just the non-ASCII characters in it? Or what sort of non-ASCII characters it contains (Latin with accents? Japanese? emoji?)?

it is correlated with the logs' timestamp

I wasn't sure quite what you meant by this. I think maybe the user is seeing some sort of failure in their browser and when you look for failures in the log at the corresponding time, this is the stack you see?

Some boilerplate questions:

1. What is your hosting environment? IIS? Kestrel? Azure? A container of some sort?

2. Do you have multiple instances running?

Hi,

It doesn't contain any special characters or ascii.

Yes, I had the user share their screen, trigger the error and check the logs to ensure it's that error preventing them from submitting a form.

No, I do not have multiple instances running.

[amcasey]

@weirdyang If you're in contact with the user, it might be interesting to ask them to try again after clearing their cookies.

@weirdyang @NitinSinghSF @ChadEQ How is your key ring stored? Is it a file on disk? A blob in Azure Storage?

[amcasey]

Does anyone have a log including namespace Microsoft.AspNetCore.DataProtection, ideally at the Trace level?

[weirdyang]

Hi

We have tried the following:

1. Private browsing session

2. Clearing cookies and cache

3. On an iPad and iPhone

4. On an android phone

5. On another user’s laptop

We still run into the same error.

On Wed, 15 Nov 2023 at 8:34 AM, Andrew Casey @.***> wrote:

@weirdyang https://github.com/weirdyang If you're in contact with the user, it might be interesting to ask them to try again after clearing their cookies.

@weirdyang https://github.com/weirdyang @NitinSinghSF https://github.com/NitinSinghSF @ChadEQ https://github.com/ChadEQ How is your key ring stored? Is it a file on disk? A blob in Azure Storage?

— Reply to this email directly, view it on GitHub https://github.com/dotnet/aspnetcore/issues/47185#issuecomment-1811612208, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGDEEVXX32FPULLO3FNWH5TYEQEXRAVCNFSM6AAAAAAV2DUB7GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJRGYYTEMRQHA . You are receiving this because you were mentioned.Message ID: @.***>

[amcasey]

@weirdyang To confirm, this happens on every form submission from this user?

@NitinSinghSF @ChadEQ Are you seeing the same? There are multiple ways this error could arise.

[amcasey]

All sorts of default behaviors are overridable, e.g. with AntiforgeryOptions, but it's not immediately obvious to me how the user's name/ID could be incorporated into the anti-forgery token or the data protection key.

Edit: it's in the anti-forgery token, which might explain why the problem follows the user. Maybe something about the way the user is encoded in the token breaks deserialization so that a bad data protection key ID is retrieved?

[amcasey]

I'm assuming no one has a repro project they're able to share?

[amcasey]

Just for the record, I suppose I should mention that netcoreapp3.1 is out of support and wouldn't be patched if that ended up being the fix. I'm tentatively operating on the assumption that @NitinSinghSF and @ChadEQ are seeing the same exception in 6.0 and 7.0, but it would be nice if they could confirm that they're seeing the same symptoms - the problem follows particular users across devices / cookie wipes.

[ChadEQ]

I'm on .net 7.0. And for me, it's extremely rare, not consistent.

[amcasey]

@ChadEQ Are you seeing it for particular users or do you just occasionally see that log message?

[amcasey]

@weirdyang Is the username unusually long or short?

[amcasey]

@weirdyang Are you working with @apetrut? #51165 appears to have an error message with the same GUID.

[weirdyang]

@weirdyang Are you working with @apetrut? #51165 appears to have an error message with the same GUID.

No

The user name is not unusually long

[ChadEQ]

Currently only see it for one user. It seems to be the first submit (the user is logging in) for the user after an IIS app pool recycle. The user name is 3 characters, but so are all the others.

[amcasey]

And is that user seeing in on multiple devices and/or after clearing cookies?

[amcasey]

Also, can you please confirm you're seeing a different GUID in your error message? I'm going to get creeped out if there are three of these.

[amcasey]

Serialization format for the anti-forgery token:

/* The serialized format of the anti-XSRF token is as follows:
 * Version: 1 byte integer
 * SecurityToken: 16 byte binary blob
 * IsCookieToken: 1 byte Boolean
 * [if IsCookieToken != true]
 *   +- IsClaimsBased: 1 byte Boolean
 *   |  [if IsClaimsBased = true]
 *   |    `- ClaimUid: 32 byte binary blob
 *   |  [if IsClaimsBased = false]
 *   |    `- Username: UTF-8 string with 7-bit integer length prefix
 *   `- AdditionalData: UTF-8 string with 7-bit integer length prefix
 */

This blob of binary is then encrypted. From the stack, this is done using a KeyRingBasedDataProtector, which produces output starting with the magic header 0x09F0C9F0 and the bytes of the GUID key ID (669d513e-a172-4851-b160-04b523abbc1e in the original report), followed by the encrypted data, etc.

Finally, the encrypted payload is base64 encoded.

Reversing this to deserialize, we expect only to have to base64 decode and discard 4 bytes of magic header to access the 16 bytes of the key ID.

While user-specific data does appear in the payload, particularly in the Username or ClaimUid, these come after the key ID, so it's not clear to me how they could cause the key ID to be deserialized incorrectly.

There are some nuances around endianness, but we've said that this repros across a number of widely-used devices.

There are definitely scenarios/bugs where you could fail to find a key, but I'm having trouble thinking of a way it could affect only particular users.

[amcasey]

I gather these apps are authenticated. What mechanism are people using? Claims-based? Which provider?

[ChadEQ]

To answer your previous questions first, but not sure if any of them where directed at me. Different key than what's reported in this issue. User hasn't tried different devices, but has tried Edge and Chrome. Interestingly but probably coincidental, it seems to be easier to reproduce using Edge than Chrome. Yes, using authentication. Cookie authentication scheme and yes using claims. But for us, the error is happening on an anonymous page, before the user has logged in, so no authentication cookie is sent, just the anti-forgery.

[amcasey]

@ChadEQ Thanks! Sorry for not being clearer with tags.

I'm relieved to hear that the guid is different. 😅

Edge and Chrome are pretty similar these days, though that should at least get you a fresh cookie. Trying on a different device would be interesting, if that's an option.

The fact that this is happening before authentication is interesting - I think it will still put a blank username in the token, but that should be the same for all users. Is the user themself distinguished in some way? Are they the first to sign in every morning (e.g. because of time zones)?

[amcasey]

@weirdyang @ChadEQ Are either of you able to debug the server while this is failing? We have public symbols and I can suggest breakpoints.

[apetrut]

@amcasey I had a similar issue when using same browser with 2 tabs. If tried using 2 different browsers then I didn't get that issue anymore.

[amcasey]

@apetrut I think two tabs in the same browser would use the same cookie and require the same key for decryption.

[amcasey]

@apetrut Any insights on the GUID collision? I'm assuming you didn't copy-paste yours from this bug.

[apetrut]

@amcasey I don't recall seeing this ticket when I opened mine.

@weirdyang This is what I've tried in order to get rid of the error and it worked for me:

services.AddAntiforgery(options => { options.FormFieldName = "AntiforgeryFieldname"; options.HeaderName = "X-CSRF-TOKEN-HEADERNAME"; options.SuppressXFrameOptionsHeader = false; });

and also:

services.AddDataProtection() .SetApplicationName("yourAppName") .PersistKeysToFileSystem(new DirectoryInfo(string.Format(@"{0}opt{0}app{0}sysfiles{0}", Path.DirectorySeparatorChar))) .ProtectKeysWithCertificate(new X509Certificate2(string.Format(@"{0}certificates{0}verify-cert.pfx", Path.DirectorySeparatorChar), Configuration["ApplicationSettings:CertificatePassword"])) .SetDefaultKeyLifetime(TimeSpan.FromDays(15));

[amcasey]

It's not obvious to me how changing the anti-forgery options could help, but explicitly configuring data protection might well bypass whatever bug/surprising behavior is causing this error.

[amcasey]

FYI, we're going into a long weekend here, so responses are likely to be slow until next week.