DistributedCache.AzureTableStorage icon indicating copy to clipboard operation
DistributedCache.AzureTableStorage copied to clipboard

Possible race condition when reading expired item

Open anth12 opened this issue 1 year ago • 1 comments

I encountered the following error from a call to DistributedCacheExtensions.GetStringAsync for a cached item accessed frequently with a short (10s) expiry.

Looking at the implementation, I suspect there's a race condition in RefreshAsync when multiple processes invoke with the same key right before it expires resulting in one removing the cache entry while the other attempts to update the LastAccessTime.

One option may be to change _tableSet.Value.UpdateAsync to _tableSet.Value.AddOrUpdateAsync. This approach would result in recreating an expired item... Alternatively, error handling to catch the Table Storage write failure could be added.

      Azure.RequestFailedException: The specified resource does not exist.
RequestId:e7c2ea29-73d9-4b2b-a79c-4f1fa27de8a3
Time:2023-06-14T08:24:18.864Z
      Status: 404 (Not Found)
      ErrorCode: ResourceNotFound

      Content:
      {"odata.error":{"code":"ResourceNotFound","message":{"lang":"en-US","value":"The
specified resource does not exist.\nRequestId:e7c2ea29-73d9-4b2b-a79c-4f1fa27de8a3\nTim
e:2023-06-14T08:24:18.864Z"}}}

      Headers:
      Server: Azurite-Table/3.23.0
      x-ms-error-code: REDACTED
      x-ms-request-id: e7c2ea29-73d9-4b2b-a79c-4f1fa27de8a3
      x-ms-version: REDACTED
      Date: Wed, 14 Jun 2023 08:24:18 GMT
      Connection: keep-alive
      Keep-Alive: REDACTED
      Transfer-Encoding: chunked
      Content-Type: application/json;odata=minimalmetadata;streaming=true;charset=utf-8


         at Azure.Data.Tables.TableRestClient.MergeEntityAsync(String table, String par
titionKey, String rowKey, Nullable`1 timeout, String ifMatch, IDictionary`2 tableEntity
Properties, QueryOptions queryOptions, CancellationToken cancellationToken)
         at Azure.Data.Tables.TableClient.UpdateEntityAsync[T](T entity, ETag ifMatch,
TableUpdateMode mode, CancellationToken cancellationToken)
         at DistributedCache.AzureTableStorage.Implementations.AzureTableStorageCache.R
efreshAsync(String key, CancellationToken cancellationToken)
         at DistributedCache.AzureTableStorage.Implementations.AzureTableStorageCache.G
etAsync(String key, CancellationToken token)
         at Microsoft.Extensions.Caching.Distributed.DistributedCacheExtensions.GetStri
ngAsync(IDistributedCache cache, String key, CancellationToken token)
...

anth12 avatar Jun 14 '23 08:06 anth12

@anth12 If you can create a example console which recreates this issue and create a PR, that would be very nice.

StefH avatar Jun 14 '23 10:06 StefH