EasyCaching icon indicating copy to clipboard operation
EasyCaching copied to clipboard
verified publisher icon dotnetcore

Serializer Issues with EfCoreSecondLevelCacheInterceptor

Open gary-roach opened this issue 3 years ago • 1 comments

Description

I've been attempting to use EasyCaching.Redis as the cache provider for the EFCoreSecondLevelCacheInterceptor, but I haven't found a Serializer that's working. I've tried all three options:

  1. EasyCaching.Serialization.Json
  2. EasyCaching.Serialization.MessagePack
  3. EasyCaching.Serialization.SystemTextJson

Both Serilization.Json and Serialization.SystemTextJson give the error: Unable to translate bytes [84] at index 0 from specified code page to Unicode.

Serialization.MessagePack gives: MessagePackSerializationException: Unexpected msgpack code 123 (positive fixint) encountered.

Related code

Here is the code that's being used to add the libraries to the middleware.

builder.Services.AddEFSecondLevelCache(options =>
{
    options.UseEasyCachingCoreProvider(providerName, CacheExpirationMode.Sliding,
        TimeSpan.FromMinutes(builder.Configuration.GetValue<double>("EFCoreSecondLevelCacheInterceptor:EntityCacheDuration")))
        .DisableLogging(!builder.Environment.IsDevelopment()).UseCacheKeyPrefix("EF_");
});

builder.Services.AddEasyCaching(option =>
{
    option.UseRedis(builder.Configuration, providerName, "EasyCaching:Redis");
    option.WithJson(providerName);
});

The EasyCache settings:

"EasyCaching": {
    "Redis": {
      "MaxRdSecond": 120, //The max random second. If this value greater then zero, the seted cache items' expiration will add a random second. This is mainly for preventing Cache Crash
      "EnableLogging": true,
      "LockMs": 5000, //Mutex key's alive time(ms), default is 5000
      "SleepMs": 300, //When mutex key alive, it will sleep some time, default is 300
      "DbConfig": {
        "Password": *******,
        "ConnectionTimeout": 5000,
        "AllowAdmin": true,
        "Endpoints": [
          {
            "Host": ********,
            "Port": 6380
          }
        ],
        "Database": 0, //The Redis database index the cache will use
        "AsyncTimeout": 5000, //Specifies the time in milliseconds that the system should allow for asynchronous operations (defaults to SyncTimeout)
        "SyncTimeout": 5000, //Specifies the time in milliseconds that the system should allow for synchronous operations (defaults to 5 seconds)
        "SerializerName": "Redis"
      }
    }
  },

The EF query that's being called is

int? restaurantId = (from c in _unitOfWork.LmsContext.Corporations
                                 join r in _unitOfWork.LmsContext.Restaurants
                                 on c.CorpID equals r.CorpID
                                 where c.CorpCode == corporationCode && r.RestaurantNumber == restNum
                                 select r.RestaurantID).Cacheable().FirstOrDefault();

The resulting exception stacktrace when using EasyCaching.Serialization.Json is

[19:17:20 ERR] An exception occurred while iterating over the results of a query for context type 'LaborManagement.Data.LmsContext'.
System.Text.DecoderFallbackException: Unable to translate bytes [84] at index 0 from specified code page to Unicode.
   at System.Text.DecoderExceptionFallbackBuffer.Throw(Byte[] bytesUnknown, Int32 index)
   at System.Text.DecoderExceptionFallbackBuffer.Fallback(Byte[] bytesUnknown, Int32 index)
   at System.Text.Encoding.GetCharsWithFallback(ReadOnlySpan`1 bytes, Int32 originalBytesLength, Span`1 chars, Int32 originalCharsLength, DecoderNLS decoder)
   at System.Text.UTF8Encoding.GetCharsWithFallback(ReadOnlySpan`1 bytes, Int32 originalBytesLength, Span`1 chars, Int32 originalCharsLength, DecoderNLS decoder)
   at System.Text.Encoding.GetCharsWithFallback(Byte* pOriginalBytes, Int32 originalByteCount, Char* pOriginalChars, Int32 originalCharCount, Int32 bytesConsumedSoFar, Int32 charsWrittenSoFar, DecoderNLS decoder)
   at System.Text.Encoding.GetChars(Byte* pBytes, Int32 byteCount, Char* pChars, Int32 charCount, DecoderNLS decoder)
   at System.Text.DecoderNLS.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, Boolean flush)
   at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex, Boolean flush)
   at System.IO.StreamReader.ReadBuffer(Span`1 userBuffer, Boolean& readToUserBuffer)
   at System.IO.StreamReader.ReadSpan(Span`1 buffer)
   at System.IO.StreamReader.Read(Char[] buffer, Int32 index, Int32 count)
   at Newtonsoft.Json.JsonTextReader.ParseValue()
   at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize[T](JsonReader reader)
   at EasyCaching.Serialization.Json.DefaultJsonSerializer.Deserialize[T](Byte[] bytes)
   at EasyCaching.Redis.DefaultRedisCachingProvider.BaseGet[T](String cacheKey)
   at EasyCaching.Core.EasyCachingAbstractProvider.Get[T](String cacheKey)
   at EFCoreSecondLevelCacheInterceptor.EFEasyCachingCoreProvider.<>c__DisplayClass7_0.<GetValue>b__0() in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\EFEasyCachingCoreProvider.cs:line 113
   at EFCoreSecondLevelCacheInterceptor.ReaderWriterLockProvider.TryReadLocked[T](Func`1 action, Int32 timeout) in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\ReaderWriterLockProvider.cs:line 56
   at EFCoreSecondLevelCacheInterceptor.DbCommandInterceptorProcessor.ProcessExecutingCommands[T](DbCommand command, DbContext context, T result) in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\DbCommandInterceptorProcessor.cs:line 164
   at EFCoreSecondLevelCacheInterceptor.SecondLevelCacheInterceptor.ReaderExecuting(DbCommand command, CommandEventData eventData, InterceptionResult`1 result) in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\SecondLevelCacheInterceptor.cs:line 206
   at Microsoft.EntityFrameworkCore.Diagnostics.Internal.RelationalCommandDiagnosticsLogger.CommandReaderExecuting(IRelationalConnection connection, DbCommand command, DbContext context, Guid commandId, Guid connectionId, DateTimeOffset startTime, CommandSource commandSource)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReader(RelationalCommandParameterObject parameterObject)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator)
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
System.Text.DecoderFallbackException: Unable to translate bytes [84] at index 0 from specified code page to Unicode.
   at System.Text.DecoderExceptionFallbackBuffer.Throw(Byte[] bytesUnknown, Int32 index)
   at System.Text.DecoderExceptionFallbackBuffer.Fallback(Byte[] bytesUnknown, Int32 index)
   at System.Text.Encoding.GetCharsWithFallback(ReadOnlySpan`1 bytes, Int32 originalBytesLength, Span`1 chars, Int32 originalCharsLength, DecoderNLS decoder)
   at System.Text.UTF8Encoding.GetCharsWithFallback(ReadOnlySpan`1 bytes, Int32 originalBytesLength, Span`1 chars, Int32 originalCharsLength, DecoderNLS decoder)
   at System.Text.Encoding.GetCharsWithFallback(Byte* pOriginalBytes, Int32 originalByteCount, Char* pOriginalChars, Int32 originalCharCount, Int32 bytesConsumedSoFar, Int32 charsWrittenSoFar, DecoderNLS decoder)
   at System.Text.Encoding.GetChars(Byte* pBytes, Int32 byteCount, Char* pChars, Int32 charCount, DecoderNLS decoder)
   at System.Text.DecoderNLS.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, Boolean flush)
   at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex, Boolean flush)
   at System.IO.StreamReader.ReadBuffer(Span`1 userBuffer, Boolean& readToUserBuffer)
   at System.IO.StreamReader.ReadSpan(Span`1 buffer)
   at System.IO.StreamReader.Read(Char[] buffer, Int32 index, Int32 count)
   at Newtonsoft.Json.JsonTextReader.ParseValue()
   at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize[T](JsonReader reader)
   at EasyCaching.Serialization.Json.DefaultJsonSerializer.Deserialize[T](Byte[] bytes)
   at EasyCaching.Redis.DefaultRedisCachingProvider.BaseGet[T](String cacheKey)
   at EasyCaching.Core.EasyCachingAbstractProvider.Get[T](String cacheKey)
   at EFCoreSecondLevelCacheInterceptor.EFEasyCachingCoreProvider.<>c__DisplayClass7_0.<GetValue>b__0() in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\EFEasyCachingCoreProvider.cs:line 113
   at EFCoreSecondLevelCacheInterceptor.ReaderWriterLockProvider.TryReadLocked[T](Func`1 action, Int32 timeout) in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\ReaderWriterLockProvider.cs:line 56
   at EFCoreSecondLevelCacheInterceptor.DbCommandInterceptorProcessor.ProcessExecutingCommands[T](DbCommand command, DbContext context, T result) in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\DbCommandInterceptorProcessor.cs:line 164
   at EFCoreSecondLevelCacheInterceptor.SecondLevelCacheInterceptor.ReaderExecuting(DbCommand command, CommandEventData eventData, InterceptionResult`1 result) in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\SecondLevelCacheInterceptor.cs:line 206
   at Microsoft.EntityFrameworkCore.Diagnostics.Internal.RelationalCommandDiagnosticsLogger.CommandReaderExecuting(IRelationalConnection connection, DbCommand command, DbContext context, Guid commandId, Guid connectionId, DateTimeOffset startTime, CommandSource commandSource)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReader(RelationalCommandParameterObject parameterObject)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator)
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()

The stacktrace for the exception when using Serialization.MessagePack is:

[19:29:51 ERR] An exception occurred while iterating over the results of a query for context type 'LaborManagement.Data.LmsContext'.
MessagePack.MessagePackSerializationException: Failed to deserialize EFCoreSecondLevelCacheInterceptor.EFCachedData value.
 ---> MessagePack.MessagePackSerializationException: Unexpected msgpack code 123 (positive fixint) encountered.
   at MessagePack.MessagePackReader.ThrowInvalidCode(Byte code)
   at MessagePack.MessagePackReader.TryReadMapHeader(Int32& count)
   at MessagePack.MessagePackReader.ReadMapHeader()
   at MessagePack.Formatters.EFCoreSecondLevelCacheInterceptor_EFCachedDataFormatter1.Deserialize(MessagePackReader& reader, MessagePackSerializerOptions options)
   at MessagePack.MessagePackSerializer.Deserialize[T](MessagePackReader& reader, MessagePackSerializerOptions options)
   --- End of inner exception stack trace ---
   at MessagePack.MessagePackSerializer.Deserialize[T](MessagePackReader& reader, MessagePackSerializerOptions options)
   at MessagePack.MessagePackSerializer.Deserialize[T](ReadOnlyMemory`1 buffer, MessagePackSerializerOptions options, CancellationToken cancellationToken)
   at EasyCaching.Serialization.MessagePack.DefaultMessagePackSerializer.Deserialize[T](Byte[] bytes)
   at EasyCaching.Redis.DefaultRedisCachingProvider.BaseGet[T](String cacheKey)
   at EasyCaching.Core.EasyCachingAbstractProvider.Get[T](String cacheKey)
   at EFCoreSecondLevelCacheInterceptor.EFEasyCachingCoreProvider.<>c__DisplayClass7_0.<GetValue>b__0() in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\EFEasyCachingCoreProvider.cs:line 113
   at EFCoreSecondLevelCacheInterceptor.ReaderWriterLockProvider.TryReadLocked[T](Func`1 action, Int32 timeout) in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\ReaderWriterLockProvider.cs:line 56
   at EFCoreSecondLevelCacheInterceptor.DbCommandInterceptorProcessor.ProcessExecutingCommands[T](DbCommand command, DbContext context, T result) in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\DbCommandInterceptorProcessor.cs:line 164
   at EFCoreSecondLevelCacheInterceptor.SecondLevelCacheInterceptor.ReaderExecuting(DbCommand command, CommandEventData eventData, InterceptionResult`1 result) in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\SecondLevelCacheInterceptor.cs:line 206
   at Microsoft.EntityFrameworkCore.Diagnostics.Internal.RelationalCommandDiagnosticsLogger.CommandReaderExecuting(IRelationalConnection connection, DbCommand command, DbContext context, Guid commandId, Guid connectionId, DateTimeOffset startTime, CommandSource commandSource)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReader(RelationalCommandParameterObject parameterObject)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator)
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
MessagePack.MessagePackSerializationException: Failed to deserialize EFCoreSecondLevelCacheInterceptor.EFCachedData value.
 ---> MessagePack.MessagePackSerializationException: Unexpected msgpack code 123 (positive fixint) encountered.
   at MessagePack.MessagePackReader.ThrowInvalidCode(Byte code)
   at MessagePack.MessagePackReader.TryReadMapHeader(Int32& count)
   at MessagePack.MessagePackReader.ReadMapHeader()
   at MessagePack.Formatters.EFCoreSecondLevelCacheInterceptor_EFCachedDataFormatter1.Deserialize(MessagePackReader& reader, MessagePackSerializerOptions options)
   at MessagePack.MessagePackSerializer.Deserialize[T](MessagePackReader& reader, MessagePackSerializerOptions options)
   --- End of inner exception stack trace ---
   at MessagePack.MessagePackSerializer.Deserialize[T](MessagePackReader& reader, MessagePackSerializerOptions options)
   at MessagePack.MessagePackSerializer.Deserialize[T](ReadOnlyMemory`1 buffer, MessagePackSerializerOptions options, CancellationToken cancellationToken)
   at EasyCaching.Serialization.MessagePack.DefaultMessagePackSerializer.Deserialize[T](Byte[] bytes)
   at EasyCaching.Redis.DefaultRedisCachingProvider.BaseGet[T](String cacheKey)
   at EasyCaching.Core.EasyCachingAbstractProvider.Get[T](String cacheKey)
   at EFCoreSecondLevelCacheInterceptor.EFEasyCachingCoreProvider.<>c__DisplayClass7_0.<GetValue>b__0() in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\EFEasyCachingCoreProvider.cs:line 113
   at EFCoreSecondLevelCacheInterceptor.ReaderWriterLockProvider.TryReadLocked[T](Func`1 action, Int32 timeout) in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\ReaderWriterLockProvider.cs:line 56
   at EFCoreSecondLevelCacheInterceptor.DbCommandInterceptorProcessor.ProcessExecutingCommands[T](DbCommand command, DbContext context, T result) in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\DbCommandInterceptorProcessor.cs:line 164
   at EFCoreSecondLevelCacheInterceptor.SecondLevelCacheInterceptor.ReaderExecuting(DbCommand command, CommandEventData eventData, InterceptionResult`1 result) in F:\Prog\1398\EFCoreSecondLevelCacheInterceptor\src\EFCoreSecondLevelCacheInterceptor\SecondLevelCacheInterceptor.cs:line 206
   at Microsoft.EntityFrameworkCore.Diagnostics.Internal.RelationalCommandDiagnosticsLogger.CommandReaderExecuting(IRelationalConnection connection, DbCommand command, DbContext context, Guid commandId, Guid connectionId, DateTimeOffset startTime, CommandSource commandSource)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReader(RelationalCommandParameterObject parameterObject)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator)
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()

Specifications

  • Provider : EasyCaching.Redis(version 1.6.1)
  • Interceptor : EFCoreSecondLevelCacheInterceptor(version 3.6.3)
  • Serializer : EasyCaching.Serialization.Json(version 1.6.1)

gary-roach avatar Sep 13 '22 23:09 gary-roach

@gary-roach Thanks for your interest in this project.

It seems that there are some encoding issues.

Could you please check and provide some test data for us to reproduce it?

catcherwong avatar Sep 14 '22 00:09 catcherwong

No response, Closed!

catcherwong avatar Jan 01 '23 03:01 catcherwong