AccessViolationException when using MsgPack.Serialization.MessagePackSerializer.Get

[chucker]

I'm getting a hard-to-reproduce problem with a serializer. The type looks like this:

namespace eL_libCommon.Container.GridLayouts
{
    public enum HashAlgorithm
    {
        MD5
    }

    public enum CompressionAlgorithm
    {
        GZip
    }

    public class GridLayoutSerializedData
    {
        public const int CURRENT_VERSION = 1;
        public int Version { get; set; }

        public HashAlgorithm HashAlgorithm { get; set; }
        public byte[] HashedData { get; set; }

        public CompressionAlgorithm CompressionAlgorithm { get; set; }
        public byte[] CompressedData { get; set; }
    }
}

I.e., quite boring. The only thing of note is that CompressedData will often take of hundreds of kilobytes.

The problematic call occurs here:

Dim serializer As MsgPack.Serialization.MessagePackSerializer(Of GridLayouts.GridLayoutSerializedData) = MsgPack.Serialization.MessagePackSerializer.Get(Of GridLayouts.GridLayoutSerializedData)()
Dim dataFile As GridLayouts.GridLayoutSerializedData = serializer.UnpackSingleObject(buffer)

…which runs in IIS / ASP.NET, and is ultimately called through .NET 2.0-style SOAP.

This works fine in the debugger, but apparently ceases to work after a while in deployment. Once the first call fails, all subsequent calls apparently fail as well. The client receives an exception:

Exception thrown: 'System.Web.Services.Protocols.SoapException' in System.Web.Services.dll

Additional information: System.Web.Services.Protocols.SoapException: Die Anforderung konnte vom Server nicht verarbeitet werden. ---> System.AccessViolationException: Es wurde versucht, im geschützten Speicher zu lesen oder zu schreiben. Dies ist häufig ein Hinweis darauf, dass anderer Speicher beschädigt ist.

   bei System.Reflection.Emit.TypeBuilder._TermCreateClass(Int32 handle, Module module)
   bei System.Reflection.Emit.TypeBuilder.CreateTypeNoLock()
   bei System.Reflection.Emit.TypeBuilder.CreateType()
   bei MsgPack.Serialization.EmittingSerializers.FieldBasedSerializerEmitter.CreateConstructor[T]()
   bei MsgPack.Serialization.EmittingSerializers.SerializerEmitter.CreateInstance[T](SerializationContext context, PolymorphismSchema schema)
   bei MsgPack.Serialization.EmittingSerializers.ILEmittingSerializerBuilder`2.<>c__DisplayClass6f.<CreateSerializerConstructor>b__6e(SerializationContext context)
   bei MsgPack.Serialization.AbstractSerializers.SerializerBuilder`3.BuildSerializerInstance(SerializationContext context, Type concreteType, PolymorphismSchema schema)
   bei MsgPack.Serialization.MessagePackSerializer.CreateInternal[T](SerializationContext context, PolymorphismSchema schema)
   bei MsgPack.Serialization.SerializationContext.GetSerializer[T](Object providerParameter)
   bei MsgPack.Serialization.MessagePackSerializer.Get[T](SerializationContext context, Object providerParameter)
   bei MsgPack.Serialization.MessagePackSerializer.Get[T](SerializationContext context)
   bei MsgPack.Serialization.MessagePackSerializer.Get[T]()
   bei el.tpm_db.db.GetSelectedGridLayout(GridLayoutContext context, Byte[][] knownHashes, Int32 userID)
   bei el_ws.DataContainer.GetSelectedGridLayout(String sessionID, GridLayoutContext context, Byte[][] knownHashes)

   --- Ende der internen Ausnahmestapelüberwachung ---

Googling this, I came across a similar stacktrace here, where someone suggests the root cause as being a memory leak. This makes me wonder if I'm using MsgPack.Serialization correctly. Am I supposed to dispose something? Or explicitly store the serializer for later reuse (I understand the repository class does this automatically?)?

[yfakariya]

Thank you for reporting. It looks messy problem...

First, I will answer your last two questions.

1. No, as long as you simply use serializers, you have not to call Dispose explicitly (and you do not have any reference to IDisposable objects except Stream).
2. The repository is SerializationContext. If you do not instantiate and specify it explicitly, the SerializationContext.Default will be used. Or, if you explicitly call new SerializationContext repeatedly, it may cause Loader Heap leak because too many serializer types are created.

I cannot reproduce in simple test case as you say, and there are no evidence to determine whether memory leak occurred and where the leak occurred (it should be loader heap or metadata itself because the TypeBuilder thrown the exception). There are suggestions which I can tell for you:

• If you instantiate SerializationContext repeatedly, it should cause this problem, then you should be able to fix it with using shared SerializationContext instance.
• If you use old .NET Framework, try update it.
• If the failure occurred with loader heap leak, recycling AppPools (processes) should prevent such failure.

[chucker]

Hi,

thanks for the response.

• I use the default SerializationContext, so the instance is presumably shared.
• this is indeed on .NET 3.5. Upgrading is a tough one. Are there significant changes to the internal implementation of serialization (or memory management, perhaps?) that suggest this will help?
• hrm, the pool should recycle every 1740 minutes, as per default. Given how rarely this code is currently used, I'm surprised the problem triggers that frequently. I've tried stresstesting the code by putting it inside a loop, but even after a 100,000 iterations, it doesn't occur while in the debugger. Nor does memory usage go up.

and there are no evidence to determine whether memory leak occurred and where the leak occurred (it should be loader heap or metadata itself because the TypeBuilder thrown the exception).

Can I configure something on my end to narrow this down the next time this occurs?

[chucker]

Also, this was with MsgPack.Cli 0.6.5. Is there a chance this will behave differently in 0.8.0?

[yfakariya]

Hi, thank you for additional information! From the given stacktrace, TypeBuilder._TermCreateClass throws AccessViolationException, this suggests CLR internal error. I think that it might be caused by loader heap was exhausted. This can be verified with performance counter (.NET CLR Loading\Bytes in Loader Heap, for example) or process dump analysis (.loadby sos mscorwks; !dumpheap; etc.).

And unfortunatelly, it should not be related to msgpack logic, so upgrading msgpack will not solve the problem...