AccessViolationException in ArrayDictionary

[KirillOsenkov]

Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at Microsoft.Build.Collections.ArrayDictionary`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Add(System.__Canon, System.__Canon)
   at Microsoft.Build.Logging.StructuredLogger.BuildEventArgsReader.CreateDictionary(System.Collections.Generic.List`1<System.ValueTuple`2<Int32,Int32>>)
   at Microsoft.Build.Logging.StructuredLogger.BuildEventArgsReader.ReadNameValueList()
   at Microsoft.Build.Logging.StructuredLogger.BuildEventArgsReader.PreprocessRecordsTillNextEvent(System.Func`2<Microsoft.Build.Logging.StructuredLogger.BinaryLogRecordKind,Boolean>)
   at Microsoft.Build.Logging.StructuredLogger.BuildEventArgsReader.Read()
   at Microsoft.Build.Logging.StructuredLogger.BinLogReader.Replay(System.IO.Stream, Microsoft.Build.Logging.StructuredLogger.Progress)
   at Microsoft.Build.Logging.StructuredLogger.BinaryLog.ReadBuild(System.IO.Stream, Microsoft.Build.Logging.StructuredLogger.Progress, Byte[], Microsoft.Build.Logging.StructuredLogger.ReaderSettings)
   at Microsoft.Build.Logging.StructuredLogger.BinaryLog.ReadBuild(System.String, Microsoft.Build.Logging.StructuredLogger.Progress, Microsoft.Build.Logging.StructuredLogger.ReaderSettings)
   at Microsoft.Build.Logging.StructuredLogger.BinaryLog.ReadBuild(System.String, Microsoft.Build.Logging.StructuredLogger.Progress)
   at Microsoft.Build.Logging.StructuredLogger.BinaryLog.ReadBuild(System.String)
   at BinlogTool.BinlogToolCommandBase.ReadBuild(System.String, Boolean)
   at BinlogTool.ListNuget.Run(System.String, System.String)
   at BinlogTool.Program.Main(System.String[])

[BenjaminBrienen]

We also encountered this

##[error]Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
at Microsoft.Build.Logging.StructuredLogger.BuildEventArgsReader.CreateDictionary(System.Collections.Generic.List`1<System.ValueTuple`2<Int32,Int32>>)
at Microsoft.Build.Logging.StructuredLogger.BuildEventArgsReader.ReadNameValueList()
at Microsoft.Build.Logging.StructuredLogger.BuildEventArgsReader.Read()
at Microsoft.Build.Logging.StructuredLogger.BinLogReader.Replay(System.IO.Stream, Microsoft.Build.Logging.StructuredLogger.Progress)
at Microsoft.Build.Logging.StructuredLogger.BinaryLog.ReadBuild(System.IO.Stream, Microsoft.Build.Logging.StructuredLogger.Progress, Byte[], Microsoft.Build.Logging.StructuredLogger.ReaderSettings)
at Microsoft.Build.Logging.StructuredLogger.BinaryLog.ReadBuild(System.String, Microsoft.Build.Logging.StructuredLogger.Progress, Microsoft.Build.Logging.StructuredLogger.ReaderSettings)
at Microsoft.Build.Logging.StructuredLogger.Serialization.Read(System.String, Microsoft.Build.Logging.StructuredLogger.Progress, Microsoft.Build.Logging.StructuredLogger.ReaderSettings)
at Microsoft.Build.Logging.StructuredLogger.Serialization.Read(System.String) 

[KirillOsenkov]

I don't have the foggiest idea what could be causing an AV here: https://github.com/KirillOsenkov/MSBuildStructuredLog/blob/ba0729bd2a130eab7edcacaf00ac3be724a07abd/src/StructuredLogger/BinaryLogger/BuildEventArgsReader.Viewer.cs#L28

Looks like a .NET runtime bug honestly, since it's very rare and intermittent.

Reading a binlog is an incredibly CPU intensive operation with a ton of parallelism that is saturating your CPUs. I can imagine this bug only surfacing under extreme load.

Or am doing something stupid somewhere, but I just can't see what it could be.

[KirillOsenkov]

If possible, try to capture a dump of the process state when this happens.

You can use this registry setting and for every crashed process a dump will appear in C:\CrashDumps: https://github.com/KirillOsenkov/dotfiles/blob/main/LocalDumps.reg

Don't forget to create the C:\CrashDumps directory first, of course.

Once we have a dump, we can look at what's going on using a native debugger. A managed debugger alone won't show what's going on here.

[KirillOsenkov]

I forgot that I opened the bug initially. Well, I suppose I'll remember to grab a dump next time I see this.

[KirillOsenkov]

The exact crash is in the Add method: https://github.com/KirillOsenkov/MSBuildStructuredLog/blob/ba0729bd2a130eab7edcacaf00ac3be724a07abd/src/StructuredLogger/BinaryLogger/BuildEventArgsReader.Viewer.cs#L35