dotnet stack frames for WER dumps are marked as 'unknown_image'

[WizzardMaker]

Environment

Version: 23.11.2 Windows 11 .Net 8 (But also tested on .Net Framework)

.symboliclirc

cache_dir = "./cache"

# The public Microsoft symbol server
[[sources]]
id = "microsoft"
type = "http"
url = "https://msdl.microsoft.com/download/symbols"
layout = { type = "ssqp" }
filters = { filetypes = ["pe", "pdb", "portablepdb"] }
is_public = true

[[sources]]
id = "local"
type = "filesystem"
path = "./custom_binaries/"
layout = { type = "unified", casing = "default" }

Steps to Reproduce

1. Create dummy .Net application that just throws an exception
2. Fetch dump from WER (%LOCALAPPDATA%\CrashDumps) 2.1 Adding the debug symbols from the build does not change the output except for dotnet host unmanaged code
3. Analyze with: .\symbolicli.exe --offline .\crash.dmp

Expected Result

I would have expected those frames to contain the correct information about my dotnet functions.

Actual Result

The CLI outputs for dotnet stack frames, an 'unknown_image' frame:

{
  "status": "unknown_image",
  "original_index": 3,
  "instruction_addr": "0x7ffbead25647",
  "trust": "cfi"
},

I know that this is the dotnet frame due to its surrounding frames.

The full output of that dump is here: test.json

Example build with crashing application is here: build.zip

Example dump of that build is here: CrashTestApp.exe.25308.dmp

[loewenheim]

I've looked into this a bit. The frame in question has no package, and we don't extract any module from the minidump that covers address 0x7ffbead25647. This will need further investigation.

[WizzardMaker]

Just to make sure it wasn't related to the minidump itself or etc. I checked with dotnet-dump whether there is actually enough information to create a full stack trace. But dotnet-dump seems to be able to correctly identify the stack at 0x7ffbead25647

Loading core dump: .\CrashTestApp.exe.25308.dmp ...
OS Thread Id: 0x7fe8
        Child SP               IP Call Site
000000010017E7D8                  [HelperMethodFrame: 000000010017e7d8]
000000010017E8D0 00007FFBEAD25647 CrashTestApp.dll!CrashTestApp.Program.Main() + 135
OS Thread Id: 0x3f24
        Child SP               IP Call Site
0000000100CFF960                  [DebuggerU2MCatchHandlerFrame: 0000000100cff960]
OS Thread Id: 0x90
        Child SP               IP Call Site
0000000100E7F7F0                  [DebuggerU2MCatchHandlerFrame: 0000000100e7f7f0]
OS Thread Id: 0x2614
        Child SP               IP Call Site
0000000100FFF308                  [HelperMethodFrame: 0000000100fff308] System.Private.CoreLib.dll!System.Threading.WaitHandle.WaitOneCore(IntPtr, Int32)
0000000100FFF410 00007FFC49DA1614 System.Private.CoreLib.dll!System.Threading.WaitHandle.WaitOneNoCheck(Int32) + 180 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/WaitHandle.cs @ 128]
0000000100FFF470 00007FFC49EAEC0C System.Private.CoreLib.dll!System.Diagnostics.Tracing.CounterGroup.PollForValues() + 540 [/_/src/libraries/System.Private.CoreLib/src/System/Diagnostics/Tracing/CounterGroup.cs @ 305]
0000000100FFF760                  [DebuggerU2MCatchHandlerFrame: 0000000100fff760]
OS Thread Id: 0x7cdc
        Child SP               IP Call Site
000000010117F480                  [InlinedCallFrame: 000000010117f480]
000000010117F480                  [InlinedCallFrame: 000000010117f480]
000000010117F450 00007FFC49C977FA System.Private.CoreLib.dll!Interop+Kernel32.GetQueuedCompletionStatus(IntPtr, UInt32 ByRef, UIntPtr ByRef, IntPtr ByRef, Int32) + 122 [/_/src/coreclr/System.Private.CoreLib/Microsoft.Interop.LibraryImportGenerator/Microsoft.Interop.LibraryImportGenerator/LibraryImports.g.cs @ 4997]
000000010117F550 00007FFC49DBD271 System.Private.CoreLib.dll!System.Threading.LowLevelLifoSemaphore.WaitForSignal(Int32) + 81 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/LowLevelLifoSemaphore.cs @ 140]
000000010117F5C0 00007FFC49DBD18B System.Private.CoreLib.dll!System.Threading.LowLevelLifoSemaphore.Wait(Int32, Boolean) + 299 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/LowLevelLifoSemaphore.cs @ 65]
000000010117F620 00007FFC49DBCBD1 System.Private.CoreLib.dll!System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart() + 449 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.WorkerThread.NonBrowser.cs @ 100]
000000010117F970                  [DebuggerU2MCatchHandlerFrame: 000000010117f970]
OS Thread Id: 0x7cc0
        Child SP               IP Call Site
00000001001BF468                  [HelperMethodFrame: 00000001001bf468] System.Private.CoreLib.dll!System.Threading.WaitHandle.WaitOneCore(IntPtr, Int32)
00000001001BF570 00007FFC49DA1614 System.Private.CoreLib.dll!System.Threading.WaitHandle.WaitOneNoCheck(Int32) + 180 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/WaitHandle.cs @ 128]
00000001001BF5D0 00007FFC49DB9921 System.Private.CoreLib.dll!System.Threading.PortableThreadPool+GateThread.GateThreadStart() + 321 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.GateThread.cs @ 54]
00000001001BF960                  [DebuggerU2MCatchHandlerFrame: 00000001001bf960]
OS Thread Id: 0x7d8c
        Child SP               IP Call Site
00000001012FF250                  [InlinedCallFrame: 00000001012ff250]
00000001012FF250                  [InlinedCallFrame: 00000001012ff250]
00000001012FF220 00007FFC49C977FA System.Private.CoreLib.dll!Interop+Kernel32.GetQueuedCompletionStatus(IntPtr, UInt32 ByRef, UIntPtr ByRef, IntPtr ByRef, Int32) + 122 [/_/src/coreclr/System.Private.CoreLib/Microsoft.Interop.LibraryImportGenerator/Microsoft.Interop.LibraryImportGenerator/LibraryImports.g.cs @ 4997]
00000001012FF320 00007FFC49DBD271 System.Private.CoreLib.dll!System.Threading.LowLevelLifoSemaphore.WaitForSignal(Int32) + 81 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/LowLevelLifoSemaphore.cs @ 140]
00000001012FF390 00007FFC49DBD18B System.Private.CoreLib.dll!System.Threading.LowLevelLifoSemaphore.Wait(Int32, Boolean) + 299 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/LowLevelLifoSemaphore.cs @ 65]
00000001012FF3F0 00007FFC49DBCBD1 System.Private.CoreLib.dll!System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart() + 449 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.WorkerThread.NonBrowser.cs @ 100]
00000001012FF740                  [DebuggerU2MCatchHandlerFrame: 00000001012ff740]
OS Thread Id: 0x90b0
        Child SP               IP Call Site
000000010147F5F0                  [InlinedCallFrame: 000000010147f5f0]
000000010147F5F0                  [InlinedCallFrame: 000000010147f5f0]
000000010147F5C0 00007FFC49C977FA System.Private.CoreLib.dll!Interop+Kernel32.GetQueuedCompletionStatus(IntPtr, UInt32 ByRef, UIntPtr ByRef, IntPtr ByRef, Int32) + 122 [/_/src/coreclr/System.Private.CoreLib/Microsoft.Interop.LibraryImportGenerator/Microsoft.Interop.LibraryImportGenerator/LibraryImports.g.cs @ 4997]
000000010147F6C0 00007FFC49DBD271 System.Private.CoreLib.dll!System.Threading.LowLevelLifoSemaphore.WaitForSignal(Int32) + 81 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/LowLevelLifoSemaphore.cs @ 140]
000000010147F730 00007FFC49DBD18B System.Private.CoreLib.dll!System.Threading.LowLevelLifoSemaphore.Wait(Int32, Boolean) + 299 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/LowLevelLifoSemaphore.cs @ 65]
000000010147F790 00007FFC49DBCBD1 System.Private.CoreLib.dll!System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart() + 449 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.WorkerThread.NonBrowser.cs @ 100]
000000010147FAE0                  [DebuggerU2MCatchHandlerFrame: 000000010147fae0]
OS Thread Id: 0x7710
        Child SP               IP Call Site
00000001001FF520                  [InlinedCallFrame: 00000001001ff520]
00000001001FF520                  [InlinedCallFrame: 00000001001ff520]
00000001001FF4E0 00007FFC49DBB720 System.Private.CoreLib.dll!System.Threading.PortableThreadPool+IOCompletionPoller.Poll() + 112 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.IO.Windows.cs @ 188]
00000001001FF840                  [DebuggerU2MCatchHandlerFrame: 00000001001ff840]
OS Thread Id: 0x5fdc
        Child SP               IP Call Site
00000001015FF3B0                  [InlinedCallFrame: 00000001015ff3b0]
00000001015FF3B0                  [InlinedCallFrame: 00000001015ff3b0]
00000001015FF380 00007FFC49C977FA System.Private.CoreLib.dll!Interop+Kernel32.GetQueuedCompletionStatus(IntPtr, UInt32 ByRef, UIntPtr ByRef, IntPtr ByRef, Int32) + 122 [/_/src/coreclr/System.Private.CoreLib/Microsoft.Interop.LibraryImportGenerator/Microsoft.Interop.LibraryImportGenerator/LibraryImports.g.cs @ 4997]
00000001015FF480 00007FFC49DBD271 System.Private.CoreLib.dll!System.Threading.LowLevelLifoSemaphore.WaitForSignal(Int32) + 81 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/LowLevelLifoSemaphore.cs @ 140]
00000001015FF4F0 00007FFC49DBD18B System.Private.CoreLib.dll!System.Threading.LowLevelLifoSemaphore.Wait(Int32, Boolean) + 299 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/LowLevelLifoSemaphore.cs @ 65]
00000001015FF550 00007FFC49DBCBD1 System.Private.CoreLib.dll!System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart() + 449 [/_/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.WorkerThread.NonBrowser.cs @ 100]
00000001015FF8A0                  [DebuggerU2MCatchHandlerFrame: 00000001015ff8a0]

[WizzardMaker]

I think the main problem is, that the C# module gets JITed to the heap, where 0x7ffbead25647 probably is located - but that is just a wild guess and I don't quite know about the available metadata to connect JIT heap and module symbols

[loewenheim]

Thanks for the additional details, that's very helpful. It might take us a bit to get to the bottom of this.

[Swatinem]

This might actually be the case that we fundamentally do not support .NET minidumps.