Crash in WKS::gc_heap::background_promote (x86, segments)

[jkotas]

Access violation at:

ntdll!KiUserExceptionDispatcher+0xf [d:\rs1\minkernel\ntos\rtl\i386\userdisp.asm @ 597] 
coreclr!WKS::mark_bit_bit_of+0xb [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 9024] 
coreclr!WKS::gc_heap::mark_array_marked+0x16 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 9030] 
coreclr!WKS::gc_heap::background_mark1+0x16 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 27214] 
coreclr!WKS::gc_heap::background_mark+0x2e [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 27234] 
coreclr!WKS::gc_heap::background_mark_simple1+0x6ae [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 28529] 
coreclr!WKS::gc_heap::background_mark_simple+0x5e [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 28637] 
coreclr!WKS::gc_heap::background_promote+0x109 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 28716] 
coreclr!PromoteObject+0xf3 [D:\a\_work\1\s\src\coreclr\gc\objecthandle.cpp @ 351] 

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1173493 Build error leg or test failing: System.Private.Xml.Tests.WorkItemExecution Pull request: https://github.com/dotnet/runtime/pull/120674

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": ["gc_heap::background_mark_simple1","gc_heap::background_promote"],
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1173493 Error message validated: [System.Private.Xml.Tests gc_heap::background_promote] Result validation: :white_check_mark: Known issue matched with the provided build. Validation performed at: 10/17/2025 2:15:29 AM UTC

Report

Build	Definition	Test	Pull Request
1173493	dotnet/runtime	System.Private.Xml.Tests.WorkItemExecution	dotnet/runtime#120674

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	1	1

Known issue validation

Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1173493 Error message validated: [gc_heap::background_mark_simple1 gc_heap::background_promote] Result validation: :white_check_mark: Known issue matched with the provided build. Validation performed at: 10/17/2025 3:17:49 AM UTC

Report

Build	Definition	Test	Pull Request
1173493	dotnet/runtime	System.Private.Xml.Tests.WorkItemExecution	dotnet/runtime#120674

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	1	1

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

[dotnet-policy-service[bot]]

Tagging subscribers to this area: @dotnet/gc See info in area-owners.md if you want to be subscribed.

[jkotas]

Analysis of the crash dump (runfo get-helix-payload -j 39e43d15-a901-4acb-8300-33ca8ab40691 -w System.Private.Xml.Tests -o c:\helix_payload):

• Background thread is trying to set a bit in the mark array for a valid object. The mark array memory is not committed.
• GC heap occupies fairly large chunk of address space with holes (note that this is segments GC on 32-bit x86):

0:000> dt coreclr!g_gc_lowest_address
0x05150000  "???"

0:000> dt coreclr!g_gc_highest_address
0x3b590000  "--- memory read error at address 0x3b590000 ---"

• GC heap is not corrupted:

0:010> !verifyheap
521,980 objects verified, 0 errors.
No heap corruption detected.

This looks like a bug or a race condition in committing of the mark_array.

[jkotas]

Small object heap
 segment    begin allocated committed allocated size       committed size      
05150000 05151000  0614e980  06150000 0xffd980 (16767360)  0x1000000 (16777216)
11a60000 11a61000  125f0c24  12611000 0xb8fc24 (12123172)  0xbb1000 (12259328) 
1b210000 1b211000  1ba64bd4  1ba85000 0x853bd4 (8731604)   0x875000 (8867840)  
1a210000 1a211000  1aadf930  1aae9000 0x8ce930 (9234736)   0x8d9000 (9277440) <--- mark_array is not committed for 1a5bd6f4 
NonGC heap
 segment    begin allocated committed allocated size       committed size      
00b6f978 08850004  089e53d8  089f0000 0x1953d4 (1659860)   0x1a0000 (1703936)  
Large object heap starts at 6151000
 segment    begin allocated committed allocated size       committed size      
06150000 06151000  0714e040  0714f000 0xffd040 (16764992)  0xfff000 (16773120) 
1c210000 1c211000  1e19af78  1e19b000 0x1f89f78 (33070968) 0x1f8b000 (33075200)
1e210000 1e211000  201e0fa0  201e1000 0x1fcffa0 (33357728) 0x1fd1000 (33361920)
20210000 20211000  242008f0  24201000 0x3fef8f0 (67041520) 0x3ff1000 (67047424)
25160000 25161000  271022e0  27103000 0x1fa12e0 (33166048) 0x1fa3000 (33173504)
27160000 27161000  29d61060  29d62000 0x2c00060 (46137440) 0x2c02000 (46145536)
2a160000 2a161000  2bacf420  2bad0000 0x196e420 (26666016) 0x1970000 (26673152)
2c160000 2c161000  2db9ae08  2db9b000 0x1a39e08 (27500040) 0x1a3b000 (27504640)
2e160000 2e161000  2f473d20  2f474000 0x1312d20 (20000032) 0x1314000 (20004864)
30160000 30161000  3343dd78  3343e000 0x32dcd78 (53333368) 0x32de000 (53338112)
34160000 34161000  35161020  35162000 0x1000020 (16777248) 0x1002000 (16785408)
36160000 36161000  38161020  38162000 0x2000020 (33554464) 0x2002000 (33562624)
39160000 39161000  3aacf6e8  3aad0000 0x196e6e8 (26666728) 0x1970000 (26673152)
Pinned object heap starts at 7151000
 segment    begin allocated committed allocated size       committed size      
07150000 07151000  071581e0  07162000 0x71e0 (29152)       0x12000 (73728)     

[jkotas]

We have number of reports of won't fixed / no repro crashes with very similar symptoms, for example: https://github.com/dotnet/runtime/issues/60825#issuecomment-952158803 . This is likely a very old bug that is hit by the CI a few times per year.

[mangod9]

since this looks to be only reproing on 32-bit most likely an existing bug in segments.

[jeffschwMSFT]

removing blocking-clean-ci as it has not failed in 30 days

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0