[6.0, 7.0] Assert failed in coreclr linux arm: GCHeapUtilities::GetGCHeap()->IsInFrozenSegment(obj)

[carlossanlop]

Error Blob

{
  "ErrorMessage": "GCHeapUtilities::GetGCHeap()->IsInFrozenSegment(obj)",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": false
}

Reproduction Steps

• PR: https://github.com/dotnet/runtime/pull/84359
• Queue: Libraries Test Run checked coreclr Linux arm Release
• Job: https://dev.azure.com/dnceng-public/public/_build/results?buildId=228808&view=logs&j=2845845c-bfb0-5f35-d712-13f6aa4acce2&t=d706022b-76ab-5f5d-9311-7aa8b18efa8a
• Log file: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-84359-merge-200e9be78c9345af97/System.Text.Json.Tests/1/console.d6363488.log?helixlogtype=result
• Output:

===========================================================================================================
/root/helix/work/workitem/e /root/helix/work/workitem/e
  Discovering: System.Text.Json.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Text.Json.Tests (found 2792 of 2854 test cases)
  Starting:    System.Text.Json.Tests (parallel test collections = on, max threads = 2)

Assert failure(PID 25 [0x00000019], Thread: 35 [0x0023]): GCHeapUtilities::GetGCHeap()->IsInFrozenSegment(obj)
    File: /__w/1/s/src/coreclr/vm/syncblk.cpp Line: 2059
    Image: /root/helix/work/correlation/dotnet

./RunTests.sh: line 162:    25 Aborted                 (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Text.Json.Tests.runtimeconfig.json --depsfile System.Text.Json.Tests.deps.json xunit.console.dll System.Text.Json.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/root/helix/work/workitem/e
----- end Wed Apr 5 18:34:11 UTC 2023 ----- exit code 134 ----------------------------------------------------------

Report

Build	Definition	Test	Pull Request
532142	dotnet/runtime	System.Collections.Immutable.Tests.WorkItemExecution	dotnet/runtime#97089
529592	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	dotnet/runtime#97063
521674	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	dotnet/runtime#96759

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	2	3

Known issue validation

Build: :mag_right: Result validation: :warning: Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: :mag_right:" line. Validation performed at: 8/1/2023 10:02:52 PM UTC

[ghost]

Tagging subscribers to this area: @dotnet/gc See info in area-owners.md if you want to be subscribed.

Issue Details

---

Error Blob

{
  "ErrorMessage": "GCHeapUtilities::GetGCHeap()->IsInFrozenSegment(obj)",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": false
}

Reproduction Steps

• Queue: Libraries Test Run checked coreclr Linux arm Release
• Job: https://dev.azure.com/dnceng-public/public/_build/results?buildId=228808&view=logs&j=2845845c-bfb0-5f35-d712-13f6aa4acce2&t=d706022b-76ab-5f5d-9311-7aa8b18efa8a
• Log file: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-84359-merge-200e9be78c9345af97/System.Text.Json.Tests/1/console.d6363488.log?helixlogtype=result
• Output:

===========================================================================================================
/root/helix/work/workitem/e /root/helix/work/workitem/e
  Discovering: System.Text.Json.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Text.Json.Tests (found 2792 of 2854 test cases)
  Starting:    System.Text.Json.Tests (parallel test collections = on, max threads = 2)

Assert failure(PID 25 [0x00000019], Thread: 35 [0x0023]): GCHeapUtilities::GetGCHeap()->IsInFrozenSegment(obj)
    File: /__w/1/s/src/coreclr/vm/syncblk.cpp Line: 2059
    Image: /root/helix/work/correlation/dotnet

./RunTests.sh: line 162:    25 Aborted                 (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Text.Json.Tests.runtimeconfig.json --depsfile System.Text.Json.Tests.deps.json xunit.console.dll System.Text.Json.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/root/helix/work/workitem/e
----- end Wed Apr 5 18:34:11 UTC 2023 ----- exit code 134 ----------------------------------------------------------

Author:	carlossanlop
Assignees:	-
Labels:	arch-arm32, os-linux, area-GC-coreclr, blocking-clean-ci, runtime-coreclr, Known Build Error
Milestone:	6.0.x

[trylek]

According to offline confirmation from @carlossanlop, this is not a new bug, the staging branch has just been populated with the content of the .NET 6 release branch. It would be great if someone on the @dotnet/gc team could share their thoughts regarding severity of this bug and / or its impact on .NET 7 / 8 in the sense of whether it's something that has since been fixed but not backported or whether it's a previously unseen rare failure we just hit for the first time.

[Maoni0]

could be https://github.com/dotnet/runtime/issues/82414 which I made a fix for in 8.0 (it has existed since forever). it does not affect running normally, only affects object validation which is only for debug builds.

[Maoni0]

do we want to do something about this? would blocking ci be a reason to backport?

[mangod9]

probably make sense to port if its now failing more consistently. But looks like it has only a couple of hits so far. @carlossanlop is this causing any issues with 6.0 servicing release?

[carlossanlop]

Just hit it in a 7.0 PR, so I changed the title:

• 7.0 PR: https://github.com/dotnet/runtime/pull/89801
• Queue: Libraries Test Run checked coreclr Linux_musl arm Release
• Affected test: System.Text.Json.Tests
• Run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=359343&view=logs&j=62876419-b96e-5483-94d3-b47175c1afe7&t=69034341-3a68-59aa-c247-b3e7d2608abb&l=56
• Log file: https://helix.dot.net/api/2019-06-17/jobs/44bd6355-46b9-4749-8ec8-cc3117231cb2/workitems/System.Text.Json.Tests/console
• Failure message:

Assert failure(PID 23 [0x00000017], Thread: 44 [0x002c]): GCHeapUtilities::GetGCHeap()->IsInFrozenSegment(obj)

[jeffschwMSFT]

removing blocking-clean-ci as it has not failed in 30 days

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0