runtime Test failure: AggregateException: Not all processes in process tree could be terminated.

Failed in: runtime-coreclr jitstress 20231009.2

Failed tests:

coreclr osx x64 Checked jitstress2 @ OSX.1200.Amd64.Open
    - baseservices/exceptions/simple/ParallelCrashMainThread/ParallelCrashMainThread.sh
    - baseservices/exceptions/simple/ParallelCrash/ParallelCrash.sh
    - baseservices/exceptions/unhandled/unhandled/unhandled.sh

Error message:

Test Infrastructure Failure: System.AggregateException: Not all processes in process tree could be terminated. (Operation not permitted)
 ---> System.ComponentModel.Win32Exception (1): Operation not permitted
   --- End of inner exception stack trace ---
   at System.Diagnostics.Process.Kill(Boolean entireProcessTree)
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDumpWithCreateDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.RunTest(String executable, String outputFile, String errorFile, String category, String testBinaryBase, String outputDir)
   at baseservices_exceptions._simple_ParallelCraMainThread_ParallelCraMainThread_._simple_ParallelCrashMainThread_ParallelCrashMainThread_sh()
Expected: True
Actual:   False

Stack trace:

   at baseservices_exceptions._simple_ParallelCraMainThread_ParallelCraMainThread_._simple_ParallelCrashMainThread_ParallelCrashMainThread_sh()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

Queued	Pipeline	Pipeline_Configuration	OS	Arch	Test	Outcome	Properties	BuildReason
2023-10-10T05:50:17.995Z	runtime-coreclr jitstress	Checked-jitstress2	osx.1200.amd64.open	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"BuildNumber":"20231009.2","DefinitionId":"109","Project":"public","System.PhaseName":"run_test_p1__osx_x64_checked","operatingSystem":"OSX.1200.Amd64.Open","System.JobName":"__default","DefinitionName":"runtime-coreclr jitstress","System.StageName":"Build","BuildId":"432736","architecture":"x64","configuration":"Checked-jitstress2","System.PhaseAttempt":"1","AzurePipelinesTestRunId":"9585790","System.JobAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.StageAttempt":"1","Reason":"Schedule"}	Schedule
2023-10-10T05:50:17.995Z	runtime-coreclr jitstress	Checked-jitstress2	osx.1200.amd64.open	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"BuildNumber":"20231009.2","DefinitionId":"109","Project":"public","System.PhaseName":"run_test_p1__osx_x64_checked","operatingSystem":"OSX.1200.Amd64.Open","System.JobName":"__default","DefinitionName":"runtime-coreclr jitstress","System.StageName":"Build","BuildId":"432736","architecture":"x64","configuration":"Checked-jitstress2","System.PhaseAttempt":"1","AzurePipelinesTestRunId":"9585790","System.JobAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.StageAttempt":"1","Reason":"Schedule"}	Schedule
2023-10-10T05:50:17.995Z	runtime-coreclr jitstress	Checked-jitstress2	osx.1200.amd64.open	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"BuildNumber":"20231009.2","DefinitionId":"109","Project":"public","System.PhaseName":"run_test_p1__osx_x64_checked","operatingSystem":"OSX.1200.Amd64.Open","System.JobName":"__default","DefinitionName":"runtime-coreclr jitstress","System.StageName":"Build","BuildId":"432736","architecture":"x64","configuration":"Checked-jitstress2","System.PhaseAttempt":"1","AzurePipelinesTestRunId":"9585790","System.JobAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.StageAttempt":"1","Reason":"Schedule"}	Schedule
2023-10-10T05:50:17.995Z	runtime-coreclr jitstress	Checked-jitstress2	osx.1200.amd64.open	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"BuildNumber":"20231009.2","DefinitionId":"109","Project":"public","System.PhaseName":"run_test_p1__osx_x64_checked","operatingSystem":"OSX.1200.Amd64.Open","System.JobName":"__default","DefinitionName":"runtime-coreclr jitstress","System.StageName":"Build","BuildId":"432736","architecture":"x64","configuration":"Checked-jitstress2","System.PhaseAttempt":"1","AzurePipelinesTestRunId":"9585790","System.JobAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.StageAttempt":"1","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule

More failures

Queued	Pipeline	Pipeline_Configuration	OS	Arch	Test	Outcome	Properties	BuildReason
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule

### Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": [ "Not all processes in process tree could be terminated.", "Operation not permitted" ],
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: :mag_right: Result validation: :warning: Provided build not found. Provide a valid build in the "Build: :mag_right:" line. Validation performed at: 4/2/2024 6:55:27 AM UTC

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

Oct 11 '23 01:10 v-wenyuxu

Tagging subscribers to this area: @dotnet/area-system-diagnostics-process See info in area-owners.md if you want to be subscribed.

Issue Details

Failed in: runtime-coreclr jitstress 20231009.2

Failed tests:

coreclr osx x64 Checked jitstress2 @ OSX.1200.Amd64.Open
    - baseservices/exceptions/simple/ParallelCrashMainThread/ParallelCrashMainThread.sh
    - baseservices/exceptions/simple/ParallelCrash/ParallelCrash.sh
    - baseservices/exceptions/unhandled/unhandled/unhandled.sh

Error message:

Test Infrastructure Failure: System.AggregateException: Not all processes in process tree could be terminated. (Operation not permitted)
 ---> System.ComponentModel.Win32Exception (1): Operation not permitted
   --- End of inner exception stack trace ---
   at System.Diagnostics.Process.Kill(Boolean entireProcessTree)
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDumpWithCreateDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.RunTest(String executable, String outputFile, String errorFile, String category, String testBinaryBase, String outputDir)
   at baseservices_exceptions._simple_ParallelCraMainThread_ParallelCraMainThread_._simple_ParallelCrashMainThread_ParallelCrashMainThread_sh()
Expected: True
Actual:   False

Stack trace:

   at baseservices_exceptions._simple_ParallelCraMainThread_ParallelCraMainThread_._simple_ParallelCrashMainThread_ParallelCrashMainThread_sh()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

Queued	Pipeline	Pipeline_Configuration	OS	Arch	Test	Outcome	Properties	BuildReason
2023-10-10T05:50:17.995Z	runtime-coreclr jitstress	Checked-jitstress2	osx.1200.amd64.open	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"BuildNumber":"20231009.2","DefinitionId":"109","Project":"public","System.PhaseName":"run_test_p1__osx_x64_checked","operatingSystem":"OSX.1200.Amd64.Open","System.JobName":"__default","DefinitionName":"runtime-coreclr jitstress","System.StageName":"Build","BuildId":"432736","architecture":"x64","configuration":"Checked-jitstress2","System.PhaseAttempt":"1","AzurePipelinesTestRunId":"9585790","System.JobAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.StageAttempt":"1","Reason":"Schedule"}	Schedule
2023-10-10T05:50:17.995Z	runtime-coreclr jitstress	Checked-jitstress2	osx.1200.amd64.open	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"BuildNumber":"20231009.2","DefinitionId":"109","Project":"public","System.PhaseName":"run_test_p1__osx_x64_checked","operatingSystem":"OSX.1200.Amd64.Open","System.JobName":"__default","DefinitionName":"runtime-coreclr jitstress","System.StageName":"Build","BuildId":"432736","architecture":"x64","configuration":"Checked-jitstress2","System.PhaseAttempt":"1","AzurePipelinesTestRunId":"9585790","System.JobAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.StageAttempt":"1","Reason":"Schedule"}	Schedule
2023-10-10T05:50:17.995Z	runtime-coreclr jitstress	Checked-jitstress2	osx.1200.amd64.open	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"BuildNumber":"20231009.2","DefinitionId":"109","Project":"public","System.PhaseName":"run_test_p1__osx_x64_checked","operatingSystem":"OSX.1200.Amd64.Open","System.JobName":"__default","DefinitionName":"runtime-coreclr jitstress","System.StageName":"Build","BuildId":"432736","architecture":"x64","configuration":"Checked-jitstress2","System.PhaseAttempt":"1","AzurePipelinesTestRunId":"9585790","System.JobAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.StageAttempt":"1","Reason":"Schedule"}	Schedule
2023-10-10T05:50:17.995Z	runtime-coreclr jitstress	Checked-jitstress2	osx.1200.amd64.open	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"BuildNumber":"20231009.2","DefinitionId":"109","Project":"public","System.PhaseName":"run_test_p1__osx_x64_checked","operatingSystem":"OSX.1200.Amd64.Open","System.JobName":"__default","DefinitionName":"runtime-coreclr jitstress","System.StageName":"Build","BuildId":"432736","architecture":"x64","configuration":"Checked-jitstress2","System.PhaseAttempt":"1","AzurePipelinesTestRunId":"9585790","System.JobAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.StageAttempt":"1","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule

More failures

Queued	Pipeline	Pipeline_Configuration	OS	Arch	Test	Outcome	Properties	BuildReason
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:17.734Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.StageAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionName":"runtime-coreclr runincontext","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseAttempt":"1","configuration":"Checked-no_tiered_compilation","AzurePipelinesTestRunId":"9537288","BuildNumber":"20231008.1","System.JobAttempt":"1","System.JobName":"__default","architecture":"x64","BuildId":"431261","System.StageName":"Build","DefinitionId":"121","Project":"public","Reason":"Schedule"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-08T14:21:16.532Z	runtime-coreclr runincontext	Checked	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","System.JobAttempt":"1","Reason":"Schedule","configuration":"Checked","DefinitionName":"runtime-coreclr runincontext","System.PhaseAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","System.StageName":"Build","BuildId":"431261","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","AzurePipelinesTestRunId":"9537286","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"x64","Project":"public","System.JobName":"__default","BuildNumber":"20231008.1","DefinitionId":"121","System.StageAttempt":"1"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule
2023-10-07T14:20:57.576Z	runtime-coreclr runincontext	Checked-no_tiered_compilation	ubuntu.2204.amd64.open.rt	x64	baseservices/exceptions/unhandled/unhandled/unhandled.sh	Failed	{"AzurePipelinesTestRunId":"9526074","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildId":"430819","System.PhaseName":"run_test_p1_RunInContext_linux_x64_checked","DefinitionId":"121","System.StageName":"Build","System.JobName":"__default","BuildNumber":"20231007.1","Project":"public","architecture":"x64","System.StageAttempt":"1","System.JobAttempt":"1","operatingSystem":"Ubuntu.2204.Amd64.Open","configuration":"Checked-no_tiered_compilation","System.JobId":"83aa44ae-94d8-5d3d-7684-a0cf5cb0d108","System.PhaseAttempt":"1","Reason":"Schedule","DefinitionName":"runtime-coreclr runincontext"}	Schedule

Author:	v-wenyuxu
Assignees:	-
Labels:	`area-System.Diagnostics.Process`, `os-mac-os-x`, `JitStress`, `arch-x64`, `untriaged`, `needs-area-label`
Milestone:	-

Oct 12 '23 16:10 ghost

Failed in: runtime-coreclr r2r-extra 20240224.1

Failed tests:

R2R-CG2 linux arm Checked jitstressregs4 @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd

Error message:

 Test Infrastructure Failure: System.AggregateException: Not all processes in process tree could be terminated. (Operation not permitted)
 ---> System.ComponentModel.Win32Exception (1): Operation not permitted
   --- End of inner exception stack trace ---
   at System.Diagnostics.Process.Kill(Boolean entireProcessTree) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.NonUap.cs:line 31
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDumpWithCreateDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.RunTest(String executable, String outputFile, String errorFile, String category, String testBinaryBase, String outputDir)
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)

Stack trace:

   at Xunit.Assert.Fail(String message) in /_/src/Microsoft.DotNet.XUnitAssert/src/FailAsserts.cs:line 34
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor251|25_252(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass25_0&)

Feb 26 '24 04:02 v-wenyuxu

Failed in: runtime-coreclr r2r-extra 20240316.1

Failed tests:

R2R-CG2 linux arm Checked jitstress1 @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked gcstress0xc @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd

Error message:

 Test Infrastructure Failure: System.AggregateException: Not all processes in process tree could be terminated. (Operation not permitted)
 ---> System.ComponentModel.Win32Exception (1): Operation not permitted
   --- End of inner exception stack trace ---
   at System.Diagnostics.Process.Kill(Boolean entireProcessTree) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.NonUap.cs:line 31
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDumpWithCreateDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.RunTest(String executable, String outputFile, String errorFile, String category, String testBinaryBase, String outputDir)
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)

Stack trace:

   at Xunit.Assert.Fail(String message) in /_/src/Microsoft.DotNet.XUnitAssert/src/FailAsserts.cs:line 34
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor251|25_252(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass25_0&)

Mar 18 '24 01:03 v-wenyuxu

@mangod9, what is the status of this failure?

Mar 18 '24 17:03 mikelle-rogers

seems to be only affecting jitstress. Did you notice this failure on a PR run?

Mar 18 '24 17:03 mangod9

One of the vendors noticed it here, as seen in the above comment.

Mar 18 '24 21:03 mikelle-rogers

Failed in: runtime-coreclr r2r-extra 20240323.1

Failed tests:

R2R-CG2 linux arm Checked jitstress2_tiered @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked gcstress0xc @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd

Error message:

 Test Infrastructure Failure: System.AggregateException: Not all processes in process tree could be terminated. (Operation not permitted)
 ---> System.ComponentModel.Win32Exception (1): Operation not permitted
   --- End of inner exception stack trace ---
   at System.Diagnostics.Process.Kill(Boolean entireProcessTree) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.NonUap.cs:line 31
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDumpWithCreateDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.RunTest(String executable, String outputFile, String errorFile, String category, String testBinaryBase, String outputDir)
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)

Stack trace:

   at Xunit.Assert.Fail(String message) in /_/src/Microsoft.DotNet.XUnitAssert/src/FailAsserts.cs:line 34
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor251|25_252(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass25_0&)

Mar 25 '24 01:03 v-wenyuxu

This is hit intermittently on both macOS and Linux, different architectures, for different tests. I have just resolved #99885 as a duplicate.

The problem seems to be a race condition in the implementation of System.Diagnostic.Process.Kill(entireProcessTree: true).

The processes in the tree can be starting and exiting while Kill(entireProcessTree: true) is trying to kill them all. There seems to be some specific pattern where the process is found, but it exits on its own as it is getting killed, that results into Operation not permitted error and exception.

Apr 02 '24 07:04 jkotas

Failed in: runtime-coreclr r2r 20240401.1

Failed tests:

R2R-CG2 linux arm Checked no_tiered_compilation @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd

Error message:

 Test Infrastructure Failure: System.AggregateException: Not all processes in process tree could be terminated. (Operation not permitted)
 ---> System.ComponentModel.Win32Exception (1): Operation not permitted
   --- End of inner exception stack trace ---
   at System.Diagnostics.Process.Kill(Boolean entireProcessTree) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.NonUap.cs:line 31
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDumpWithCreateDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.RunTest(String executable, String outputFile, String errorFile, String category, String testBinaryBase, String outputDir)
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)

Stack trace:

    at Xunit.Assert.Fail(String message) in /_/src/Microsoft.DotNet.XUnitAssert/src/FailAsserts.cs:line 34
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor251|25_252(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass25_0&)

Apr 03 '24 01:04 v-wenyuxu

Failed in: runtime-coreclr r2r-extra 20240407.1

Failed tests:

R2R-CG2 linux arm Checked jitstressregs0x80 @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked jitstressregs8 @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked jitstressregs0x2000 @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked jitstress2 @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked forcerelocs @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked jitstressregs3 @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked jitminopts @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked jitstress2_tiered @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked gcstress0xc @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd

Error message:

 Test Infrastructure Failure: System.AggregateException: Not all processes in process tree could be terminated. (Operation not permitted)
 ---> System.ComponentModel.Win32Exception (1): Operation not permitted
   --- End of inner exception stack trace ---
   at System.Diagnostics.Process.Kill(Boolean entireProcessTree) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.NonUap.cs:line 31
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDumpWithCreateDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.RunTest(String executable, String outputFile, String errorFile, String category, String testBinaryBase, String outputDir)
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)

Stack trace:

   at Xunit.Assert.Fail(String message) in /_/src/Microsoft.DotNet.XUnitAssert/src/FailAsserts.cs:line 34
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor253|25_254(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass25_0&)

Queued	Pipeline	Pipeline_Configuration	OS	Arch	Test	Outcome	Properties	BuildReason
2024-03-31T07:33:02.486Z	coreclr-release-outerloop-nightly	Release-no_tiered_compilation	ubuntu.2004.armarch.open	arm	Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd	Failed	{"CollectionUri":"https://dev.azure.com/dnceng-public/","System.StageAttempt":"1","AzurePipelinesTestRunId":"15321360","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","System.StageName":"Build","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","operatingSystem":"(Alpine.316.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.17-helix-arm32v7","architecture":"arm","BuildNumber":"20240330.1","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.JobName":"__default","Reason":"Schedule","System.JobAttempt":"1","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionId":"135","BuildId":"625141"}	Schedule
2024-03-29T07:32:29.466Z	coreclr-release-outerloop-nightly	Release-no_tiered_compilation	ubuntu.2004.armarch.open	arm	Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd	Failed	{"Project":"public","Reason":"Schedule","BuildId":"623537","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","architecture":"arm","AzurePipelinesTestRunId":"15273954","System.JobName":"__default","DefinitionId":"135","System.PhaseAttempt":"1","System.StageAttempt":"1","BuildNumber":"20240328.1","System.JobAttempt":"1","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","operatingSystem":"(Alpine.316.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.17-helix-arm32v7","configuration":"Release-no_tiered_compilation","System.StageName":"Build","DefinitionName":"coreclr-release-outerloop-nightly"}	Schedule

Apr 09 '24 01:04 v-wenyuxu

Failed in: runtime-coreclr r2r-extra 20240413.1

Failed tests:

R2R-CG2 linux arm Checked jitstressregs0x1000 @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked jitstressregs3 @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked jitstress1_tiered @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked jitstressregs0x10 @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked jitstressregs8 @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked gcstress0xc @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd
R2R-CG2 linux arm Checked jitstress2 @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd

Error message:

 Test Infrastructure Failure: System.AggregateException: Not all processes in process tree could be terminated. (Operation not permitted)
 ---> System.ComponentModel.Win32Exception (1): Operation not permitted
   --- End of inner exception stack trace ---
   at System.Diagnostics.Process.Kill(Boolean entireProcessTree) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.NonUap.cs:line 31
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDumpWithCreateDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.RunTest(String executable, String outputFile, String errorFile, String category, String testBinaryBase, String outputDir)
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)

Stack trace:

   at Xunit.Assert.Fail(String message) in /_/src/Microsoft.DotNet.XUnitAssert/src/FailAsserts.cs:line 34
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor253|25_254(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass25_0&)

Queued	Pipeline	Pipeline_Configuration	OS	Arch	Test	Outcome	Properties	BuildReason
2024-03-31T07:33:02.486Z	coreclr-release-outerloop-nightly	Release-no_tiered_compilation	ubuntu.2004.armarch.open	arm	Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd	Failed	{"CollectionUri":"https://dev.azure.com/dnceng-public/","System.StageAttempt":"1","AzurePipelinesTestRunId":"15321360","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","System.StageName":"Build","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","operatingSystem":"(Alpine.316.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.17-helix-arm32v7","architecture":"arm","BuildNumber":"20240330.1","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.JobName":"__default","Reason":"Schedule","System.JobAttempt":"1","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionId":"135","BuildId":"625141"}	Schedule
2024-03-29T07:32:29.466Z	coreclr-release-outerloop-nightly	Release-no_tiered_compilation	ubuntu.2004.armarch.open	arm	Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd	Failed	{"Project":"public","Reason":"Schedule","BuildId":"623537","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","architecture":"arm","AzurePipelinesTestRunId":"15273954","System.JobName":"__default","DefinitionId":"135","System.PhaseAttempt":"1","System.StageAttempt":"1","BuildNumber":"20240328.1","System.JobAttempt":"1","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","operatingSystem":"(Alpine.316.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.17-helix-arm32v7","configuration":"Release-no_tiered_compilation","System.StageName":"Build","DefinitionName":"coreclr-release-outerloop-nightly"}	Schedule

Apr 15 '24 02:04 v-wenyuxu

cc @dotnet/area-system-diagnostics-process

Apr 15 '24 16:04 mikelle-rogers

Failed in: runtime-coreclr r2r 20240415.2

Failed tests:

R2R-CG2 linux arm Checked no_tiered_compilation @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd

Error message:

 Test Infrastructure Failure: System.AggregateException: Not all processes in process tree could be terminated. (Operation not permitted)
 ---> System.ComponentModel.Win32Exception (1): Operation not permitted
   --- End of inner exception stack trace ---
   at System.Diagnostics.Process.Kill(Boolean entireProcessTree) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.NonUap.cs:line 31
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDumpWithCreateDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.RunTest(String executable, String outputFile, String errorFile, String category, String testBinaryBase, String outputDir)
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)

Stack trace:

   at Xunit.Assert.Fail(String message) in /_/src/Microsoft.DotNet.XUnitAssert/src/FailAsserts.cs:line 34
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor253|25_254(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass25_0&)

Queued	Pipeline	Pipeline_Configuration	OS	Arch	Test	Outcome	Properties	BuildReason
2024-03-31T07:33:02.486Z	coreclr-release-outerloop-nightly	Release-no_tiered_compilation	ubuntu.2004.armarch.open	arm	Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd	Failed	{"CollectionUri":"https://dev.azure.com/dnceng-public/","System.StageAttempt":"1","AzurePipelinesTestRunId":"15321360","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","System.StageName":"Build","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","operatingSystem":"(Alpine.316.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.17-helix-arm32v7","architecture":"arm","BuildNumber":"20240330.1","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.JobName":"__default","Reason":"Schedule","System.JobAttempt":"1","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionId":"135","BuildId":"625141"}	Schedule
2024-03-29T07:32:29.466Z	coreclr-release-outerloop-nightly	Release-no_tiered_compilation	ubuntu.2004.armarch.open	arm	Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd	Failed	{"Project":"public","Reason":"Schedule","BuildId":"623537","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","architecture":"arm","AzurePipelinesTestRunId":"15273954","System.JobName":"__default","DefinitionId":"135","System.PhaseAttempt":"1","System.StageAttempt":"1","BuildNumber":"20240328.1","System.JobAttempt":"1","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","operatingSystem":"(Alpine.316.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.17-helix-arm32v7","configuration":"Release-no_tiered_compilation","System.StageName":"Build","DefinitionName":"coreclr-release-outerloop-nightly"}	Schedule

Apr 17 '24 01:04 v-wenyuxu

Failed in: runtime-coreclr r2r 20240416.1

Failed tests:

R2R-CG2 linux arm Checked no_tiered_compilation @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd

Error message:

 Test Infrastructure Failure: System.AggregateException: Not all processes in process tree could be terminated. (Operation not permitted)
 ---> System.ComponentModel.Win32Exception (1): Operation not permitted
   --- End of inner exception stack trace ---
   at System.Diagnostics.Process.Kill(Boolean entireProcessTree) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.NonUap.cs:line 31
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDumpWithCreateDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.RunTest(String executable, String outputFile, String errorFile, String category, String testBinaryBase, String outputDir)
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)

Stack trace:

   at Xunit.Assert.Fail(String message) in /_/src/Microsoft.DotNet.XUnitAssert/src/FailAsserts.cs:line 34
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor253|25_254(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass25_0&)

Queued	Pipeline	Pipeline_Configuration	OS	Arch	Test	Outcome	Properties	BuildReason
2024-03-31T07:33:02.486Z	coreclr-release-outerloop-nightly	Release-no_tiered_compilation	ubuntu.2004.armarch.open	arm	Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd	Failed	{"CollectionUri":"https://dev.azure.com/dnceng-public/","System.StageAttempt":"1","AzurePipelinesTestRunId":"15321360","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","System.StageName":"Build","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","operatingSystem":"(Alpine.316.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.17-helix-arm32v7","architecture":"arm","BuildNumber":"20240330.1","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.JobName":"__default","Reason":"Schedule","System.JobAttempt":"1","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionId":"135","BuildId":"625141"}	Schedule
2024-03-29T07:32:29.466Z	coreclr-release-outerloop-nightly	Release-no_tiered_compilation	ubuntu.2004.armarch.open	arm	Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd	Failed	{"Project":"public","Reason":"Schedule","BuildId":"623537","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","architecture":"arm","AzurePipelinesTestRunId":"15273954","System.JobName":"__default","DefinitionId":"135","System.PhaseAttempt":"1","System.StageAttempt":"1","BuildNumber":"20240328.1","System.JobAttempt":"1","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","operatingSystem":"(Alpine.316.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.17-helix-arm32v7","configuration":"Release-no_tiered_compilation","System.StageName":"Build","DefinitionName":"coreclr-release-outerloop-nightly"}	Schedule

Apr 18 '24 01:04 v-wenyuxu

Failed in: runtime-coreclr r2r 20240417.1

Failed tests:

R2R-CG2 linux arm Checked no_tiered_compilation @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd

Error message:

 Test Infrastructure Failure: System.AggregateException: Not all processes in process tree could be terminated. (Operation not permitted)
 ---> System.ComponentModel.Win32Exception (1): Operation not permitted
   --- End of inner exception stack trace ---
   at System.Diagnostics.Process.Kill(Boolean entireProcessTree) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.NonUap.cs:line 31
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDumpWithCreateDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.RunTest(String executable, String outputFile, String errorFile, String category, String testBinaryBase, String outputDir)
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)

Stack trace:

   at Xunit.Assert.Fail(String message) in /_/src/Microsoft.DotNet.XUnitAssert/src/FailAsserts.cs:line 34
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor253|25_254(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass25_0&)

Queued	Pipeline	Pipeline_Configuration	OS	Arch	Test	Outcome	Properties	BuildReason
2024-03-31T07:33:02.486Z	coreclr-release-outerloop-nightly	Release-no_tiered_compilation	ubuntu.2004.armarch.open	arm	Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd	Failed	{"CollectionUri":"https://dev.azure.com/dnceng-public/","System.StageAttempt":"1","AzurePipelinesTestRunId":"15321360","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","System.StageName":"Build","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","operatingSystem":"(Alpine.316.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.17-helix-arm32v7","architecture":"arm","BuildNumber":"20240330.1","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.JobName":"__default","Reason":"Schedule","System.JobAttempt":"1","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionId":"135","BuildId":"625141"}	Schedule
2024-03-29T07:32:29.466Z	coreclr-release-outerloop-nightly	Release-no_tiered_compilation	ubuntu.2004.armarch.open	arm	Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd	Failed	{"Project":"public","Reason":"Schedule","BuildId":"623537","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","architecture":"arm","AzurePipelinesTestRunId":"15273954","System.JobName":"__default","DefinitionId":"135","System.PhaseAttempt":"1","System.StageAttempt":"1","BuildNumber":"20240328.1","System.JobAttempt":"1","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","operatingSystem":"(Alpine.316.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.17-helix-arm32v7","configuration":"Release-no_tiered_compilation","System.StageName":"Build","DefinitionName":"coreclr-release-outerloop-nightly"}	Schedule

Apr 19 '24 01:04 v-wenyuxu

Failed in: runtime-coreclr r2r 20240419.1

Failed tests:

R2R-CG2 linux arm Checked @ (Ubuntu.1804.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd

Error message:

 Test Infrastructure Failure: System.AggregateException: Not all processes in process tree could be terminated. (Operation not permitted)
 ---> System.ComponentModel.Win32Exception (1): Operation not permitted
   --- End of inner exception stack trace ---
   at System.Diagnostics.Process.Kill(Boolean entireProcessTree) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.NonUap.cs:line 31
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDumpWithCreateDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.CollectCrashDump(Process process, String crashDumpPath, StreamWriter outputWriter)
   at CoreclrTestLib.CoreclrTestWrapperLib.RunTest(String executable, String outputFile, String errorFile, String category, String testBinaryBase, String outputDir)
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)

Stack trace:

   at Xunit.Assert.Fail(String message) in /_/src/Microsoft.DotNet.XUnitAssert/src/FailAsserts.cs:line 34
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor253|25_254(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass25_0&)

Queued	Pipeline	Pipeline_Configuration	OS	Arch	Test	Outcome	Properties	BuildReason
2024-03-31T07:33:02.486Z	coreclr-release-outerloop-nightly	Release-no_tiered_compilation	ubuntu.2004.armarch.open	arm	Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd	Failed	{"CollectionUri":"https://dev.azure.com/dnceng-public/","System.StageAttempt":"1","AzurePipelinesTestRunId":"15321360","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","System.StageName":"Build","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","operatingSystem":"(Alpine.316.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.17-helix-arm32v7","architecture":"arm","BuildNumber":"20240330.1","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.JobName":"__default","Reason":"Schedule","System.JobAttempt":"1","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionId":"135","BuildId":"625141"}	Schedule
2024-03-29T07:32:29.466Z	coreclr-release-outerloop-nightly	Release-no_tiered_compilation	ubuntu.2004.armarch.open	arm	Interop/NativeLibrary/Callback/CallbackStressTest_TargetUnix/CallbackStressTest_TargetUnix.cmd	Failed	{"Project":"public","Reason":"Schedule","BuildId":"623537","CollectionUri":"https://dev.azure.com/dnceng-public/","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","architecture":"arm","AzurePipelinesTestRunId":"15273954","System.JobName":"__default","DefinitionId":"135","System.PhaseAttempt":"1","System.StageAttempt":"1","BuildNumber":"20240328.1","System.JobAttempt":"1","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","operatingSystem":"(Alpine.316.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.17-helix-arm32v7","configuration":"Release-no_tiered_compilation","System.StageName":"Build","DefinitionName":"coreclr-release-outerloop-nightly"}	Schedule

Apr 22 '24 02:04 v-wenyuxu

@jeffhandley, what is the status of this test failure?

Apr 22 '24 17:04 mikelle-rogers

@jkotas would it be OK to ignore "operation not permitted" errors on Kill(entireProcessTree: true)?

Apr 29 '24 16:04 jozkee

@mikelle-rogers we currently don't have a fix nor a consistent repro, we could spend more time looking into it but not sure if we could get a repro at all.

Apr 29 '24 16:04 jozkee

would it be OK to ignore "operation not permitted" errors on Kill(entireProcessTree: true)?

I do not know. We need to understand what leads to the "operation not permitted" errors.

May 06 '24 15:05 jkotas

Delete workaround added in https://github.com/dotnet/runtime/pull/102958 once this gets fixed.

Jun 02 '24 14:06 jkotas

runtime runtime copied to clipboard

Test failure: AggregateException: Not all processes in process tree could be terminated.

Known issue validation

Report

Summary

runtime
runtime copied to clipboard