runtime Set names to runtime internal threads

trafficstars

It gives names to internal threads so that they can be distinguished easier when debugging, triaging, and/or profiling.

(gdb) info threads
  Id   Target Id                                            Frame
* 1    Thread 0x7ffff7a4a3c0 (LWP 296950) "corerun"         __GI___libc_read (nbytes=1024, buf=0x7fffffffc810, fd=0) at ../sysdeps/unix/sysv/linux/read.c:26
  2    Thread 0x7ffff6afd640 (LWP 296953) "corerun-ust"     syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
  3    Thread 0x7ffff62fc640 (LWP 296954) "corerun-ust"     syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
  4    Thread 0x7ffff5afb640 (LWP 296955) ".NET SynchManag" 0x00007ffff7b63d7f in __GI___poll (fds=0x7ffff5afa9b8, nfds=1, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
  5    Thread 0x7ffff52fa640 (LWP 296956) ".NET Eventpipe " 0x00007ffff7b63d7f in __GI___poll (fds=0x7fff70008f20, nfds=1, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
  6    Thread 0x7ffff4ad4640 (LWP 296957) ".NET Debug Tran" 0x00007ffff7b5f764 in __libc_open64 (file=0x55555559fa44 "/tmp/clr-debug-pipe-296950-42007406-in", oflag=0) at ../sysdeps/unix/sysv/linux/open64.c:41
  7    Thread 0x7fffe7fff640 (LWP 296958) ".NET DebuggerRC" __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x555555644e10) at ./nptl/futex-internal.c:57
  8    Thread 0x7fff77cc5640 (LWP 296959) ".NET Finalizer"  __futex_abstimed_wait_common64 (private=1432319836, cancel=true, abstime=0x7fff77cc40b8, op=137, expected=0, futex_word=0x555555665950) at ./nptl/futex-internal.c:57
  9    Thread 0x7fffe6e02640 (LWP 296960) ".NET SigHandler" __GI___libc_read (nbytes=1, buf=0x7fffe6e01e47, fd=33) at ../sysdeps/unix/sysv/linux/read.c:26

Sep 06 '22 06:09 HJLeee

Tagging subscribers to this area: @tommcdon See info in area-owners.md if you want to be subscribed.

Issue Details

It gives names to internal thread so that they can be distinguished easier when debugging, triaging, and/or profiling.

(gdb) info threads
  Id   Target Id                                            Frame
* 1    Thread 0x7ffff7a4a3c0 (LWP 296950) "corerun"         __GI___libc_read (nbytes=1024, buf=0x7fffffffc810, fd=0) at ../sysdeps/unix/sysv/linux/read.c:26
  2    Thread 0x7ffff6afd640 (LWP 296953) "corerun-ust"     syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
  3    Thread 0x7ffff62fc640 (LWP 296954) "corerun-ust"     syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
  4    Thread 0x7ffff5afb640 (LWP 296955) ".NET SynchManag" 0x00007ffff7b63d7f in __GI___poll (fds=0x7ffff5afa9b8, nfds=1, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
  5    Thread 0x7ffff52fa640 (LWP 296956) ".NET Eventpipe " 0x00007ffff7b63d7f in __GI___poll (fds=0x7fff70008f20, nfds=1, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
  6    Thread 0x7ffff4ad4640 (LWP 296957) ".NET Debug Tran" 0x00007ffff7b5f764 in __libc_open64 (file=0x55555559fa44 "/tmp/clr-debug-pipe-296950-42007406-in", oflag=0) at ../sysdeps/unix/sysv/linux/open64.c:41
  7    Thread 0x7fffe7fff640 (LWP 296958) ".NET DebuggerRC" __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x555555644e10) at ./nptl/futex-internal.c:57
  8    Thread 0x7fff77cc5640 (LWP 296959) ".NET Finalizer"  __futex_abstimed_wait_common64 (private=1432319836, cancel=true, abstime=0x7fff77cc40b8, op=137, expected=0, futex_word=0x555555665950) at ./nptl/futex-internal.c:57
  9    Thread 0x7fffe6e02640 (LWP 296960) ".NET SigHandler" __GI___libc_read (nbytes=1, buf=0x7fffe6e01e47, fd=33) at ../sysdeps/unix/sysv/linux/read.c:26

Author:	HJLeee
Assignees:	-
Labels:	`area-Diagnostics-coreclr`
Milestone:	-

Sep 06 '22 06:09 ghost

Note that thread names are limited to 15 chars on Linux and perhaps macOS and FreeBSD, so all names must be distinguishable in the first 15 chars. With ".NET " that leaves 10. The ones in this PR seem fine but we have not done this in some places eg:

https://github.com/dotnet/runtime/blob/bcd44bfc3d435ffe5af372491818a9cfcd0a959b/src/libraries/System.Private.CoreLib/src/System/Threading/ThreadPoolWorkQueue.cs#L1358

https://github.com/dotnet/runtime/blob/bcd44bfc3d435ffe5af372491818a9cfcd0a959b/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.GateThread.cs#L244

https://github.com/dotnet/runtime/blob/6251d496e307ed39f1e600cd036106cc1bd7e340/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.WaitThread.cs#L190

Are not distinguishable. Windows from memory has a higher limit, so places in Windows specific code should be fine.

Might be worth follow up cleanup. IMO I'd keep the prefix but use some contracted form like ".NET TPool Wait"

We maybe should Assert in the PAL for > 15

Sep 06 '22 07:09 danmoseley

Mac builds broke


Users/runner/work/1/s/src/native/libs/System.Native/pal_signal.c:532:47: error: too many arguments to function call, expected 1, have 2

Sep 06 '22 07:09 danmoseley

Thank you for the note. According to the below MAC allows longer thread names. Also, it doesn't allow to change other threads name.

https://github.com/dotnet/runtime/blob/2dcd5bf697538d6b3a1cf5f00888114fe3d97e9a/src/coreclr/pal/src/thread/thread.cpp#L94-L100

https://github.com/dotnet/runtime/blob/2dcd5bf697538d6b3a1cf5f00888114fe3d97e9a/src/coreclr/pal/src/thread/thread.cpp#L1662-L1668

Sep 06 '22 08:09 HJLeee

Test failures seem infrastructural... failed to load CLR, entry point not found, etc

Sep 07 '22 04:09 danmoseley

The failures looks like a real problem with the change to me.

Sep 07 '22 05:09 jkotas

Suspicious mono test is passed with https://github.com/dotnet/runtime/pull/75113/commits/ebca9d434ba20ea0f05f1f3dbef1290dadb7f37d. https://github.com/dotnet/runtime/pull/75185 is submitted for double checking and it seems there is no problem with mono tests.

Sep 07 '22 11:09 HJLeee

@danmoseley @jkotas Would you merge this please?

Sep 18 '22 21:09 HJLeee

The tests are failing due to changes in this PR

Sep 19 '22 06:09 jkotas

Build Browser wasm windows Release LibraryTests_AOT [01:12:42] info: Waiting to flush log messages with a timeout of 120 secs .. fail: Microsoft.AspNetCore.Server.Kestrel[13] Connection id "0HMKPRFK1AHDC", Request id "0HMKPRFK1AHDC:00000001": An unhandled exception was thrown by the application. System.InvalidOperationException: Message processor is not running. Make sure to call RunAsync first at Microsoft.DotNet.XHarness.CLI.Commands.Wasm.WasmTestMessagesProcessor.InvokeAsync(String message, Boolean isError) in //src/Microsoft.DotNet.XHarness.CLI/Commands/WASM/WasmTestMessagesProcessor.cs:line 94 at Microsoft.DotNet.XHarness.CLI.Commands.Wasm.WasmBrowserTestRunner.RunConsoleMessagesPump(WebSocket socket, CancellationToken token) in //src/Microsoft.DotNet.XHarness.CLI/Commands/WASM/Browser/WasmBrowserTestRunner.cs:line 196 at Microsoft.DotNet.XHarness.CLI.Commands.WebServer.TestWebServerStartup.<>c__DisplayClass3_0.<<Configure>b__2>d.MoveNext() in /_/src/Microsoft.DotNet.XHarness.CLI/Commands/WebServer.cs:line 163 --- End of stack trace from previous location --- at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application) [01:12:42] fail: [out of order message from the browser]: http://127.0.0.1:49193/dotnet.js 4:1872 "[main] websocket closed: [object CloseEvent]" CloseEvent [01:12:42] fail: Application has finished with exit code TESTS_FAILED but 0 was expected XHarness exit code: 71 (GENERAL_FAILURE)
Build iOS arm64 Release AllSubsets_Mono [16:35:12] fail: Application test run crashed No test log file was produced Check logs for more information
Libraries Test Run release coreclr windows x86 Release 257 Error(s) C:\h\w\B6BF09BC\w\A61D091B\e>"C:\h\w\B6BF09BC\p\dotnet.exe" exec --runtimeconfig System.Xml.XmlSerializer.ReflectionOnly.Tests.runtimeconfig.json --depsfile System.Xml.XmlSerializer.ReflectionOnly.Tests.deps.json xunit.console.dll System.Xml.XmlSerializer.ReflectionOnly.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
Failed to load the dll from [C:\h\w\B6BF09BC\p\shared\Microsoft.NETCore.App\8.0.0\coreclr.dll], HRESULT: 0x8007007F
Libraries Test Run release coreclr windows x64 Debug 257 Error(s) C:\h\w\A86F0922\w\C3030A5C\e>"C:\h\w\A86F0922\p\dotnet.exe" exec --runtimeconfig Common.Tests.runtimeconfig.json --depsfile Common.Tests.deps.json xunit.console.dll Common.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
Failed to load the dll from [C:\h\w\A86F0922\p\shared\Microsoft.NETCore.App\8.0.0\coreclr.dll], HRESULT: 0x8007007F
Libraries Test Run checked coreclr windows x86 Debug 257 Error(s) C:\h\w\9FE50944\w\976A0843\e>"C:\h\w\9FE50944\p\dotnet.exe" exec --runtimeconfig System.Xml.XmlSerializer.ReflectionOnly.Tests.runtimeconfig.json --depsfile System.Xml.XmlSerializer.ReflectionOnly.Tests.deps.json xunit.console.dll System.Xml.XmlSerializer.ReflectionOnly.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
Failed to load the dll from [C:\h\w\9FE50944\p\shared\Microsoft.NETCore.App\8.0.0\coreclr.dll], HRESULT: 0x8007007F
Libraries Test Run checked coreclr windows x64 Release 257 Error(s) C:\h\w\9CD30919\w\A683088E\e>"C:\h\w\9CD30919\p\dotnet.exe" exec --runtimeconfig System.Diagnostics.EventLog.Tests.runtimeconfig.json --depsfile System.Diagnostics.EventLog.Tests.deps.json xunit.console.dll System.Diagnostics.EventLog.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
Failed to load the dll from [C:\h\w\9CD30919\p\shared\Microsoft.NETCore.App\8.0.0\coreclr.dll], HRESULT: 0x8007007F
coreclr Pri0 Runtime Tests Run windows x86 checked JIT\SIMD\CtorFromArray_ro\CtorFromArray_ro.cmd [FAIL] Failed to load: 'C:\h\w\B1C6096C\p\coreclr.dll'. Error: 0x0000007f
mono llvmaot Pri0 Runtime Tests Run Linux x64 release Expected: 100 Actual: 0 END EXECUTION - FAILED Test Harness Exitcode is : 1 To run the test: > set CORE_ROOT=/datadisks/disk1/work/B79209F6/p > /datadisks/disk1/work/B79209F6/w/A5340917/e/JIT/HardwareIntrinsics/X86/Aes/Aes_ro/Aes_ro.sh Expected: True Actual: False Stack Trace: at JIT_HardwareIntrinsics.X86_Aes_Aes_ro_Aes_ro._X86_Aes_Aes_ro_Aes_ro_sh() at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor) at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

Sep 19 '22 06:09 HJLeee

@jkotas I think it's ready. .NET CI is nice. :)

Sep 20 '22 02:09 HJLeee

Now it looks like this on Linux.

(gdb) info threads
  Id   Target Id                                            Frame 
* 1    Thread 0x7ffff7a4c3c0 (LWP 619311) "corerun"         __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737348158400) at ./nptl/pthread_kill.c:44
  2    Thread 0x7ffff6afc640 (LWP 619314) "corerun-ust"     syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
  3    Thread 0x7ffff62fb640 (LWP 619315) "corerun-ust"     syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
  4    Thread 0x7ffff5afa640 (LWP 619316) ".NET SynchManag" 0x00007ffff7b65d7f in __GI___poll (fds=0x0, nfds=0, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
  5    Thread 0x7ffff52f9640 (LWP 619317) ".NET EventPipe"  0x00007ffff7b65d7f in __GI___poll (fds=0x7fff70008f40, nfds=1, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
  6    Thread 0x7ffff4ad3640 (LWP 619318) ".NET DebugPipe"  0x00007ffff7b61764 in __libc_open64 (file=0x55555559fa44 "/tmp/clr-debug-pipe-619311-18359507-in", oflag=0) at ../sysdeps/unix/sysv/linux/open64.c:41
  7    Thread 0x7fffe7fff640 (LWP 619319) ".NET Debugger"   __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x555555645670) at ./nptl/futex-internal.c:57
  8    Thread 0x7fff77cc4640 (LWP 619320) ".NET Finalizer"  __futex_abstimed_wait_common64 (private=1432321820, cancel=true, abstime=0x7fff77cc30b8, op=137, expected=0, futex_word=0x555555666360) at ./nptl/futex-internal.c:57
  9    Thread 0x7fffe6e02640 (LWP 619321) ".NET SigHandler" __GI___libc_read (nbytes=1, buf=0x7fffe6e01e47, fd=33) at ../sysdeps/unix/sysv/linux/read.c:26

Sep 21 '22 00:09 HJLeee

Thank you!

Sep 22 '22 03:09 jkotas

Thank you for your kind reviews and advice.

Sep 22 '22 05:09 HJLeee

runtime runtime copied to clipboard

Set names to runtime internal threads

runtime
runtime copied to clipboard