BenchmarkDotNet Linux thread priority/nice set to lower-priority than normal starting from the second benchmark.

I use nice -n -1 when running the benchmark to minimize the effect of background processes on the system (on Linux). I am using NativeAOT 9.0.2 with the InProcessNoEmit toolchain. I observe that when running the first benchmark, the nice value is expectedly -1. However, starting from the second benchmark method, the nice value becomes 10, which is low priority. It is quite surprising to me, but I didn't find where the thread priority is changed.

Mar 10 '25 01:03 eliphatfs

The process and thread priority is set here. https://github.com/dotnet/BenchmarkDotNet/blob/6ce97955ca5c9f960babf75708fbcab4b71e6c3a/src/BenchmarkDotNet/Toolchains/InProcess/NoEmit/InProcessNoEmitExecutor.cs#L94-L95 They are set to highest (just under realtime). They are set for each benchmark, so I'm not sure why you are observing different behavior between benchmarks. Are you actually reading the value before the first benchmark?

Mar 10 '25 02:03 timcassell

It is halfway running the benchmark. I use htop (like windows task manager) to monitor the processes and threads. The first row is the process and the second row is the thread. The NI column is 0 by default for normal priority threads. -11 is high priority. 10 is low priority. Thus, the process has quite high priority, but the thread running actual job (71.1% out of 72.4% cpu) is in lower priority. Strangely, during the first benchmarked method is running, the thread has high priority. Then starting at the second benchmarked method, the priority becomes lower.

Mar 10 '25 03:03 eliphatfs

My guess is thread.TrySetPriority(ThreadPriority.Highest, parameters.Logger); fails as linux prevents a thread to be more prioritized than the process, and the enums of thread.Priority don't map exactly to the NICE levels of linux. Thus, when the finally block of the first benchmark executes, the thread may be set to a lower priority due to mapping mismatches. I am trying to trace down the issue.

Mar 10 '25 04:03 eliphatfs

https://github.com/dotnet/runtime/blob/v9.0.2/src/coreclr/nativeaot/System.Private.CoreLib/src/System/Threading/Thread.NativeAot.Unix.cs

Priority = ... seems to be a no-op on nativeaot on linux. I'm not sure how it happens now.

Mar 10 '25 04:03 eliphatfs

https://github.com/dotnet/runtime/issues/60114#issuecomment-940558809

Mar 10 '25 05:03 timcassell

I am aware of this behavior; but it is still weird that the thread's priority becomes lower than normal priority.

Mar 10 '25 05:03 eliphatfs

I agree that is strange. Unfortunately I don't know how to debug it.

Mar 10 '25 05:03 timcassell