akka.net
akka.net copied to clipboard
akkadotnet
Updated `RemotePingPing` to also support .NET 9
Won't run on the build system without some updates, but figured we should give it a try
.NET 8
First run
Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority.
OSVersion: Unix 6.9.3.76060903
ProcessorCount: 16
ClockSpeed: 0 MHZ
Actor Count: 32
Messages sent/received per client: 200000 (2e5)
Is Server GC: True
Thread count: 42
Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
1, 200000, 118977, 1681.70, 42, 68
5, 1000000, 372163, 2687.16, 76, 80
10, 2000000, 737464, 2712.57, 88, 90
15, 3000000, 766088, 3916.48, 98, 98
20, 4000000, 765258, 5227.18, 106, 107
25, 5000000, 777726, 6429.57, 115, 93
30, 6000000, 801176, 7489.40, 101, 93
Second run:
Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority.
OSVersion: Unix 6.9.3.76060903
ProcessorCount: 16
ClockSpeed: 0 MHZ
Actor Count: 32
Messages sent/received per client: 200000 (2e5)
Is Server GC: True
Thread count: 43
Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
1, 200000, 108933, 1836.81, 43, 70
5, 1000000, 322894, 3097.82, 78, 80
10, 2000000, 718391, 2784.02, 88, 93
15, 3000000, 754717, 3975.14, 101, 101
20, 4000000, 749626, 5336.03, 109, 101
25, 5000000, 769586, 6497.42, 109, 95
30, 6000000, 758342, 7912.86, 103, 95
Third run:
OSVersion: Unix 6.9.3.76060903
ProcessorCount: 16
ClockSpeed: 0 MHZ
Actor Count: 32
Messages sent/received per client: 200000 (2e5)
Is Server GC: True
Thread count: 43
Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
1, 200000, 108578, 1842.73, 43, 68
5, 1000000, 353858, 2826.18, 76, 80
10, 2000000, 726745, 2752.39, 88, 94
15, 3000000, 712082, 4213.18, 102, 102
20, 4000000, 740056, 5405.47, 110, 102
25, 5000000, 776037, 6443.32, 110, 94
30, 6000000, 751127, 7988.91, 102, 94
.NET 9
First run
OSVersion: Unix 6.9.3.76060903
ProcessorCount: 16
ClockSpeed: 0 MHZ
Actor Count: 32
Messages sent/received per client: 200000 (2e5)
Is Server GC: True
Thread count: 42
Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
1, 200000, 117165, 1707.04, 42, 56
5, 1000000, 398884, 2507.26, 64, 66
10, 2000000, 691563, 2892.56, 74, 87
15, 3000000, 729750, 4111.34, 95, 95
20, 4000000, 737736, 5422.05, 103, 97
25, 5000000, 735836, 6795.78, 105, 89
30, 6000000, 736830, 8143.54, 97, 87
Done..
Second run:
Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority.
OSVersion: Unix 6.9.3.76060903
ProcessorCount: 16
ClockSpeed: 0 MHZ
Actor Count: 32
Messages sent/received per client: 200000 (2e5)
Is Server GC: True
Thread count: 42
Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
1, 200000, 132539, 1509.65, 42, 55
5, 1000000, 331017, 3021.78, 63, 68
10, 2000000, 696865, 2870.39, 76, 84
15, 3000000, 710901, 4220.58, 92, 92
20, 4000000, 730461, 5476.37, 100, 93
25, 5000000, 717670, 6967.97, 101, 86
30, 6000000, 710985, 8439.67, 94, 78
Third run:
Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority.
OSVersion: Unix 6.9.3.76060903
ProcessorCount: 16
ClockSpeed: 0 MHZ
Actor Count: 32
Messages sent/received per client: 200000 (2e5)
Is Server GC: True
Thread count: 42
Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
1, 200000, 117717, 1699.07, 42, 57
5, 1000000, 448230, 2231.97, 65, 68
10, 2000000, 727538, 2749.12, 76, 76
15, 3000000, 706548, 4246.51, 84, 92
20, 4000000, 710985, 5626.97, 100, 100
25, 5000000, 720254, 6942.15, 108, 85
30, 6000000, 723241, 8296.13, 93, 77
Takeaways
There are some data points that make it look like .NET 9 can really outperform .NET 8 (i.e. ~450k on the step 2 of the third iteration) but overall the results are quite a bit lower on .NET 9 than they are on .NET 8, by about 8-10%.
I'll retry this later from a cold start because I wonder if CPU heating is a factor - and my grafana dashboard for monitoring that machine doesn't give me that info right now - that would also explain why all of the .NET 8 numbers are lower than they were on the initial run too.
I'll re-run the numbers again from a cold start but I'm getting https://github.com/akkadotnet/akka.net/issues/5385 vibes from this so far
Was given a hint on Twitter that the culprit with the performance numbers might be the new GC mode DATAS introduced in .NET 9, so I've decided to re-run the numbers with that setting disabled via MSBuild properties.
.NET 9 w/ DATAS Disabled
Run 1
Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority.
OSVersion: Unix 6.9.3.76060903
ProcessorCount: 16
ClockSpeed: 0 MHZ
Actor Count: 32
Messages sent/received per client: 200000 (2e5)
Is Server GC: True
Thread count: 41
Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
1, 200000, 113379, 1764.75, 41, 66
5, 1000000, 379795, 2633.39, 74, 76
10, 2000000, 706714, 2830.85, 84, 84
15, 3000000, 766088, 3916.06, 92, 107
20, 4000000, 762486, 5246.25, 115, 115
25, 5000000, 788147, 6344.19, 123, 99
30, 6000000, 791975, 7576.56, 107, 88
Run 2
Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority.
OSVersion: Unix 6.9.3.76060903
ProcessorCount: 16
ClockSpeed: 0 MHZ
Actor Count: 32
Messages sent/received per client: 200000 (2e5)
Is Server GC: True
Thread count: 41
Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
1, 200000, 125550, 1593.19, 41, 67
5, 1000000, 323311, 3093.21, 75, 78
10, 2000000, 776398, 2576.24, 86, 86
15, 3000000, 757768, 3959.69, 94, 99
20, 4000000, 772052, 5181.50, 107, 109
25, 5000000, 775555, 6447.73, 117, 93
30, 6000000, 781556, 7677.74, 101, 93
Run 3
Attempted to elevate process priority, but failed due to Permission denied - carrying on at normal process priority.
OSVersion: Unix 6.9.3.76060903
ProcessorCount: 16
ClockSpeed: 0 MHZ
Actor Count: 32
Messages sent/received per client: 200000 (2e5)
Is Server GC: True
Thread count: 42
Num clients, Total [msg], Msgs/sec, Total [ms], Start Threads, End Threads
1, 200000, 119332, 1676.91, 42, 66
5, 1000000, 330034, 3030.71, 74, 75
10, 2000000, 732601, 2730.50, 83, 85
15, 3000000, 770417, 3894.92, 93, 95
20, 4000000, 793022, 5044.85, 103, 103
25, 5000000, 774474, 6456.14, 111, 87
30, 6000000, 810045, 7407.47, 95, 87
Takeaways
It definitely looks like DATAS, which is enabled by default in .NET 9, was responsible for the throughput degradation we observed. With it disabled it looks like .NET 9 performs slightly faster than .NET 8 and with a peak observed performance that is higher in absolute terms. This is only looking at throughput data - these benchmarks don't consider GC.
So if it's just a GC setting (and therefore, not a bug in the runtime), no risk of https://github.com/akkadotnet/akka.net/issues/5385 - we'll just need to see how things going with real-world applications that upgrade to .NET 9.