Benchmarks icon indicating copy to clipboard operation
Benchmarks copied to clipboard

Published 20 hours ago • verified publisher icon aspnet

HttpClient Benchmarks: server build crashes on 9.0

Open CarnaViire opened this issue 6 months ago • 1 comments

Follow-up from https://github.com/aspnet/Benchmarks/issues/2006

Server app crashes the build job if 9.0 TFM is requested, but 8.0 works. Client app builds and works properly on 9.0 though.

--profile intel-lin-app --profile amd-lin2-load, --server.framework net9.0

Logs 
➜ ~  crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=1.1 --variable useHttps=true --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=10 --variable responseSize=256 --client.framework net9.0 --server.framework net9.0 --client.options.collectCounters true
[08:45:29.594] Running session 'aba271939cbb463b991183f5a0ffcd3b' with description ''
[08:45:29.990] Starting job 'server' ...
[08:45:30.171] Submitted job: http://asp-citrine-amd2:5001/jobs/151
[08:45:30.531] 'server' has been selected by the server ...
[08:45:30.704] 'server' is now building ... http://asp-citrine-amd2:5001/jobs/151/buildlog
[08:45:33.591] 'server' failed on agent, stopping...
An unexpected error occurred while building the job: One or more errors occurred. (Object reference not set to an instance of an object.)
[08:45:33.946] Job has failed, interrupting benchmarks ...
[08:45:33.949] Stopping job 'server' ...
[08:45:35.483] Deleting job 'server' ...

Not able to repro on a local agent (on Windows, Crank Agent version 0.2.0-alpha.24379.1+a59cbdbe3f8fc7aef08c17652cc2cef6136464ca):

--profile local, --server.framework net9.0

Logs 
➜ ~  ✗ crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --scenario httpclient-kestrel-get --profile local --variable useHttpMessageInvoker=true --variable httpVersion=1.1 --variable useHttps=true --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=10 --variable responseSize=256 --client.framework net9.0 --server.framework net9.0 --client.options.collectCounters true
[08:57:03.867] Running session 'fe2a0f689db84a6cb11ba8ae7061a6b4' with description ''
[08:57:03.907] Starting job 'server' ...
[08:57:04.021] Submitted job: http://localhost:5010/jobs/1
[08:57:08.098] 'server' has been selected by the server ...
[08:57:08.102] 'server' is now building ... http://localhost:5010/jobs/1/buildlog
[08:59:14.528] 'server' is running ... http://localhost:5010/jobs/1/output
[08:59:14.537] Starting job 'client' ...
[08:59:14.540] Submitted job: http://localhost:5010/jobs/2
[08:59:16.566] 'client' has been selected by the server ...
[08:59:16.567] 'client' is now building ... http://localhost:5010/jobs/2/buildlog
[08:59:26.686] 'client' is running ... http://localhost:5010/jobs/2/output
[09:00:00.997] Stopping job 'client' ...
[09:00:02.051] Deleting job 'client' ...
[09:00:02.054] Stopping job 'server' ...
[09:00:04.086] Deleting job 'server' ...

| server                  |                                  |
| ----------------------- | -------------------------------- |
| Max CPU Usage (%)       | 47                               |
| Max Cores usage (%)     | 560                              |
| Max Working Set (MB)    | 75                               |
| Max Private Memory (MB) | 32                               |
| Build Time (ms)         | 45,501                           |
| Start Time (ms)         | 5,682                            |
| Published Size (KB)     | 101,208                          |
| Symbols Size (KB)       | 23                               |
| .NET Core SDK Version   | 9.0.100-rc.1.24415.1             |
| ASP.NET Core Version    | 9.0.0-rc.1.24412.15+abbd67b97144 |
| .NET Runtime Version    | 9.0.0-rc.1.24413.14+605ff6f64851 |


| client                                  |                                  |
| --------------------------------------- | -------------------------------- |
| Max CPU Usage (%)                       | 33                               |
| Max Cores usage (%)                     | 391                              |
| Max Working Set (MB)                    | 61                               |
| Max Private Memory (MB)                 | 30                               |
| Build Time (ms)                         | 3,688                            |
| Start Time (ms)                         | 1,162                            |
| Published Size (KB)                     | 74,917                           |
| Symbols Size (KB)                       | 19                               |
| .NET Core SDK Version                   | 9.0.100-rc.1.24415.1             |
| ASP.NET Core Version                    | 9.0.0-rc.1.24412.15+abbd67b97144 |
| .NET Runtime Version                    | 9.0.0-rc.1.24413.14+605ff6f64851 |
| Processor Count                         | 12                               |
| First request duration (ms)             | 794                              |
| Requests                                | 657,722                          |
| Bad Status Code Requests                | 0                                |
| Exceptions                              | 0                                |
| Mean RPS                                | 43,805                           |
| Max CPU Usage (%)                       | 31                               |
| Max Working Set (MB)                    | 63                               |
| Max GC Heap Size (MB)                   | 5                                |
| Size of committed memory by the GC (MB) | 6                                |
| Max Number of Gen 0 GCs / sec           | 26.00                            |
| Max Number of Gen 1 GCs / sec           | 1.00                             |
| Max Number of Gen 2 GCs / sec           | 0.00                             |
| Max Gen 0 GC Budget (MB)                | 4                                |
| Max Time in GC (%)                      | 5.00                             |
| Max Gen 0 Size (B)                      | 1,702,744                        |
| Max Gen 1 Size (B)                      | 819,664                          |
| Max Gen 2 Size (B)                      | 1,574,656                        |
| Max LOH Size (B)                        | 0                                |
| Max POH Size (B)                        | 24,528                           |
| Max Allocation Rate (B/sec)             | 65,893,200                       |
| Max GC Heap Fragmentation (%)           | 4,135%                           |
| # of Assemblies Loaded                  | 42                               |
| Max Exceptions (#/s)                    | 2                                |
| Max Lock Contention (#/s)               | 5                                |
| Max ThreadPool Threads Count            | 20                               |
| Max ThreadPool Queue Length             | 1                                |
| Max ThreadPool Items (#/s)              | 88,857                           |
| Max Active Timers                       | 2                                |
| IL Jitted (B)                           | 282,332                          |
| Methods Jitted                          | 3,147                            |
Crank Agent logs 
➜ crank git:(main) crank-agent
[20:56:53 INF] Crank Agent version 0.2.0-alpha.24379.1+a59cbdbe3f8fc7aef08c17652cc2cef6136464ca
[20:56:53 INF] Starting agent on http://*:5010...

....

[20:57:08 INF] Starting job 'server' (1)

....

[20:57:10 INF] Installing dotnet runtimes and sdk
[20:57:10 INF] Specific target framework: 'net9.0'
[20:57:10 INF] Patching project file 'C:\Users\knatalia\AppData\Local\Temp\benchmarks-agent\benchmarks-server-30908\5x21ruwe.2vt\src\BenchmarksApps\HttpClientBenchmarks\Servers\Kestrel\Kestrel.csproj'
[20:57:12 INF] Runtime: 9.0.0-rc.1.24413.14 (Latest - From 9.0 SDK)
[20:57:12 INF] SDK: 9.0.100-rc.1.24415.1 (Latest - From Product Commit)
[20:57:13 INF] ASP.NET: 9.0.0-rc.1.24412.15 (Latest - From 9.0 SDK)
[20:57:13 INF] Creating custom global.json
[20:57:13 INF] Desktop: 9.0.0-preview.7.24405.2 (Latest)
[20:57:13 INF] Installing SDK '9.0.100-rc.1.24415.1' ...
[20:57:13 INF] Checking package: https://dotnetbuilds.azureedge.net/public/Sdk/9.0.100-rc.1.24415.1/dotnet-sdk-9.0.100-rc.1.24415.1-win-x64.zip
[20:58:16 INF] Installing Runtime '9.0.0-rc.1.24413.14' ...
[20:58:16 INF] Checking package: https://dotnetbuilds.azureedge.net/public/Runtime/9.0.0-rc.1.24413.14/dotnet-runtime-9.0.0-rc.1.24413.14-win-x64.zip
[20:58:18 INF] Installing Desktop runtime '9.0.0-preview.7.24405.2' ...
[20:58:18 INF] Checking package: https://dotnetbuilds.azureedge.net/public/Runtime/9.0.0-preview.7.24405.2/windowsdesktop-runtime-9.0.0-preview.7.24405.2-win-x64.zip
[20:58:19 INF] Checking package: https://dotnetcli.azureedge.net/dotnet/Runtime/9.0.0-preview.7.24405.2/windowsdesktop-runtime-9.0.0-preview.7.24405.2-win-x64.zip
[20:58:19 INF] Installing ASP.NET runtime '9.0.0-rc.1.24412.15' ...
[20:58:19 INF] Checking package: https://dotnetbuilds.azureedge.net/public/aspnetcore/Runtime/9.0.0-rc.1.24412.15/aspnetcore-runtime-9.0.0-rc.1.24412.15-win-x64.zip
[20:58:22 INF] Working directory: C:\Users\knatalia\AppData\Local\Temp\benchmarks-agent\benchmarks-server-30908\5x21ruwe.2vt\src\BenchmarksApps\HttpClientBenchmarks\Servers\Kestrel
[20:58:22 INF] Command line: C:\Users\knatalia\AppData\Local\Temp\benchmarks-agent\benchmarks-server-30908\bqtroxbz.0w0\dotnet.exe publish Kestrel.csproj -c Release -o C:\Users\knatalia\AppData\Local\Temp\benchmarks-agent\benchmarks-server-30908\5x21ruwe.2vt\src\BenchmarksApps\HttpClientBenchmarks\Servers\Kestrel\published /p:MicrosoftNETCoreAppPackageVersion=9.0.0-rc.1.24413.14 /p:MicrosoftAspNetCoreAppPackageVersion=9.0.0-rc.1.24412.15 /p:GenerateErrorForMissingTargetingPacks=false /p:RestoreNoCache=true /p:MicrosoftWindowsDesktopAppPackageVersion=9.0.0-rc.1.24414.1 /p:MicrosoftNETPlatformLibrary=Microsoft.NETCore.App --framework net9.0 --self-contained -r win-x64
[20:59:07 INF] Application published successfully in 45501.9321 ms

....

[20:59:18 INF] Starting job 'client' (2)

....

[20:59:19 INF] Installing dotnet runtimes and sdk
[20:59:19 INF] Specific target framework: 'net9.0'
[20:59:19 INF] Patching project file 'C:\Users\knatalia\AppData\Local\Temp\benchmarks-agent\benchmarks-server-30908\5vkazsrp.a35\src\BenchmarksApps\HttpClientBenchmarks\Clients\HttpClient\HttpClient.csproj'
[20:59:21 INF] Runtime: 9.0.0-rc.1.24413.14 (Latest - From 9.0 SDK)
[20:59:21 INF] SDK: 9.0.100-rc.1.24415.1 (Latest - From Product Commit)
[20:59:21 INF] ASP.NET: 9.0.0-rc.1.24412.15 (Latest - From 9.0 SDK)
[20:59:21 INF] Creating custom global.json
[20:59:21 INF] Desktop: 9.0.0-preview.7.24405.2 (Latest)
[20:59:21 INF] Working directory: C:\Users\knatalia\AppData\Local\Temp\benchmarks-agent\benchmarks-server-30908\5vkazsrp.a35\src\BenchmarksApps\HttpClientBenchmarks\Clients\HttpClient
[20:59:21 INF] Command line: C:\Users\knatalia\AppData\Local\Temp\benchmarks-agent\benchmarks-server-30908\bqtroxbz.0w0\dotnet.exe publish HttpClient.csproj -c Release -o C:\Users\knatalia\AppData\Local\Temp\benchmarks-agent\benchmarks-server-30908\5vkazsrp.a35\src\BenchmarksApps\HttpClientBenchmarks\Clients\HttpClient\published /p:MicrosoftNETCoreAppPackageVersion=9.0.0-rc.1.24413.14 /p:MicrosoftAspNetCoreAppPackageVersion=9.0.0-rc.1.24412.15 /p:GenerateErrorForMissingTargetingPacks=false /p:RestoreNoCache=true /p:MicrosoftWindowsDesktopAppPackageVersion=9.0.0-rc.1.24414.1 /p:MicrosoftNETPlatformLibrary=Microsoft.NETCore.App --framework net9.0 --self-contained -r win-x64
[20:59:25 INF] Application published successfully in 3688.6041 ms

....

CarnaViire avatar Aug 21 '24 12:08 CarnaViire