dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard
verified publisher icon DataDog

[Profiler] New stackwalker for Linux

Open gleocadie opened this issue 2 years ago • 4 comments

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

gleocadie avatar Oct 30 '23 13:10 gleocadie

Datadog Report

Branch report: gleocadie/unwind-using-clr-internal-structure Commit report: 9b98633

:x: dd-trace-dotnet: 25 Failed (0 Known Flaky), 0 New Flaky, 292042 Passed, 1106 Skipped, 37m 36.6s Wall Time

:x: Failed Tests (25)

This report shows up to 5 failed tests.

  • CheckCpuLimit - Datadog.Profiler.IntegrationTests.SmokeTests.CpuLimitTest - Details

    Expand for error 
    xit code of "catchsegv" should be 0 instead of 139 (= 0x8B)
xpected: True
ctual:   False

  • CheckCpuLimit - Datadog.Profiler.IntegrationTests.SmokeTests.CpuLimitTest - Details

    Expand for error 
    xit code of "catchsegv" should be 0 instead of 139 (= 0x8B)
xpected: True
ctual:   False

  • CheckCpuLimit - Datadog.Profiler.IntegrationTests.SmokeTests.CpuLimitTest - Details

    Expand for error 
    xit code of "catchsegv" should be 0 instead of 139 (= 0x8B)
xpected: True
ctual:   False

  • CheckCpuLimit - Datadog.Profiler.IntegrationTests.SmokeTests.CpuLimitTest - Details

    Expand for error 
    xit code of "catchsegv" should be 0 instead of 139 (= 0x8B)
xpected: True
ctual:   False

  • CheckCpuLimit - Datadog.Profiler.IntegrationTests.SmokeTests.CpuLimitTest - Details

    Expand for error 
    xit code of "dotnet" should be 0 instead of 139 (= 0x8B)
xpected: True
ctual:   False

datadog-ddstaging[bot] avatar Oct 30 '23 14:10 datadog-ddstaging[bot]

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4781) - mean (71ms)  : 62, 80
     .   : milestone, 71,
    master - mean (70ms)  : 63, 78
     .   : milestone, 70,

    section CallTarget+Inlining+NGEN
    This PR (4781) - mean (993ms)  : 969, 1018
     .   : milestone, 993,
    master - mean (996ms)  : 979, 1014
     .   : milestone, 996,


gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4781) - mean (106ms)  : 103, 109
     .   : milestone, 106,
    master - mean (106ms)  : 102, 110
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (4781) - mean (687ms)  : 660, 713
     .   : milestone, 687,
    master - mean (681ms)  : 666, 696
     .   : milestone, 681,


gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4781) - mean (90ms)  : 88, 93
     .   : milestone, 90,
    master - mean (90ms)  : 88, 92
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (4781) - mean (655ms)  : 633, 678
     .   : milestone, 655,
    master - mean (654ms)  : 634, 673
     .   : milestone, 654,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4781) - mean (188ms)  : 185, 191
     .   : milestone, 188,
    master - mean (189ms)  : 185, 192
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (4781) - mean (1,141ms)  : 1118, 1165
     .   : milestone, 1141,
    master - mean (1,144ms)  : 1128, 1161
     .   : milestone, 1144,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4781) - mean (271ms)  : 267, 276
     .   : milestone, 271,
    master - mean (272ms)  : 268, 277
     .   : milestone, 272,

    section CallTarget+Inlining+NGEN
    This PR (4781) - mean (1,096ms)  : 1072, 1120
     .   : milestone, 1096,
    master - mean (1,100ms)  : 1082, 1118
     .   : milestone, 1100,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4781) - mean (262ms)  : 258, 266
     .   : milestone, 262,
    master - mean (263ms)  : 259, 267
     .   : milestone, 263,

    section CallTarget+Inlining+NGEN
    This PR (4781) - mean (1,066ms)  : 1041, 1090
     .   : milestone, 1066,
    master - mean (1,069ms)  : 1042, 1095
     .   : milestone, 1069,

andrewlock avatar Oct 30 '23 14:10 andrewlock

Benchmarks Report :snail:

Benchmarks for #4781 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.180
  • 4 benchmarks are slower, with geometric mean 1.181
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.29μs 45ns 263ns 0.0256 0.0128 0 7.29 KB
master StartStopWithChild netcoreapp3.1 10μs 48.8ns 207ns 0.0243 0.00972 0 7.38 KB
master StartStopWithChild net472 15.7μs 46.4ns 180ns 1.29 0.335 0.114 7.66 KB
#4781 StartStopWithChild net6.0 8.23μs 41ns 188ns 0.0253 0.0127 0 7.29 KB
#4781 StartStopWithChild netcoreapp3.1 10.1μs 44.4ns 166ns 0.0398 0.0199 0 7.39 KB
#4781 StartStopWithChild net472 15.7μs 37.6ns 146ns 1.31 0.355 0.116 7.66 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 483μs 218ns 844ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 638μs 381ns 1.47μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 793μs 224ns 837ns 0.396 0 0 3.3 KB
#4781 WriteAndFlushEnrichedTraces net6.0 482μs 321ns 1.24μs 0 0 0 2.7 KB
#4781 WriteAndFlushEnrichedTraces netcoreapp3.1 631μs 96.9ns 363ns 0 0 0 2.7 KB
#4781 WriteAndFlushEnrichedTraces net472 795μs 342ns 1.32μs 0.396 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #4781

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1 1.180 201.30 170.60

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 43.7μs 96.6ns 374ns 0.0216 0 0 2.03 KB
master AllCycleSimpleBody netcoreapp3.1 46μs 164ns 636ns 0.0225 0 0 2.01 KB
master AllCycleSimpleBody net472 47.5μs 29.3ns 109ns 0.327 0 0 2.08 KB
master AllCycleMoreComplexBody net6.0 232μs 86.6ns 336ns 0.116 0 0 8.63 KB
master AllCycleMoreComplexBody netcoreapp3.1 233μs 122ns 472ns 0 0 0 8.52 KB
master AllCycleMoreComplexBody net472 239μs 94.2ns 365ns 1.31 0 0 8.7 KB
master ObjectExtractorSimpleBody net6.0 122ns 0.0847ns 0.317ns 0.00396 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 202ns 0.118ns 0.44ns 0.00373 0 0 272 B
master ObjectExtractorSimpleBody net472 150ns 0.0931ns 0.361ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.04μs 1.81ns 7ns 0.0547 0 0 3.88 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.12μs 1.65ns 6.17ns 0.0515 0 0 3.78 KB
master ObjectExtractorMoreComplexBody net472 4.09μs 2.35ns 9.12ns 0.618 0.00614 0 3.89 KB
#4781 AllCycleSimpleBody net6.0 43.9μs 32.8ns 127ns 0.022 0 0 2.03 KB
#4781 AllCycleSimpleBody netcoreapp3.1 45.1μs 59.5ns 231ns 0.0224 0 0 2.01 KB
#4781 AllCycleSimpleBody net472 46.6μs 14.4ns 53.9ns 0.323 0 0 2.08 KB
#4781 AllCycleMoreComplexBody net6.0 229μs 62.8ns 235ns 0.114 0 0 8.63 KB
#4781 AllCycleMoreComplexBody netcoreapp3.1 234μs 105ns 391ns 0.117 0 0 8.52 KB
#4781 AllCycleMoreComplexBody net472 239μs 92.1ns 357ns 1.31 0 0 8.7 KB
#4781 ObjectExtractorSimpleBody net6.0 120ns 0.105ns 0.394ns 0.00395 0 0 280 B
#4781 ObjectExtractorSimpleBody netcoreapp3.1 171ns 0.083ns 0.311ns 0.0038 0 0 272 B
#4781 ObjectExtractorSimpleBody net472 146ns 0.137ns 0.511ns 0.0446 0 0 281 B
#4781 ObjectExtractorMoreComplexBody net6.0 2.99μs 1.32ns 4.94ns 0.0549 0 0 3.88 KB
#4781 ObjectExtractorMoreComplexBody netcoreapp3.1 3.98μs 1.04ns 4.04ns 0.0518 0 0 3.78 KB
#4781 ObjectExtractorMoreComplexBody net472 4.12μs 4.7ns 18.2ns 0.618 0.00614 0 3.89 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 12.9μs 5.2ns 19.4ns 0.128 0 0 9.42 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 19.4μs 7.94ns 30.8ns 0.126 0 0 9.42 KB
master RunWaf(args=NestedMap (10)) net472 28.6μs 30.6ns 115ns 1.5 0 0 9.48 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 77.2μs 70ns 262ns 0.232 0 0 15.77 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 87.3μs 87.1ns 337ns 0.214 0 0 15.72 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 100μs 97.1ns 363ns 2.54 0 0 16.04 KB
master RunWaf(args=NestedMap (100)) net6.0 24μs 13.1ns 49.1ns 0.275 0 0 19.66 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 37.9μs 22.1ns 85.6ns 0.266 0 0 20.42 KB
master RunWaf(args=NestedMap (100)) net472 55.1μs 42.5ns 165ns 3.26 0.0274 0 20.63 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 92.3μs 90.8ns 340ns 0.368 0 0 26.01 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 110μs 176ns 682ns 0.328 0 0 26.72 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 134μs 263ns 1.02μs 4.27 0.0657 0 27.19 KB
master RunWaf(args=NestedMap (20)) net6.0 23.9μs 12.6ns 47ns 0.276 0 0 19.4 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 37.6μs 17.2ns 66.8ns 0.264 0 0 19.84 KB
master RunWaf(args=NestedMap (20)) net472 54.8μs 60.2ns 233ns 3.17 0.0551 0 20.04 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 94.2μs 66.7ns 250ns 0.328 0 0 25.74 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 108μs 104ns 402ns 0.327 0 0 26.14 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 131μs 149ns 517ns 4.18 0.0654 0 26.6 KB
#4781 RunWaf(args=NestedMap (10)) net6.0 12.6μs 3.43ns 12.8ns 0.133 0 0 9.42 KB
#4781 RunWaf(args=NestedMap (10)) netcoreapp3.1 19.6μs 11.3ns 42.1ns 0.128 0 0 9.42 KB
#4781 RunWaf(args=NestedMap (10)) net472 28.6μs 26.6ns 103ns 1.49 0 0 9.48 KB
#4781 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 77μs 41ns 154ns 0.193 0 0 15.77 KB
#4781 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 86.9μs 228ns 882ns 0.217 0 0 15.72 KB
#4781 RunWafWithAttack(args=Neste(...)tack) [22]) net472 101μs 199ns 769ns 2.53 0 0 16.04 KB
#4781 RunWaf(args=NestedMap (100)) net6.0 24.1μs 7.62ns 29.5ns 0.277 0 0 19.66 KB
#4781 RunWaf(args=NestedMap (100)) netcoreapp3.1 38.4μs 12.8ns 49.7ns 0.269 0 0 20.42 KB
#4781 RunWaf(args=NestedMap (100)) net472 55.5μs 69.1ns 268ns 3.26 0.0276 0 20.63 KB
#4781 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 94μs 82.9ns 321ns 0.375 0 0 26.01 KB
#4781 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 110μs 185ns 694ns 0.327 0 0 26.72 KB
#4781 RunWafWithAttack(args=Neste(...)tack) [23]) net472 133μs 121ns 453ns 4.26 0.0666 0 27.19 KB
#4781 RunWaf(args=NestedMap (20)) net6.0 23.7μs 9.71ns 37.6ns 0.276 0 0 19.4 KB
#4781 RunWaf(args=NestedMap (20)) netcoreapp3.1 38μs 15.5ns 60ns 0.265 0 0 19.84 KB
#4781 RunWaf(args=NestedMap (20)) net472 54.9μs 50ns 194ns 3.18 0.0553 0 20.04 KB
#4781 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 93.5μs 163ns 631ns 0.328 0 0 25.74 KB
#4781 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 112μs 96.7ns 349ns 0.332 0 0 26.14 KB
#4781 RunWafWithAttack(args=Neste(...)tack) [22]) net472 134μs 171ns 663ns 4.18 0.0664 0 26.6 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 169μs 168ns 630ns 0.169 0 0 18.04 KB
master SendRequest netcoreapp3.1 190μs 255ns 986ns 0.143 0 0 20.2 KB
master SendRequest net472 0.000814ns 0.000247ns 0.000956ns 0 0 0 0 b
#4781 SendRequest net6.0 173μs 193ns 747ns 0.173 0 0 18.04 KB
#4781 SendRequest netcoreapp3.1 190μs 234ns 874ns 0.189 0 0 20.2 KB
#4781 SendRequest net472 0.000125ns 6.91E‑05ns 0.000268ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: More allocations :warning:

More allocations :warning: in #4781

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.56 KB 41.79 KB 229 B 0.55%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 531μs 504ns 1.89μs 0.548 0 0 41.56 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 666μs 1.58μs 6.13μs 0.329 0 0 41.81 KB
master WriteAndFlushEnrichedTraces net472 845μs 3.02μs 11.7μs 8.17 2.45 0.408 53.25 KB
#4781 WriteAndFlushEnrichedTraces net6.0 570μs 423ns 1.64μs 0.551 0 0 41.79 KB
#4781 WriteAndFlushEnrichedTraces netcoreapp3.1 643μs 805ns 3.12μs 0.326 0 0 41.69 KB
#4781 WriteAndFlushEnrichedTraces net472 832μs 3.92μs 15.2μs 8.33 2.5 0.417 53.27 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.04μs 0.723ns 2.8ns 0.0107 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.41μs 1.69ns 6.55ns 0.0106 0 0 768 B
master ExecuteNonQuery net472 1.66μs 0.667ns 2.31ns 0.116 0 0 730 B
#4781 ExecuteNonQuery net6.0 1.07μs 0.75ns 2.9ns 0.0107 0 0 768 B
#4781 ExecuteNonQuery netcoreapp3.1 1.36μs 0.655ns 2.54ns 0.0103 0 0 768 B
#4781 ExecuteNonQuery net472 1.62μs 1.35ns 5.06ns 0.116 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.13μs 0.37ns 1.38ns 0.013 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.43μs 0.554ns 2.15ns 0.0125 0 0 936 B
master CallElasticsearch net472 2.51μs 0.505ns 1.96ns 0.151 0 0 955 B
master CallElasticsearchAsync net6.0 1.28μs 1.08ns 4.04ns 0.0129 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.57μs 0.692ns 2.68ns 0.0133 0 0 984 B
master CallElasticsearchAsync net472 2.4μs 0.696ns 2.6ns 0.16 0 0 1.01 KB
#4781 CallElasticsearch net6.0 1.11μs 0.392ns 1.47ns 0.013 0 0 936 B
#4781 CallElasticsearch netcoreapp3.1 1.34μs 1.27ns 4.76ns 0.0127 0 0 936 B
#4781 CallElasticsearch net472 2.51μs 0.599ns 2.32ns 0.151 0 0 955 B
#4781 CallElasticsearchAsync net6.0 1.32μs 0.872ns 3.38ns 0.0124 0 0 912 B
#4781 CallElasticsearchAsync netcoreapp3.1 1.51μs 1.21ns 4.54ns 0.013 0 0 984 B
#4781 CallElasticsearchAsync net472 2.52μs 1.56ns 6.06ns 0.161 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.21μs 0.865ns 3.12ns 0.0127 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.5μs 1.12ns 4.2ns 0.0127 0 0 912 B
master ExecuteAsync net472 1.6μs 0.532ns 1.99ns 0.138 0 0 875 B
#4781 ExecuteAsync net6.0 1.25μs 0.747ns 2.89ns 0.0125 0 0 912 B
#4781 ExecuteAsync netcoreapp3.1 1.44μs 0.747ns 2.79ns 0.0122 0 0 912 B
#4781 ExecuteAsync net472 1.7μs 1.12ns 4.33ns 0.139 0 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 3.89μs 2.69ns 10.1ns 0.0273 0 0 1.9 KB
master SendAsync netcoreapp3.1 4.46μs 1.13ns 4.21ns 0.0312 0 0 2.43 KB
master SendAsync net472 7.05μs 1.87ns 6.98ns 0.474 0 0 2.99 KB
#4781 SendAsync net6.0 3.83μs 1.93ns 7.21ns 0.0268 0 0 1.9 KB
#4781 SendAsync netcoreapp3.1 4.4μs 1.95ns 7.28ns 0.033 0 0 2.43 KB
#4781 SendAsync net472 7.14μs 3.76ns 14.6ns 0.472 0 0 2.99 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.3μs 1.03ns 3.85ns 0.0219 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 1.97μs 0.663ns 2.39ns 0.0214 0 0 1.57 KB
master EnrichedLog net472 2.29μs 2.05ns 7.92ns 0.237 0 0 1.49 KB
#4781 EnrichedLog net6.0 1.4μs 0.824ns 3.08ns 0.0224 0 0 1.57 KB
#4781 EnrichedLog netcoreapp3.1 1.88μs 8.1ns 30.3ns 0.0211 0 0 1.57 KB
#4781 EnrichedLog net472 2.2μs 2.05ns 7.68ns 0.237 0 0 1.49 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 112μs 109ns 422ns 0.0557 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 116μs 116ns 433ns 0.0578 0 0 4.21 KB
master EnrichedLog net472 146μs 100ns 389ns 0.659 0.22 0 4.38 KB
#4781 EnrichedLog net6.0 111μs 152ns 590ns 0.0555 0 0 4.21 KB
#4781 EnrichedLog netcoreapp3.1 117μs 175ns 655ns 0 0 0 4.21 KB
#4781 EnrichedLog net472 150μs 88.5ns 343ns 0.678 0.226 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.91μs 0.669ns 2.59ns 0.0291 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 3.92μs 3.94ns 15.3ns 0.0295 0 0 2.13 KB
master EnrichedLog net472 4.54μs 1.88ns 7.02ns 0.306 0 0 1.93 KB
#4781 EnrichedLog net6.0 2.92μs 0.901ns 3.49ns 0.0293 0 0 2.13 KB
#4781 EnrichedLog netcoreapp3.1 3.76μs 1.58ns 5.9ns 0.0282 0 0 2.13 KB
#4781 EnrichedLog net472 4.53μs 1.37ns 5.3ns 0.306 0 0 1.93 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.29μs 0.506ns 1.82ns 0.0154 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.57μs 3.21ns 12.4ns 0.0148 0 0 1.1 KB
master SendReceive net472 1.95μs 0.89ns 3.33ns 0.177 0 0 1.12 KB
#4781 SendReceive net6.0 1.31μs 0.857ns 3.32ns 0.0155 0 0 1.1 KB
#4781 SendReceive netcoreapp3.1 1.59μs 1.67ns 6.47ns 0.015 0 0 1.1 KB
#4781 SendReceive net472 1.87μs 2.44ns 9.44ns 0.177 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.5μs 1.15ns 4.45ns 0.0215 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.62μs 1.39ns 5.38ns 0.0217 0 0 1.58 KB
master EnrichedLog net472 3.96μs 1.97ns 7.63ns 0.31 0 0 1.96 KB
#4781 EnrichedLog net6.0 2.5μs 1.01ns 3.65ns 0.0212 0 0 1.53 KB
#4781 EnrichedLog netcoreapp3.1 3.54μs 0.897ns 3.36ns 0.0213 0 0 1.58 KB
#4781 EnrichedLog net472 4.05μs 1.54ns 5.78ns 0.31 0 0 1.96 KB
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #4781

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 1.239 544.33 674.39
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 1.185 399.34 473.18
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472 1.160 602.00 698.25
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 1.144 823.34 942.08

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 399ns 0.122ns 0.474ns 0.00755 0 0 536 B
master StartFinishSpan netcoreapp3.1 544ns 0.148ns 0.533ns 0.00714 0 0 536 B
master StartFinishSpan net472 602ns 0.154ns 0.595ns 0.0853 0 0 538 B
master StartFinishScope net6.0 500ns 0.134ns 0.501ns 0.00921 0 0 656 B
master StartFinishScope netcoreapp3.1 703ns 0.211ns 0.762ns 0.00908 0 0 656 B
master StartFinishScope net472 823ns 0.467ns 1.81ns 0.098 0 0 618 B
#4781 StartFinishSpan net6.0 473ns 0.152ns 0.588ns 0.00747 0 0 536 B
#4781 StartFinishSpan netcoreapp3.1 674ns 0.545ns 2.11ns 0.00738 0 0 536 B
#4781 StartFinishSpan net472 698ns 0.184ns 0.689ns 0.0854 0 0 538 B
#4781 StartFinishScope net6.0 546ns 0.189ns 0.73ns 0.00904 0 0 656 B
#4781 StartFinishScope netcoreapp3.1 769ns 0.187ns 0.725ns 0.00878 0 0 656 B
#4781 StartFinishScope net472 942ns 0.24ns 0.929ns 0.0977 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 594ns 0.465ns 1.8ns 0.00914 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 790ns 0.277ns 1.07ns 0.00868 0 0 656 B
master RunOnMethodBegin net472 1.03μs 0.274ns 1.02ns 0.0982 0 0 618 B
#4781 RunOnMethodBegin net6.0 636ns 0.517ns 2ns 0.00898 0 0 656 B
#4781 RunOnMethodBegin netcoreapp3.1 830ns 0.195ns 0.754ns 0.00905 0 0 656 B
#4781 RunOnMethodBegin net472 1.05μs 0.73ns 2.83ns 0.0981 0 0 618 B

andrewlock avatar Oct 30 '23 15:10 andrewlock

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4781) (11.167M)   : 0, 11166976
    master (11.242M)   : 0, 11242466
    benchmarks/2.38.0 (11.867M)   : 0, 11866956
    benchmarks/2.9.0 (11.148M)   : 0, 11148256

    section Automatic
    This PR (4781) (7.725M)   : 0, 7724821
    master (7.717M)   : 0, 7716578
    benchmarks/2.38.0 (8.176M)   : 0, 8175636
    benchmarks/2.9.0 (8.025M)   : 0, 8025233

    section Trace stats
    This PR (4781) (8.056M)   : 0, 8056198
    master (8.010M)   : 0, 8009578
    benchmarks/2.38.0 (8.450M)   : 0, 8450370

    section Manual
    This PR (4781) (9.802M)   : 0, 9802132
    master (9.947M)   : 0, 9947441
    benchmarks/2.38.0 (10.334M)   : 0, 10334368

    section Manual + Automatic
    This PR (4781) (7.293M)   : 0, 7293420
    master (7.326M)   : 0, 7326031
    benchmarks/2.38.0 (7.750M)   : 0, 7750484

    section Version Conflict
    This PR (4781) (6.488M)   : 0, 6488352
    master (6.719M)   : 0, 6718535
    benchmarks/2.38.0 (7.137M)   : 0, 7136691


gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4781) (9.798M)   : 0, 9797936
    master (9.290M)   : 0, 9289639
    benchmarks/2.38.0 (9.548M)   : 0, 9548121
    benchmarks/2.9.0 (9.594M)   : 0, 9594335

    section Automatic
    This PR (4781) (6.553M)   : 0, 6552774
    master (6.690M)   : 0, 6689611
    benchmarks/2.38.0 (6.747M)   : 0, 6747227

    section Trace stats
    This PR (4781) (6.860M)   : 0, 6860002
    master (6.889M)   : 0, 6888843
    benchmarks/2.38.0 (6.815M)   : 0, 6814846

    section Manual
    This PR (4781) (8.510M)   : 0, 8510342
    master (8.434M)   : 0, 8434167
    benchmarks/2.38.0 (8.263M)   : 0, 8263131

    section Manual + Automatic
    This PR (4781) (6.184M)   : 0, 6184070
    master (6.416M)   : 0, 6416152
    benchmarks/2.38.0 (6.275M)   : 0, 6275411

    section Version Conflict
    This PR (4781) (5.714M)   : 0, 5713906
    master (5.822M)   : 0, 5821752
    benchmarks/2.38.0 (5.670M)   : 0, 5669744


gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4781) (9.710M)   : 0, 9709753
    master (9.485M)   : 0, 9484728
    benchmarks/2.38.0 (9.007M)   : 0, 9007066
    benchmarks/2.9.0 (9.326M)   : 0, 9326015

    section Automatic
    This PR (4781) (6.779M)   : 0, 6778623
    master (6.767M)   : 0, 6767292
    benchmarks/2.38.0 (6.423M)   : 0, 6422756
    benchmarks/2.9.0 (6.908M)   : 0, 6907621

    section Trace stats
    This PR (4781) (6.989M)   : 0, 6988661
    master (6.890M)   : 0, 6890346
    benchmarks/2.38.0 (6.721M)   : 0, 6720838

    section Manual
    This PR (4781) (8.435M)   : 0, 8435128
    master (8.255M)   : 0, 8254600
    benchmarks/2.38.0 (7.977M)   : 0, 7977494

    section Manual + Automatic
    This PR (4781) (6.434M)   : 0, 6434296
    master (6.420M)   : 0, 6420444
    benchmarks/2.38.0 (6.349M)   : 0, 6349066

    section Version Conflict
    This PR (4781) (5.961M)   : 0, 5960998
    master (5.847M)   : 0, 5846561
    benchmarks/2.38.0 (5.682M)   : 0, 5681654


gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    master (7.709M)   : 0, 7709011
    benchmarks/2.38.0 (7.578M)   : 0, 7577544
    benchmarks/2.9.0 (8.068M)   : 0, 8067939

    section No attack
    master (2.230M)   : 0, 2229524
    benchmarks/2.38.0 (2.184M)   : 0, 2184279
    benchmarks/2.9.0 (3.350M)   : 0, 3349512

    section Attack
    master (1.776M)   : 0, 1775817
    benchmarks/2.38.0 (1.711M)   : 0, 1710542
    benchmarks/2.9.0 (2.643M)   : 0, 2643387

    section Blocking
    master (3.554M)   : 0, 3554060
    benchmarks/2.38.0 (3.486M)   : 0, 3486124

    section IAST default
    master (7.089M)   : 0, 7089314

    section IAST full
    master (6.330M)   : 0, 6329800

    section Base vuln
    master (0.994M)   : 0, 993532

    section IAST vuln
    master (0.937M)   : 0, 937108

andrewlock avatar Oct 30 '23 15:10 andrewlock