dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard
verified publisher icon DataDog

[WIP] Update CI support for release branches

Open andrewlock opened this issue 2 years ago • 5 comments

Summary of changes

Update CI to support release/2.x and 3.x on master

Reason for change

We're cutting a release/2.x branch, and want to make sure our automations work on those branches

Implementation details

  • Update a bunch of test stages to run on merges to master or release/**
  • Update a github action to run on all release branches
  • Update various aspects to run on v2 release branches as well as v1
  • Only publish :latest docker tags on master (to be discussed)
  • Only update benchmark branches

Test coverage

Yeah... we can't really test all this without doing a release 😬

Other details

To be decided

  • [] Do we want to permanently keep a "latest" 2.x benchmark branch for comparison?
  • [] How should we handle the "latest" lib-injection containers. I've updated them to only run for 3.x but we need to discuss how we're going to roll this out

andrewlock avatar Nov 03 '23 12:11 andrewlock

Datadog Report

Branch report: andrew/ci/release-branches Commit report: 8584c01

:white_check_mark: dd-trace-dotnet: 0 Failed, 0 New Flaky, 291899 Passed, 945 Skipped, 54m 15.54s Wall Time

datadog-ddstaging[bot] avatar Nov 03 '23 13:11 datadog-ddstaging[bot]

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4811) - mean (73ms)  : 65, 82
     .   : milestone, 73,
    master - mean (73ms)  : 65, 80
     .   : milestone, 73,

    section CallTarget+Inlining+NGEN
    This PR (4811) - mean (1,000ms)  : 973, 1026
     .   : milestone, 1000,
    master - mean (1,003ms)  : 983, 1022
     .   : milestone, 1003,


gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4811) - mean (111ms)  : 108, 114
     .   : milestone, 111,
    master - mean (111ms)  : 106, 115
     .   : milestone, 111,

    section CallTarget+Inlining+NGEN
    This PR (4811) - mean (716ms)  : 687, 745
     .   : milestone, 716,
    master - mean (717ms)  : 692, 742
     .   : milestone, 717,


gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4811) - mean (94ms)  : 91, 96
     .   : milestone, 94,
    master - mean (94ms)  : 90, 98
     .   : milestone, 94,

    section CallTarget+Inlining+NGEN
    This PR (4811) - mean (674ms)  : 645, 702
     .   : milestone, 674,
    master - mean (676ms)  : 651, 700
     .   : milestone, 676,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4811) - mean (188ms)  : 185, 191
     .   : milestone, 188,
    master - mean (187ms)  : 182, 193
     .   : milestone, 187,

    section CallTarget+Inlining+NGEN
    This PR (4811) - mean (1,081ms)  : 1058, 1104
     .   : milestone, 1081,
    master - mean (1,077ms)  : 1056, 1097
     .   : milestone, 1077,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4811) - mean (272ms)  : 267, 277
     .   : milestone, 272,
    master - mean (270ms)  : 264, 277
     .   : milestone, 270,

    section CallTarget+Inlining+NGEN
    This PR (4811) - mean (874ms)  : 846, 901
     .   : milestone, 874,
    master - mean (868ms)  : 847, 889
     .   : milestone, 868,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4811) - mean (260ms)  : 256, 264
     .   : milestone, 260,
    master - mean (260ms)  : 253, 266
     .   : milestone, 260,

    section CallTarget+Inlining+NGEN
    This PR (4811) - mean (857ms)  : 829, 885
     .   : milestone, 857,
    master - mean (855ms)  : 824, 887
     .   : milestone, 855,

andrewlock avatar Nov 03 '23 13:11 andrewlock

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4811) (11.416M)   : 0, 11416427
    master (11.488M)   : 0, 11487640
    benchmarks/2.9.0 (11.392M)   : 0, 11391935

    section Automatic
    This PR (4811) (7.882M)   : 0, 7881731
    master (7.846M)   : 0, 7845687
    benchmarks/2.9.0 (8.262M)   : 0, 8262391

    section Trace stats
    This PR (4811) (8.208M)   : 0, 8208131
    master (8.313M)   : 0, 8313072

    section Manual
    This PR (4811) (10.010M)   : 0, 10009622
    master (9.948M)   : 0, 9947556

    section Manual + Automatic
    This PR (4811) (7.544M)   : 0, 7543860
    master (7.411M)   : 0, 7411345

    section Version Conflict
    This PR (4811) (6.657M)   : 0, 6657310
    master (6.757M)   : 0, 6757124


gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4811) (9.422M)   : 0, 9421769
    master (9.676M)   : 0, 9675896
    benchmarks/2.9.0 (9.682M)   : 0, 9682370

    section Automatic
    This PR (4811) (6.577M)   : 0, 6576504
    master (6.648M)   : 0, 6648346

    section Trace stats
    This PR (4811) (6.867M)   : 0, 6867028
    master (6.856M)   : 0, 6855794

    section Manual
    This PR (4811) (8.293M)   : 0, 8292826
    master (8.399M)   : 0, 8399398

    section Manual + Automatic
    This PR (4811) (5.966M)   : 0, 5966199
    master (6.231M)   : 0, 6231131

    section Version Conflict
    This PR (4811) (5.752M)   : 0, 5751550
    master (5.698M)   : 0, 5698324


gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4811) (8.733M)   : 0, 8732891
    master (8.706M)   : 0, 8706036
    benchmarks/2.9.0 (8.781M)   : 0, 8781024

    section Automatic
    This PR (4811) (6.262M)   : 0, 6262112
    master (6.118M)   : 0, 6118101
    benchmarks/2.9.0 (6.470M)   : 0, 6470482

    section Trace stats
    This PR (4811) (6.492M)   : 0, 6492420
    master (6.327M)   : 0, 6327387

    section Manual
    This PR (4811) (7.915M)   : 0, 7914869
    master (7.824M)   : 0, 7823664

    section Manual + Automatic
    This PR (4811) (6.045M)   : 0, 6044505
    master (5.891M)   : 0, 5890564

    section Version Conflict
    This PR (4811) (5.299M)   : 0, 5298827
    master (5.248M)   : 0, 5248258

andrewlock avatar Nov 03 '23 19:11 andrewlock

Benchmarks Report :snail:

Benchmarks for #4811 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.254
  • 5 benchmarks are slower, with geometric mean 1.170
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.11μs 44.1ns 312ns 0.0271 0.0116 0 7.29 KB
master StartStopWithChild netcoreapp3.1 10.2μs 53.8ns 323ns 0.0246 0.00984 0 7.39 KB
master StartStopWithChild net472 15.7μs 57.2ns 222ns 1.3 0.332 0.111 7.66 KB
#4811 StartStopWithChild net6.0 8.05μs 44.5ns 281ns 0.0288 0.0123 0 7.29 KB
#4811 StartStopWithChild netcoreapp3.1 9.89μs 56.2ns 420ns 0.0288 0.0144 0 7.39 KB
#4811 StartStopWithChild net472 15.6μs 50.7ns 196ns 1.28 0.313 0.0939 7.65 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 492μs 140ns 524ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 660μs 232ns 900ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 806μs 536ns 2.08μs 0.398 0 0 3.3 KB
#4811 WriteAndFlushEnrichedTraces net6.0 494μs 399ns 1.54μs 0 0 0 2.7 KB
#4811 WriteAndFlushEnrichedTraces netcoreapp3.1 629μs 198ns 768ns 0 0 0 2.7 KB
#4811 WriteAndFlushEnrichedTraces net472 791μs 366ns 1.32μs 0.393 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #4811

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1 1.254 224.50 179.01

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 44.1μs 18.2ns 68ns 0.0222 0 0 2.03 KB
master AllCycleSimpleBody netcoreapp3.1 44.7μs 85.8ns 332ns 0.0226 0 0 2.01 KB
master AllCycleSimpleBody net472 47.5μs 110ns 424ns 0.32 0 0 2.08 KB
master AllCycleMoreComplexBody net6.0 230μs 82.9ns 321ns 0.115 0 0 8.63 KB
master AllCycleMoreComplexBody netcoreapp3.1 235μs 269ns 1.04μs 0 0 0 8.52 KB
master AllCycleMoreComplexBody net472 241μs 145ns 561ns 1.31 0 0 8.7 KB
master ObjectExtractorSimpleBody net6.0 121ns 0.103ns 0.4ns 0.00395 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 224ns 0.177ns 0.687ns 0.00363 0 0 272 B
master ObjectExtractorSimpleBody net472 146ns 0.08ns 0.31ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.01μs 2.04ns 7.64ns 0.0545 0 0 3.88 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.13μs 2.01ns 7.78ns 0.0516 0 0 3.78 KB
master ObjectExtractorMoreComplexBody net472 4.11μs 2.82ns 10.9ns 0.617 0.00617 0 3.89 KB
#4811 AllCycleSimpleBody net6.0 44.1μs 181ns 702ns 0.0223 0 0 2.03 KB
#4811 AllCycleSimpleBody netcoreapp3.1 45.6μs 92.3ns 358ns 0.0228 0 0 2.01 KB
#4811 AllCycleSimpleBody net472 47μs 128ns 494ns 0.326 0 0 2.08 KB
#4811 AllCycleMoreComplexBody net6.0 231μs 99.4ns 385ns 0 0 0 8.63 KB
#4811 AllCycleMoreComplexBody netcoreapp3.1 236μs 133ns 515ns 0 0 0 8.52 KB
#4811 AllCycleMoreComplexBody net472 240μs 138ns 536ns 1.31 0 0 8.7 KB
#4811 ObjectExtractorSimpleBody net6.0 121ns 0.122ns 0.455ns 0.00392 0 0 280 B
#4811 ObjectExtractorSimpleBody netcoreapp3.1 179ns 0.0986ns 0.382ns 0.00371 0 0 272 B
#4811 ObjectExtractorSimpleBody net472 145ns 0.0703ns 0.272ns 0.0446 0 0 281 B
#4811 ObjectExtractorMoreComplexBody net6.0 2.99μs 1.28ns 4.78ns 0.0541 0 0 3.88 KB
#4811 ObjectExtractorMoreComplexBody netcoreapp3.1 4.16μs 1.87ns 7.23ns 0.0501 0 0 3.78 KB
#4811 ObjectExtractorMoreComplexBody net472 4.18μs 3.46ns 13.4ns 0.617 0.0063 0 3.89 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 12.7μs 3.41ns 12.8ns 0.133 0 0 9.42 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 20.1μs 6.3ns 23.6ns 0.121 0 0 9.42 KB
master RunWaf(args=NestedMap (10)) net472 28.3μs 15.8ns 61.4ns 1.5 0 0 9.48 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 76.4μs 33.8ns 131ns 0.23 0 0 15.77 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 84.6μs 65.8ns 228ns 0.214 0 0 15.72 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 98.1μs 55.1ns 213ns 2.54 0 0 16.04 KB
master RunWaf(args=NestedMap (100)) net6.0 24.1μs 13ns 50.5ns 0.278 0 0 19.66 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 38μs 12.1ns 43.5ns 0.266 0 0 20.42 KB
master RunWaf(args=NestedMap (100)) net472 54.5μs 22ns 82.4ns 3.26 0.0544 0 20.63 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 93.2μs 45.2ns 169ns 0.325 0 0 26.01 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 112μs 90.2ns 349ns 0.337 0 0 26.72 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 131μs 127ns 490ns 4.3 0.0651 0 27.19 KB
master RunWaf(args=NestedMap (20)) net6.0 24.3μs 16.6ns 64.4ns 0.267 0 0 19.4 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 37.5μs 23.2ns 87ns 0.261 0 0 19.84 KB
master RunWaf(args=NestedMap (20)) net472 53.4μs 12.7ns 43.9ns 3.16 0.0535 0 20.04 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 91.6μs 72.8ns 282ns 0.364 0 0 25.74 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 110μs 121ns 453ns 0.33 0 0 26.14 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 133μs 199ns 770ns 4.22 0.0659 0 26.6 KB
#4811 RunWaf(args=NestedMap (10)) net6.0 12.6μs 12.8ns 47.7ns 0.133 0 0 9.42 KB
#4811 RunWaf(args=NestedMap (10)) netcoreapp3.1 19.4μs 4.62ns 16ns 0.127 0 0 9.42 KB
#4811 RunWaf(args=NestedMap (10)) net472 28.2μs 15.5ns 57.8ns 1.49 0 0 9.48 KB
#4811 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 76.2μs 42.2ns 158ns 0.189 0 0 15.77 KB
#4811 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 86.2μs 166ns 644ns 0.216 0 0 15.72 KB
#4811 RunWafWithAttack(args=Neste(...)tack) [22]) net472 98.3μs 40.3ns 156ns 2.52 0 0 16.04 KB
#4811 RunWaf(args=NestedMap (100)) net6.0 23.9μs 11.8ns 44.2ns 0.276 0 0 19.66 KB
#4811 RunWaf(args=NestedMap (100)) netcoreapp3.1 38μs 13.6ns 52.9ns 0.266 0 0 20.42 KB
#4811 RunWaf(args=NestedMap (100)) net472 54μs 33.2ns 124ns 3.26 0.0539 0 20.63 KB
#4811 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 92.4μs 91.6ns 355ns 0.371 0 0 26.01 KB
#4811 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 108μs 184ns 687ns 0.324 0 0 26.72 KB
#4811 RunWafWithAttack(args=Neste(...)tack) [23]) net472 131μs 45.8ns 177ns 4.27 0.0647 0 27.19 KB
#4811 RunWaf(args=NestedMap (20)) net6.0 23.5μs 8.73ns 30.3ns 0.271 0 0 19.4 KB
#4811 RunWaf(args=NestedMap (20)) netcoreapp3.1 37.7μs 12.7ns 49.3ns 0.264 0 0 19.84 KB
#4811 RunWaf(args=NestedMap (20)) net472 56.8μs 32.5ns 121ns 3.18 0.0567 0 20.04 KB
#4811 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 90.2μs 40.6ns 157ns 0.362 0 0 25.74 KB
#4811 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 107μs 130ns 504ns 0.32 0 0 26.14 KB
#4811 RunWafWithAttack(args=Neste(...)tack) [22]) net472 129μs 42.6ns 154ns 4.19 0.0644 0 26.6 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 169μs 228ns 883ns 0.252 0 0 18.04 KB
master SendRequest netcoreapp3.1 191μs 297ns 1.15μs 0.189 0 0 20.2 KB
master SendRequest net472 0.00123ns 0.000403ns 0.00151ns 0 0 0 0 b
#4811 SendRequest net6.0 166μs 90.4ns 350ns 0.248 0 0 18.04 KB
#4811 SendRequest netcoreapp3.1 191μs 306ns 1.18μs 0.19 0 0 20.2 KB
#4811 SendRequest net472 0.0016ns 0.00044ns 0.00171ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 538μs 1.3μs 5.05μs 0.53 0 0 41.54 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 672μs 1.06μs 3.95μs 0.338 0 0 41.69 KB
master WriteAndFlushEnrichedTraces net472 840μs 3.2μs 12.4μs 8.12 2.44 0.406 53.26 KB
#4811 WriteAndFlushEnrichedTraces net6.0 539μs 425ns 1.65μs 0.541 0 0 41.59 KB
#4811 WriteAndFlushEnrichedTraces netcoreapp3.1 656μs 890ns 3.45μs 0.324 0 0 41.59 KB
#4811 WriteAndFlushEnrichedTraces net472 882μs 4.36μs 18μs 8.3 2.62 0.437 53.23 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.03μs 0.908ns 3.52ns 0.0109 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.24μs 0.381ns 1.42ns 0.0105 0 0 768 B
master ExecuteNonQuery net472 1.65μs 1.54ns 5.56ns 0.115 0 0 730 B
#4811 ExecuteNonQuery net6.0 1.07μs 0.651ns 2.52ns 0.0107 0 0 768 B
#4811 ExecuteNonQuery netcoreapp3.1 1.35μs 0.407ns 1.58ns 0.0101 0 0 768 B
#4811 ExecuteNonQuery net472 1.72μs 3.8ns 14.7ns 0.115 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.21μs 0.504ns 1.95ns 0.0127 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.38μs 0.79ns 2.96ns 0.0124 0 0 936 B
master CallElasticsearch net472 2.21μs 0.732ns 2.83ns 0.151 0 0 955 B
master CallElasticsearchAsync net6.0 1.16μs 0.411ns 1.59ns 0.0128 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.41μs 0.655ns 2.45ns 0.0133 0 0 984 B
master CallElasticsearchAsync net472 2.49μs 1.04ns 4.04ns 0.16 0 0 1.01 KB
#4811 CallElasticsearch net6.0 1.25μs 0.39ns 1.46ns 0.0131 0 0 936 B
#4811 CallElasticsearch netcoreapp3.1 1.36μs 0.683ns 2.64ns 0.0123 0 0 936 B
#4811 CallElasticsearch net472 2.31μs 0.82ns 3.18ns 0.151 0 0 955 B
#4811 CallElasticsearchAsync net6.0 1.2μs 0.89ns 3.45ns 0.0126 0 0 912 B
#4811 CallElasticsearchAsync netcoreapp3.1 1.55μs 0.437ns 1.69ns 0.0131 0 0 984 B
#4811 CallElasticsearchAsync net472 2.51μs 1.08ns 4.19ns 0.16 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.27μs 0.505ns 1.96ns 0.0127 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.41μs 0.499ns 1.93ns 0.012 0 0 912 B
master ExecuteAsync net472 1.67μs 0.59ns 2.21ns 0.139 0.000837 0 875 B
#4811 ExecuteAsync net6.0 1.25μs 0.836ns 3.24ns 0.0124 0 0 912 B
#4811 ExecuteAsync netcoreapp3.1 1.44μs 0.853ns 3.3ns 0.0123 0 0 912 B
#4811 ExecuteAsync net472 1.67μs 2.95ns 11.4ns 0.139 0 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 3.65μs 0.936ns 3.62ns 0.0256 0 0 1.9 KB
master SendAsync netcoreapp3.1 4.43μs 1.69ns 6.31ns 0.0334 0 0 2.43 KB
master SendAsync net472 7.16μs 1.48ns 5.34ns 0.472 0 0 2.99 KB
#4811 SendAsync net6.0 3.61μs 1.38ns 5.15ns 0.0252 0 0 1.9 KB
#4811 SendAsync netcoreapp3.1 4.49μs 4.78ns 18.5ns 0.0336 0 0 2.43 KB
#4811 SendAsync net472 7.13μs 2.17ns 8.41ns 0.473 0 0 2.99 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.51μs 0.69ns 2.49ns 0.0221 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 1.9μs 0.568ns 2.13ns 0.0207 0 0 1.57 KB
master EnrichedLog net472 2.35μs 3.37ns 12.6ns 0.237 0 0 1.49 KB
#4811 EnrichedLog net6.0 1.38μs 0.436ns 1.63ns 0.0219 0 0 1.57 KB
#4811 EnrichedLog netcoreapp3.1 2.09μs 0.92ns 3.56ns 0.021 0 0 1.57 KB
#4811 EnrichedLog net472 2.21μs 2.37ns 8.87ns 0.237 0 0 1.49 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 112μs 105ns 408ns 0.056 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 117μs 119ns 460ns 0.059 0 0 4.21 KB
master EnrichedLog net472 148μs 105ns 406ns 0.664 0.221 0 4.38 KB
#4811 EnrichedLog net6.0 115μs 161ns 625ns 0.0576 0 0 4.21 KB
#4811 EnrichedLog netcoreapp3.1 119μs 222ns 799ns 0 0 0 4.21 KB
#4811 EnrichedLog net472 147μs 128ns 494ns 0.664 0.221 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.87μs 1.03ns 3.98ns 0.0289 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 3.91μs 1.34ns 4.82ns 0.0276 0 0 2.13 KB
master EnrichedLog net472 4.57μs 1.97ns 7.11ns 0.306 0 0 1.93 KB
#4811 EnrichedLog net6.0 2.79μs 1.46ns 5.66ns 0.0296 0 0 2.13 KB
#4811 EnrichedLog netcoreapp3.1 3.9μs 1.52ns 5.87ns 0.029 0 0 2.13 KB
#4811 EnrichedLog net472 4.46μs 1.87ns 7.24ns 0.306 0 0 1.93 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.26μs 0.436ns 1.63ns 0.0158 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.57μs 1.54ns 5.97ns 0.0148 0 0 1.1 KB
master SendReceive net472 2.02μs 2.49ns 9.64ns 0.177 0 0 1.12 KB
#4811 SendReceive net6.0 1.27μs 0.617ns 2.31ns 0.0153 0 0 1.1 KB
#4811 SendReceive netcoreapp3.1 1.57μs 0.725ns 2.81ns 0.0149 0 0 1.1 KB
#4811 SendReceive net472 1.98μs 2.27ns 8.78ns 0.177 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.52μs 0.925ns 3.58ns 0.0215 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.75μs 0.62ns 2.32ns 0.0203 0 0 1.58 KB
master EnrichedLog net472 3.98μs 1.56ns 6.03ns 0.309 0 0 1.96 KB
#4811 EnrichedLog net6.0 2.62μs 1.72ns 6.44ns 0.021 0 0 1.53 KB
#4811 EnrichedLog netcoreapp3.1 3.54μs 0.818ns 3.06ns 0.0214 0 0 1.58 KB
#4811 EnrichedLog net472 4.05μs 1.16ns 4.47ns 0.309 0 0 1.96 KB
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #4811

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 1.281 717.78 919.21
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 1.144 394.35 450.94
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.138 453.27 515.67
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472 1.124 618.74 695.49

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 394ns 0.107ns 0.401ns 0.00749 0 0 536 B
master StartFinishSpan netcoreapp3.1 535ns 0.165ns 0.573ns 0.00733 0 0 536 B
master StartFinishSpan net472 619ns 0.193ns 0.747ns 0.0851 0 0 538 B
master StartFinishScope net6.0 453ns 0.128ns 0.496ns 0.00913 0 0 656 B
master StartFinishScope netcoreapp3.1 616ns 0.208ns 0.807ns 0.009 0 0 656 B
master StartFinishScope net472 718ns 0.446ns 1.73ns 0.0979 0 0 618 B
#4811 StartFinishSpan net6.0 451ns 0.141ns 0.528ns 0.00747 0 0 536 B
#4811 StartFinishSpan netcoreapp3.1 546ns 0.106ns 0.383ns 0.00717 0 0 536 B
#4811 StartFinishSpan net472 696ns 0.181ns 0.702ns 0.0852 0 0 538 B
#4811 StartFinishScope net6.0 516ns 0.181ns 0.701ns 0.00913 0 0 656 B
#4811 StartFinishScope netcoreapp3.1 670ns 0.295ns 1.11ns 0.00912 0 0 656 B
#4811 StartFinishScope net472 920ns 0.533ns 2.06ns 0.098 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #4811

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.173 618.02 725.00

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 618ns 0.638ns 2.47ns 0.00918 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 757ns 0.302ns 1.17ns 0.00873 0 0 656 B
master RunOnMethodBegin net472 990ns 0.778ns 3.01ns 0.0982 0 0 618 B
#4811 RunOnMethodBegin net6.0 725ns 0.279ns 1.08ns 0.00913 0 0 656 B
#4811 RunOnMethodBegin netcoreapp3.1 774ns 0.249ns 0.861ns 0.0087 0 0 656 B
#4811 RunOnMethodBegin net472 1.02μs 0.165ns 0.638ns 0.0978 0 0 618 B

andrewlock avatar Nov 04 '23 00:11 andrewlock

Datadog Report

Branch report: andrew/ci/release-branches Commit report: dfaddf0

:white_check_mark: dd-trace-dotnet: 0 Failed, 0 New Flaky, 304036 Passed, 978 Skipped, 40m 49.48s Wall Time

datadog-ddstaging[bot] avatar Nov 17 '23 17:11 datadog-ddstaging[bot]

Datadog Report

Branch report: andrew/ci/release-branches Commit report: 9b8e168 Test service: dd-trace-dotnet

:white_check_mark: 0 Failed, 337630 Passed, 1615 Skipped, 45m 48.71s Wall Time

datadog-ddstaging[bot] avatar Mar 26 '24 15:03 datadog-ddstaging[bot]

Benchmarks Report for tracer :snail:

Benchmarks for #4811 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.143
  • 1 benchmarks are slower, with geometric mean 1.142
  • 1 benchmarks have fewer allocations
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.83μs 49.5ns 328ns 0.0173 0.00433 0 7.51 KB
master StartStopWithChild netcoreapp3.1 10.5μs 58.4ns 369ns 0.0256 0.0102 0 7.6 KB
master StartStopWithChild net472 17.1μs 60.8ns 236ns 1.36 0.356 0.119 8.06 KB
#4811 StartStopWithChild net6.0 8.58μs 46.5ns 267ns 0.0177 0.00444 0 7.51 KB
#4811 StartStopWithChild netcoreapp3.1 10.6μs 59.5ns 399ns 0.0374 0.016 0 7.6 KB
#4811 StartStopWithChild net472 16.9μs 64.3ns 249ns 1.35 0.341 0.0916 8.05 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 495μs 222ns 832ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 615μs 118ns 426ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 814μs 252ns 977ns 0.406 0 0 3.3 KB
#4811 WriteAndFlushEnrichedTraces net6.0 455μs 242ns 907ns 0 0 0 2.7 KB
#4811 WriteAndFlushEnrichedTraces netcoreapp3.1 611μs 218ns 785ns 0 0 0 2.7 KB
#4811 WriteAndFlushEnrichedTraces net472 814μs 195ns 730ns 0.406 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 172μs 221ns 856ns 0.258 0 0 18.33 KB
master SendRequest netcoreapp3.1 193μs 124ns 463ns 0.193 0 0 20.49 KB
master SendRequest net472 0.000894ns 0.000269ns 0.00104ns 0 0 0 0 b
#4811 SendRequest net6.0 171μs 159ns 575ns 0.255 0 0 18.33 KB
#4811 SendRequest netcoreapp3.1 193μs 110ns 410ns 0.191 0 0 20.5 KB
#4811 SendRequest net472 0.000802ns 0.000254ns 0.000984ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: More allocations :warning:

More allocations :warning: in #4811

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.7 KB 42.01 KB 311 B 0.75%

Fewer allocations :tada: in #4811

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.82 KB 41.49 KB -332 B -0.79%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 556μs 640ns 2.48μs 0.571 0 0 41.82 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 661μs 817ns 3.16μs 0.331 0 0 41.7 KB
master WriteAndFlushEnrichedTraces net472 863μs 4.15μs 16.6μs 8.19 2.59 0.431 53.24 KB
#4811 WriteAndFlushEnrichedTraces net6.0 533μs 626ns 2.42μs 0.548 0 0 41.49 KB
#4811 WriteAndFlushEnrichedTraces netcoreapp3.1 652μs 1.77μs 6.87μs 0.322 0 0 42.01 KB
#4811 WriteAndFlushEnrichedTraces net472 878μs 2.72μs 9.42μs 8.42 2.66 0.443 53.23 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.17μs 0.656ns 2.54ns 0.0106 0 0 784 B
master ExecuteNonQuery netcoreapp3.1 1.52μs 0.491ns 1.84ns 0.0105 0 0 784 B
master ExecuteNonQuery net472 1.77μs 2.66ns 9.96ns 0.118 0 0 746 B
#4811 ExecuteNonQuery net6.0 1.2μs 0.679ns 2.63ns 0.0109 0 0 784 B
#4811 ExecuteNonQuery netcoreapp3.1 1.51μs 0.697ns 2.51ns 0.0106 0 0 784 B
#4811 ExecuteNonQuery net472 1.85μs 0.704ns 2.54ns 0.118 0 0 746 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.32μs 0.836ns 3.24ns 0.0138 0 0 1.01 KB
master CallElasticsearch netcoreapp3.1 1.56μs 0.76ns 2.95ns 0.0133 0 0 1.01 KB
master CallElasticsearch net472 2.5μs 0.854ns 3.31ns 0.162 0 0 1.02 KB
master CallElasticsearchAsync net6.0 1.32μs 0.991ns 3.84ns 0.0137 0 0 984 B
master CallElasticsearchAsync netcoreapp3.1 1.72μs 0.942ns 3.4ns 0.0143 0 0 1.06 KB
master CallElasticsearchAsync net472 2.78μs 2.23ns 8.36ns 0.17 0 0 1.08 KB
#4811 CallElasticsearch net6.0 1.39μs 0.913ns 3.54ns 0.0139 0 0 1.01 KB
#4811 CallElasticsearch netcoreapp3.1 1.56μs 0.627ns 2.34ns 0.0136 0 0 1.01 KB
#4811 CallElasticsearch net472 2.59μs 2.49ns 8.99ns 0.162 0 0 1.02 KB
#4811 CallElasticsearchAsync net6.0 1.32μs 0.661ns 2.56ns 0.0139 0 0 984 B
#4811 CallElasticsearchAsync netcoreapp3.1 1.62μs 1.12ns 4.35ns 0.0145 0 0 1.06 KB
#4811 CallElasticsearchAsync net472 2.75μs 4.41ns 16.5ns 0.17 0 0 1.08 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.34μs 0.924ns 3.58ns 0.0128 0 0 928 B
master ExecuteAsync netcoreapp3.1 1.73μs 1.59ns 5.95ns 0.0129 0 0 928 B
master ExecuteAsync net472 1.9μs 1.03ns 4.01ns 0.141 0 0 891 B
#4811 ExecuteAsync net6.0 1.34μs 0.544ns 2.11ns 0.0128 0 0 928 B
#4811 ExecuteAsync netcoreapp3.1 1.7μs 0.778ns 2.91ns 0.0121 0 0 928 B
#4811 ExecuteAsync net472 1.96μs 0.579ns 2.24ns 0.141 0 0 891 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.04μs 3.32ns 12.9ns 0.0303 0 0 2.16 KB
master SendAsync netcoreapp3.1 5.08μs 4.72ns 18.3ns 0.0382 0 0 2.7 KB
master SendAsync net472 7.72μs 4.58ns 17.7ns 0.483 0 0 3.05 KB
#4811 SendAsync net6.0 4.19μs 1.4ns 5.22ns 0.0297 0 0 2.16 KB
#4811 SendAsync netcoreapp3.1 5.13μs 2.32ns 8.68ns 0.036 0 0 2.7 KB
#4811 SendAsync net472 7.63μs 4.1ns 15.9ns 0.484 0 0 3.05 KB
Benchmarks.Trace.ILoggerBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #4811

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1 1.143 2,399.61 2,099.35

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.48μs 0.876ns 3.16ns 0.0231 0 0 1.65 KB
master EnrichedLog netcoreapp3.1 2.4μs 2.14ns 7.7ns 0.0212 0 0 1.65 KB
master EnrichedLog net472 2.65μs 1.76ns 6.59ns 0.25 0 0 1.57 KB
#4811 EnrichedLog net6.0 1.5μs 0.576ns 2.23ns 0.0232 0 0 1.65 KB
#4811 EnrichedLog netcoreapp3.1 2.1μs 1.14ns 4.26ns 0.022 0 0 1.65 KB
#4811 EnrichedLog net472 2.73μs 1.58ns 6.13ns 0.25 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 112μs 143ns 552ns 0.0558 0 0 4.23 KB
master EnrichedLog netcoreapp3.1 117μs 77.3ns 289ns 0 0 0 4.23 KB
master EnrichedLog net472 148μs 76.5ns 296ns 0.663 0.221 0 4.41 KB
#4811 EnrichedLog net6.0 113μs 98.8ns 382ns 0.0565 0 0 4.23 KB
#4811 EnrichedLog netcoreapp3.1 117μs 232ns 898ns 0 0 0 4.23 KB
#4811 EnrichedLog net472 145μs 89.8ns 324ns 0.654 0.218 0 4.41 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.18μs 1.13ns 4.23ns 0.0302 0 0 2.21 KB
master EnrichedLog netcoreapp3.1 4.31μs 2.14ns 8.28ns 0.0304 0 0 2.21 KB
master EnrichedLog net472 4.95μs 2.42ns 9.37ns 0.32 0 0 2.02 KB
#4811 EnrichedLog net6.0 2.97μs 1.17ns 4.36ns 0.031 0 0 2.21 KB
#4811 EnrichedLog netcoreapp3.1 4.15μs 2.54ns 9.83ns 0.029 0 0 2.21 KB
#4811 EnrichedLog net472 4.96μs 1.67ns 6.01ns 0.319 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.45μs 0.712ns 2.67ns 0.0166 0 0 1.18 KB
master SendReceive netcoreapp3.1 1.86μs 1.01ns 3.92ns 0.0159 0 0 1.18 KB
master SendReceive net472 2.24μs 1.11ns 4.14ns 0.187 0 0 1.18 KB
#4811 SendReceive net6.0 1.46μs 0.648ns 2.51ns 0.0161 0 0 1.18 KB
#4811 SendReceive netcoreapp3.1 1.83μs 1.26ns 4.54ns 0.0163 0 0 1.18 KB
#4811 SendReceive net472 2.21μs 2.17ns 8.41ns 0.187 0 0 1.18 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.64μs 0.68ns 2.63ns 0.0212 0 0 1.55 KB
master EnrichedLog netcoreapp3.1 3.98μs 4.5ns 16.8ns 0.0219 0 0 1.6 KB
master EnrichedLog net472 4.47μs 1.91ns 7.14ns 0.314 0 0 1.99 KB
#4811 EnrichedLog net6.0 2.69μs 1.1ns 4.27ns 0.0214 0 0 1.55 KB
#4811 EnrichedLog netcoreapp3.1 4μs 7.83ns 29.3ns 0.0217 0 0 1.6 KB
#4811 EnrichedLog net472 4.47μs 2.76ns 10.7ns 0.316 0 0 1.99 KB
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #4811

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.142 543.88 621.09

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 469ns 0.574ns 2.15ns 0.00774 0 0 552 B
master StartFinishSpan netcoreapp3.1 747ns 1.27ns 4.93ns 0.00755 0 0 552 B
master StartFinishSpan net472 817ns 1.4ns 5.24ns 0.0879 0 0 554 B
master StartFinishScope net6.0 544ns 0.687ns 2.66ns 0.00949 0 0 672 B
master StartFinishScope netcoreapp3.1 873ns 0.94ns 3.64ns 0.00894 0 0 672 B
master StartFinishScope net472 936ns 1.21ns 4.35ns 0.101 0 0 634 B
#4811 StartFinishSpan net6.0 458ns 0.829ns 3.1ns 0.00767 0 0 552 B
#4811 StartFinishSpan netcoreapp3.1 704ns 0.928ns 3.6ns 0.00741 0 0 552 B
#4811 StartFinishSpan net472 748ns 0.978ns 3.66ns 0.0879 0 0 554 B
#4811 StartFinishScope net6.0 620ns 0.932ns 3.61ns 0.00946 0 0 672 B
#4811 StartFinishScope netcoreapp3.1 873ns 1.42ns 5.5ns 0.00882 0 0 672 B
#4811 StartFinishScope net472 929ns 1.28ns 4.95ns 0.1 0 0 634 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 601ns 0.905ns 3.5ns 0.00946 0 0 672 B
master RunOnMethodBegin netcoreapp3.1 893ns 1.98ns 7.66ns 0.00893 0 0 672 B
master RunOnMethodBegin net472 1.07μs 2.16ns 8.36ns 0.101 0 0 634 B
#4811 RunOnMethodBegin net6.0 653ns 1.14ns 4.42ns 0.00953 0 0 672 B
#4811 RunOnMethodBegin netcoreapp3.1 984ns 1.96ns 7.59ns 0.00888 0 0 672 B
#4811 RunOnMethodBegin net472 1.03μs 2.61ns 10.1ns 0.101 0 0 634 B

andrewlock avatar Mar 26 '24 16:03 andrewlock