dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard
verified publisher icon DataDog

Rollback Nuget/Rider fixes

Open andrewlock opened this issue 2 years ago • 4 comments

Summary of changes

Rollback the NuGet fixes, now that they're fixed in the latest version of Rider

Reason for change

We don't need them any more

Implementation details

Revert the commits that added them, re-add the extra tweaks

Test coverage

N/A

Other details

We could just leave them. They don't do any harm 🤷

andrewlock avatar Dec 27 '23 13:12 andrewlock

Datadog Report

Branch report: andrew/rollback-rider-changes Commit report: b395d19

:white_check_mark: dd-trace-dotnet: 0 Failed, 0 New Flaky, 303925 Passed, 994 Skipped, 54m 35.24s Wall Time

datadog-ddstaging[bot] avatar Dec 27 '23 13:12 datadog-ddstaging[bot]

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5006) - mean (71ms)  : 63, 79
     .   : milestone, 71,
    master - mean (71ms)  : 63, 80
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (5006) - mean (1,031ms)  : 1004, 1057
     .   : milestone, 1031,
    master - mean (1,018ms)  : 1001, 1034
     .   : milestone, 1018,


gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5006) - mean (106ms)  : 104, 108
     .   : milestone, 106,
    master - mean (106ms)  : 102, 110
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (5006) - mean (733ms)  : 714, 751
     .   : milestone, 733,
    master - mean (721ms)  : 699, 744
     .   : milestone, 721,


gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5006) - mean (91ms)  : 87, 95
     .   : milestone, 91,
    master - mean (90ms)  : 83, 96
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (5006) - mean (693ms)  : 675, 711
     .   : milestone, 693,
    master - mean (692ms)  : 671, 714
     .   : milestone, 692,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5006) - mean (188ms)  : 186, 191
     .   : milestone, 188,
    master - mean (188ms)  : 186, 191
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (5006) - mean (1,128ms)  : 1107, 1148
     .   : milestone, 1128,
    master - mean (1,126ms)  : 1098, 1154
     .   : milestone, 1126,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5006) - mean (272ms)  : 269, 276
     .   : milestone, 272,
    master - mean (273ms)  : 267, 278
     .   : milestone, 273,

    section CallTarget+Inlining+NGEN
    This PR (5006) - mean (1,084ms)  : 1059, 1109
     .   : milestone, 1084,
    master - mean (1,083ms)  : 1058, 1108
     .   : milestone, 1083,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5006) - mean (262ms)  : 259, 265
     .   : milestone, 262,
    master - mean (262ms)  : 259, 265
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (5006) - mean (1,050ms)  : 1025, 1076
     .   : milestone, 1050,
    master - mean (1,047ms)  : 1023, 1071
     .   : milestone, 1047,

andrewlock avatar Dec 27 '23 13:12 andrewlock

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5006) (11.117M)   : 0, 11116668
    master (11.071M)   : 0, 11070623
    benchmarks/2.9.0 (10.850M)   : 0, 10849500

    section Automatic
    This PR (5006) (7.584M)   : 0, 7584487
    master (7.621M)   : 0, 7621454
    benchmarks/2.9.0 (7.823M)   : 0, 7822730

    section Trace stats
    This PR (5006) (7.886M)   : 0, 7886391
    master (7.915M)   : 0, 7914703

    section Manual
    This PR (5006) (9.668M)   : 0, 9668435
    master (9.656M)   : 0, 9656230

    section Manual + Automatic
    This PR (5006) (7.095M)   : 0, 7095036
    master (7.357M)   : 0, 7357268

    section Version Conflict
    This PR (5006) (6.473M)   : 0, 6472932
    master (6.783M)   : 0, 6783047


gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5006) (9.591M)   : 0, 9590576
    master (9.471M)   : 0, 9471361
    benchmarks/2.9.0 (9.680M)   : 0, 9679996

    section Automatic
    This PR (5006) (6.574M)   : 0, 6574058
    master (6.618M)   : 0, 6618111

    section Trace stats
    This PR (5006) (6.788M)   : 0, 6787760
    master (6.926M)   : 0, 6925986

    section Manual
    This PR (5006) (8.134M)   : 0, 8133556
    master (8.211M)   : 0, 8210732

    section Manual + Automatic
    This PR (5006) (6.125M)   : 0, 6125203
    master (6.248M)   : 0, 6247903

    section Version Conflict
    This PR (5006) (5.528M)   : 0, 5527970
    master (5.788M)   : 0, 5788282


gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5006) (10.998M)   : 0, 10997793
    master (10.761M)   : 0, 10760960
    benchmarks/2.9.0 (10.863M)   : 0, 10863181

    section Automatic
    This PR (5006) (7.764M)   : 0, 7763919
    master (7.646M)   : 0, 7646171
    benchmarks/2.9.0 (8.130M)   : 0, 8129748

    section Trace stats
    This PR (5006) (7.865M)   : 0, 7864777
    master (7.986M)   : 0, 7985757

    section Manual
    This PR (5006) (9.136M)   : 0, 9136024
    master (9.414M)   : 0, 9413937

    section Manual + Automatic
    This PR (5006) (6.925M)   : crit ,0, 6925285
    master (7.309M)   : 0, 7309209

    section Version Conflict
    This PR (5006) (6.390M)   : 0, 6389939
    master (6.536M)   : 0, 6536418


gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    master (7.290M)   : 0, 7289762
    benchmarks/2.9.0 (7.835M)   : 0, 7834738

    section No attack
    master (1.970M)   : 0, 1969964
    benchmarks/2.9.0 (3.201M)   : 0, 3200714

    section Attack
    master (1.572M)   : 0, 1571978
    benchmarks/2.9.0 (2.469M)   : 0, 2468672

    section Blocking
    master (3.262M)   : 0, 3262059

    section IAST default
    master (6.494M)   : 0, 6494125

    section IAST full
    master (5.795M)   : 0, 5794600

    section Base vuln
    master (0.933M)   : 0, 933398

    section IAST vuln
    master (0.871M)   : 0, 871187

andrewlock avatar Dec 27 '23 15:12 andrewlock

Benchmarks Report :snail:

Benchmarks for #5006 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.151
  • 1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.9μs 48ns 280ns 0.0303 0.013 0.00433 7.48 KB
master StartStopWithChild netcoreapp3.1 10.9μs 60.3ns 372ns 0.0264 0.0106 0 7.58 KB
master StartStopWithChild net472 17.1μs 34.8ns 135ns 1.34 0.358 0.119 7.95 KB
#5006 StartStopWithChild net6.0 8.69μs 46.6ns 255ns 0.0246 0.00819 0 7.48 KB
#5006 StartStopWithChild netcoreapp3.1 10.7μs 59.8ns 392ns 0.031 0.0155 0 7.58 KB
#5006 StartStopWithChild net472 16.9μs 40.2ns 156ns 1.34 0.373 0.11 7.95 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 450μs 324ns 1.17μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 616μs 197ns 736ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 777μs 383ns 1.48μs 0.386 0 0 3.3 KB
#5006 WriteAndFlushEnrichedTraces net6.0 463μs 87.7ns 316ns 0 0 0 2.7 KB
#5006 WriteAndFlushEnrichedTraces netcoreapp3.1 628μs 162ns 604ns 0 0 0 2.7 KB
#5006 WriteAndFlushEnrichedTraces net472 788μs 477ns 1.85μs 0.393 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 43.3μs 114ns 441ns 0.0219 0 0 1.77 KB
master AllCycleSimpleBody netcoreapp3.1 46.5μs 63.5ns 238ns 0 0 0 1.74 KB
master AllCycleSimpleBody net472 51μs 44.4ns 172ns 0.278 0 0 1.81 KB
master AllCycleMoreComplexBody net6.0 215μs 233ns 904ns 0.108 0 0 9.25 KB
master AllCycleMoreComplexBody netcoreapp3.1 243μs 191ns 714ns 0.122 0 0 9.14 KB
master AllCycleMoreComplexBody net472 239μs 86.9ns 325ns 1.43 0 0 9.32 KB
master ObjectExtractorSimpleBody net6.0 132ns 0.087ns 0.314ns 0.00395 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 188ns 0.0637ns 0.247ns 0.0037 0 0 272 B
master ObjectExtractorSimpleBody net472 164ns 0.168ns 0.629ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 2.82μs 1.06ns 4.1ns 0.0536 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.63μs 0.71ns 2.66ns 0.049 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 3.4μs 1.11ns 4.16ns 0.603 0.00512 0 3.8 KB
#5006 AllCycleSimpleBody net6.0 43.4μs 151ns 545ns 0.0219 0 0 1.77 KB
#5006 AllCycleSimpleBody netcoreapp3.1 47μs 109ns 422ns 0.024 0 0 1.74 KB
#5006 AllCycleSimpleBody net472 49.7μs 34.6ns 134ns 0.271 0 0 1.81 KB
#5006 AllCycleMoreComplexBody net6.0 216μs 68.6ns 257ns 0.108 0 0 9.25 KB
#5006 AllCycleMoreComplexBody netcoreapp3.1 227μs 245ns 948ns 0.114 0 0 9.14 KB
#5006 AllCycleMoreComplexBody net472 240μs 214ns 829ns 1.44 0 0 9.32 KB
#5006 ObjectExtractorSimpleBody net6.0 131ns 0.0374ns 0.14ns 0.00391 0 0 280 B
#5006 ObjectExtractorSimpleBody netcoreapp3.1 181ns 0.134ns 0.521ns 0.00374 0 0 272 B
#5006 ObjectExtractorSimpleBody net472 155ns 0.398ns 1.54ns 0.0446 0 0 281 B
#5006 ObjectExtractorMoreComplexBody net6.0 2.74μs 1.32ns 4.94ns 0.0533 0 0 3.78 KB
#5006 ObjectExtractorMoreComplexBody netcoreapp3.1 3.54μs 1.69ns 6.1ns 0.0496 0 0 3.69 KB
#5006 ObjectExtractorMoreComplexBody net472 3.41μs 0.779ns 2.92ns 0.602 0.00509 0 3.8 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 49.2μs 20.7ns 74.5ns 0.222 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 70.9μs 333ns 1.29μs 0.212 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) net472 98.1μs 521ns 2.95μs 2.56 0.0947 0 16.14 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 117μs 52.2ns 188ns 0.301 0 0 22.41 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 134μs 72.6ns 262ns 0.286 0 0 22.36 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 172μs 857ns 3.64μs 3.56 0.166 0 22.7 KB
master RunWaf(args=NestedMap (100)) net6.0 96.8μs 26.7ns 103ns 0.436 0 0 32.76 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 131μs 681ns 3.47μs 0.405 0 0 33.33 KB
master RunWaf(args=NestedMap (100)) net472 189μs 891ns 3.45μs 5.32 0.367 0 33.67 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 165μs 58.8ns 220ns 0.493 0 0 39.1 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 209μs 1.01μs 3.9μs 0.528 0 0 39.63 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 271μs 202ns 783ns 6.38 0.521 0 40.23 KB
master RunWaf(args=NestedMap (20)) net6.0 102μs 570ns 3.56μs 0.423 0 0 32.18 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 137μs 109ns 424ns 0.433 0 0 32.3 KB
master RunWaf(args=NestedMap (20)) net472 186μs 931ns 4.06μs 5.16 0.362 0 32.63 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 172μs 847ns 3.88μs 0.523 0 0 38.53 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 209μs 1.15μs 6.59μs 0.498 0 0 38.6 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 264μs 248ns 959ns 6.17 0.536 0 39.2 KB
#5006 RunWaf(args=NestedMap (10)) net6.0 52.3μs 189ns 733ns 0.227 0 0 16.06 KB
#5006 RunWaf(args=NestedMap (10)) netcoreapp3.1 69.7μs 352ns 1.53μs 0.207 0 0 16.06 KB
#5006 RunWaf(args=NestedMap (10)) net472 98.6μs 57.7ns 224ns 2.54 0.0941 0 16.14 KB
#5006 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 123μs 379ns 1.47μs 0.295 0 0 22.41 KB
#5006 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 137μs 571ns 2.55μs 0.269 0 0 22.36 KB
#5006 RunWafWithAttack(args=Neste(...)tack) [22]) net472 171μs 807ns 3.12μs 3.61 0.168 0 22.7 KB
#5006 RunWaf(args=NestedMap (100)) net6.0 107μs 185ns 715ns 0.421 0 0 32.76 KB
#5006 RunWaf(args=NestedMap (100)) netcoreapp3.1 134μs 289ns 1.12μs 0.456 0 0 33.33 KB
#5006 RunWaf(args=NestedMap (100)) net472 183μs 90.4ns 350ns 5.31 0.366 0 33.67 KB
#5006 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 174μs 968ns 5.96μs 0.582 0 0 39.1 KB
#5006 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 210μs 997ns 4.23μs 0.425 0 0 39.63 KB
#5006 RunWafWithAttack(args=Neste(...)tack) [23]) net472 273μs 210ns 785ns 6.37 0.407 0 40.23 KB
#5006 RunWaf(args=NestedMap (20)) net6.0 102μs 547ns 2.89μs 0.457 0 0 32.18 KB
#5006 RunWaf(args=NestedMap (20)) netcoreapp3.1 129μs 258ns 1E+03ns 0.381 0 0 32.3 KB
#5006 RunWaf(args=NestedMap (20)) net472 190μs 950ns 4.14μs 5.11 0.365 0 32.63 KB
#5006 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 172μs 63.6ns 246ns 0.494 0 0 38.53 KB
#5006 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 212μs 421ns 1.46μs 0.517 0 0 38.6 KB
#5006 RunWafWithAttack(args=Neste(...)tack) [22]) net472 262μs 177ns 663ns 6.16 0.524 0 39.2 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 172μs 115ns 447ns 0.257 0 0 18.25 KB
master SendRequest netcoreapp3.1 194μs 209ns 783ns 0.193 0 0 20.41 KB
master SendRequest net472 0.000212ns 0.000113ns 0.000421ns 0 0 0 0 b
#5006 SendRequest net6.0 169μs 189ns 731ns 0.17 0 0 18.25 KB
#5006 SendRequest netcoreapp3.1 190μs 303ns 1.17μs 0.19 0 0 20.41 KB
#5006 SendRequest net472 0.000463ns 0.00019ns 0.000735ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #5006

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 42 KB 41.57 KB -434 B -1.03%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 557μs 2.68μs 11.1μs 0.539 0 0 41.63 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 655μs 879ns 3.29μs 0.329 0 0 42 KB
master WriteAndFlushEnrichedTraces net472 855μs 4.12μs 16.5μs 8.39 2.52 0.419 53.22 KB
#5006 WriteAndFlushEnrichedTraces net6.0 545μs 1.17μs 4.52μs 0.541 0 0 41.72 KB
#5006 WriteAndFlushEnrichedTraces netcoreapp3.1 658μs 1.57μs 6.3μs 0.329 0 0 41.57 KB
#5006 WriteAndFlushEnrichedTraces net472 846μs 4.26μs 18.1μs 8.08 2.55 0.425 53.25 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.15μs 0.619ns 2.4ns 0.0104 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.46μs 1.85ns 7.15ns 0.0101 0 0 768 B
master ExecuteNonQuery net472 1.83μs 0.849ns 3.18ns 0.115 0 0 730 B
#5006 ExecuteNonQuery net6.0 1.13μs 0.395ns 1.48ns 0.0108 0 0 768 B
#5006 ExecuteNonQuery netcoreapp3.1 1.51μs 1ns 3.88ns 0.00998 0 0 768 B
#5006 ExecuteNonQuery net472 1.78μs 0.473ns 1.71ns 0.115 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.12μs 0.596ns 2.31ns 0.0129 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.58μs 2.1ns 8.15ns 0.0127 0 0 936 B
master CallElasticsearch net472 2.6μs 0.659ns 2.55ns 0.152 0 0 955 B
master CallElasticsearchAsync net6.0 1.27μs 0.62ns 2.4ns 0.0127 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.65μs 0.752ns 2.71ns 0.0134 0 0 984 B
master CallElasticsearchAsync net472 2.73μs 1.85ns 6.68ns 0.161 0 0 1.01 KB
#5006 CallElasticsearch net6.0 1.19μs 0.384ns 1.38ns 0.0132 0 0 936 B
#5006 CallElasticsearch netcoreapp3.1 1.5μs 0.676ns 2.34ns 0.0127 0 0 936 B
#5006 CallElasticsearch net472 2.51μs 0.773ns 2.99ns 0.152 0 0 955 B
#5006 CallElasticsearchAsync net6.0 1.28μs 0.527ns 1.97ns 0.0127 0 0 912 B
#5006 CallElasticsearchAsync netcoreapp3.1 1.64μs 0.816ns 2.94ns 0.0131 0 0 984 B
#5006 CallElasticsearchAsync net472 2.66μs 0.524ns 1.96ns 0.16 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.34μs 1.44ns 5.58ns 0.0126 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.54μs 0.328ns 1.23ns 0.0123 0 0 912 B
master ExecuteAsync net472 1.89μs 0.291ns 1.09ns 0.139 0 0 875 B
#5006 ExecuteAsync net6.0 1.42μs 1.45ns 5.6ns 0.0128 0 0 912 B
#5006 ExecuteAsync netcoreapp3.1 1.61μs 0.536ns 2.08ns 0.0121 0 0 912 B
#5006 ExecuteAsync net472 1.76μs 1.38ns 5.15ns 0.139 0 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.12μs 2.35ns 8.78ns 0.0288 0 0 2.1 KB
master SendAsync netcoreapp3.1 4.92μs 2.16ns 8.35ns 0.0345 0 0 2.63 KB
master SendAsync net472 7.67μs 2.76ns 10.7ns 0.525 0 0 3.31 KB
#5006 SendAsync net6.0 4.15μs 1.58ns 5.9ns 0.029 0 0 2.1 KB
#5006 SendAsync netcoreapp3.1 4.84μs 1.51ns 5.86ns 0.0339 0 0 2.63 KB
#5006 SendAsync net472 7.73μs 14.4ns 55.9ns 0.525 0 0 3.31 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #5006

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑netcoreapp3.1 1.151 54,200.00 62,400.00 several?

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 53.7μs 279ns 1.42μs 0 0 0 43.44 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 54.2μs 298ns 1.69μs 0 0 0 42.64 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 38.6μs 211ns 1.18μs 0 0 0 58.37 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 100μs 2.89μs 28.9μs 0 0 0 43.29 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 102μs 3.25μs 31.9μs 0 0 0 42.64 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 64.3μs 295ns 1.47μs 0 0 0 57.34 KB
#5006 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 58.9μs 706ns 6.98μs 0 0 0 43.44 KB
#5006 StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 63.8μs 886ns 8.55μs 0 0 0 42.64 KB
#5006 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 39.7μs 207ns 1.02μs 0 0 0 58.59 KB
#5006 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 101μs 2.62μs 26.2μs 0 0 0 43.29 KB
#5006 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 114μs 3.08μs 30.5μs 0 0 0 42.64 KB
#5006 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 63.8μs 242ns 837ns 0 0 0 57.34 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.4μs 1.17ns 4.38ns 0.0221 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 2.33μs 1.17ns 4.39ns 0.0211 0 0 1.57 KB
master EnrichedLog net472 2.52μs 2.21ns 7.95ns 0.238 0 0 1.5 KB
#5006 EnrichedLog net6.0 1.5μs 0.526ns 2.04ns 0.0218 0 0 1.57 KB
#5006 EnrichedLog netcoreapp3.1 2.21μs 0.706ns 2.45ns 0.0212 0 0 1.57 KB
#5006 EnrichedLog net472 2.69μs 2.62ns 10.2ns 0.237 0 0 1.5 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 116μs 140ns 525ns 0.0582 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 119μs 248ns 960ns 0 0 0 4.21 KB
master EnrichedLog net472 147μs 151ns 584ns 0.658 0.219 0 4.39 KB
#5006 EnrichedLog net6.0 113μs 127ns 476ns 0.0567 0 0 4.21 KB
#5006 EnrichedLog netcoreapp3.1 119μs 124ns 479ns 0.0607 0 0 4.21 KB
#5006 EnrichedLog net472 150μs 236ns 916ns 0.673 0.224 0 4.39 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.15μs 1.13ns 4.39ns 0.0297 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 4.09μs 1.47ns 5.7ns 0.0275 0 0 2.13 KB
master EnrichedLog net472 5.02μs 1.58ns 6.13ns 0.308 0 0 1.95 KB
#5006 EnrichedLog net6.0 2.99μs 1.12ns 4.34ns 0.03 0 0 2.13 KB
#5006 EnrichedLog netcoreapp3.1 4.16μs 3.11ns 12.1ns 0.0288 0 0 2.13 KB
#5006 EnrichedLog net472 4.89μs 1.42ns 5.52ns 0.307 0 0 1.95 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.31μs 0.772ns 2.89ns 0.0157 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.71μs 0.771ns 2.99ns 0.0145 0 0 1.1 KB
master SendReceive net472 2.1μs 0.942ns 3.52ns 0.177 0 0 1.12 KB
#5006 SendReceive net6.0 1.32μs 0.778ns 2.91ns 0.0152 0 0 1.1 KB
#5006 SendReceive netcoreapp3.1 1.76μs 0.835ns 3.01ns 0.0149 0 0 1.1 KB
#5006 SendReceive net472 2.15μs 2.19ns 8.48ns 0.176 0.00107 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.68μs 0.7ns 2.62ns 0.0214 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.87μs 3.34ns 12.9ns 0.0212 0 0 1.58 KB
master EnrichedLog net472 4.25μs 1.36ns 5.27ns 0.31 0 0 1.97 KB
#5006 EnrichedLog net6.0 2.56μs 0.923ns 3.45ns 0.0217 0 0 1.53 KB
#5006 EnrichedLog netcoreapp3.1 3.83μs 2.35ns 9.11ns 0.0209 0 0 1.58 KB
#5006 EnrichedLog net472 4.49μs 1.53ns 5.73ns 0.312 0 0 1.97 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 534ns 0.223ns 0.866ns 0.00751 0 0 536 B
master StartFinishSpan netcoreapp3.1 749ns 0.251ns 0.97ns 0.00706 0 0 536 B
master StartFinishSpan net472 754ns 0.257ns 0.996ns 0.0853 0 0 538 B
master StartFinishScope net6.0 661ns 0.186ns 0.719ns 0.00901 0 0 656 B
master StartFinishScope netcoreapp3.1 921ns 1.08ns 4.2ns 0.00872 0 0 656 B
master StartFinishScope net472 965ns 0.737ns 2.85ns 0.098 0 0 618 B
#5006 StartFinishSpan net6.0 545ns 0.189ns 0.708ns 0.00753 0 0 536 B
#5006 StartFinishSpan netcoreapp3.1 687ns 0.18ns 0.672ns 0.00726 0 0 536 B
#5006 StartFinishSpan net472 749ns 0.309ns 1.2ns 0.0852 0 0 538 B
#5006 StartFinishScope net6.0 611ns 0.238ns 0.92ns 0.00925 0 0 656 B
#5006 StartFinishScope netcoreapp3.1 868ns 0.267ns 0.999ns 0.00887 0 0 656 B
#5006 StartFinishScope net472 1.04μs 0.827ns 3.2ns 0.0977 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 630ns 0.185ns 0.716ns 0.00918 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 919ns 0.4ns 1.5ns 0.00878 0 0 656 B
master RunOnMethodBegin net472 1.17μs 0.309ns 1.16ns 0.098 0 0 618 B
#5006 RunOnMethodBegin net6.0 578ns 0.138ns 0.516ns 0.00923 0 0 656 B
#5006 RunOnMethodBegin netcoreapp3.1 969ns 0.331ns 1.24ns 0.00877 0 0 656 B
#5006 RunOnMethodBegin net472 1.09μs 0.34ns 1.32ns 0.0979 0 0 618 B

andrewlock avatar Dec 27 '23 21:12 andrewlock