dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard
verified publisher icon DataDog

[Tracing] Enable nullable reference types for AWS integrations

Open zacharycmontoya opened this issue 2 years ago • 5 comments

Summary of changes

Enable nullable reference types for AWS integrations

Reason for change

To better prevent future occurrences of NullReferenceException

Implementation details

  • Adds #nullable enable to all of the files included in the AWS integrations
  • Adds some new null-checks
  • Changes the type of the exception argument in the OnAsyncMethodEnd and OnMethodEnd methods from Exception to Exception?
  • Makes all the properties in the Tags implementations nullable (except for constant values like SpanKind and InstrumentationName)
  • In our IL helper classes, asserts that reflection objects are non-null

Test coverage

No new tests, but the compiler complained about some problematic code paths that are now fixed.

Other details

This PR is currently based on https://github.com/DataDog/dd-trace-dotnet/pull/4954, but if we want to merge this separately, I can retarget it to main and remove the updates to the nullability csv from the PR.

zacharycmontoya avatar Dec 05 '23 21:12 zacharycmontoya

Datadog Report

Branch report: zach.montoya/rd-week/nullability-aws Commit report: e3fad3e

:white_check_mark: dd-trace-dotnet: 0 Failed, 0 New Flaky, 303027 Passed, 1025 Skipped, 30m 24.17s Wall Time

datadog-ddstaging[bot] avatar Dec 05 '23 21:12 datadog-ddstaging[bot]

Let me know if there's something where I should peek more exhaustively, as of now, it looks like this shouldn't affect anything we were doing before. I imagine this is becaues of .NET 8?

I added some PR comments to indicate specific code changes that you should be aware of. Everything else just makes sure that if the reference type could be a null value, it now has a nullable type (e.g. Datadog.Trace.Scope => Datadog.Trace.Scope?)

zacharycmontoya avatar Dec 05 '23 22:12 zacharycmontoya

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4957) - mean (72ms)  : 63, 80
     .   : milestone, 72,
    master - mean (72ms)  : 62, 82
     .   : milestone, 72,

    section CallTarget+Inlining+NGEN
    This PR (4957) - mean (998ms)  : 978, 1018
     .   : milestone, 998,
    master - mean (1,000ms)  : 981, 1019
     .   : milestone, 1000,


gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4957) - mean (106ms)  : 103, 108
     .   : milestone, 106,
    master - mean (107ms)  : 103, 111
     .   : milestone, 107,

    section CallTarget+Inlining+NGEN
    This PR (4957) - mean (687ms)  : 670, 703
     .   : milestone, 687,
    master - mean (693ms)  : 671, 715
     .   : milestone, 693,


gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4957) - mean (90ms)  : 88, 92
     .   : milestone, 90,
    master - mean (90ms)  : 86, 93
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (4957) - mean (663ms)  : 638, 688
     .   : milestone, 663,
    master - mean (663ms)  : 640, 687
     .   : milestone, 663,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4957) - mean (190ms)  : 186, 194
     .   : milestone, 190,
    master - mean (189ms)  : 187, 192
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (4957) - mean (1,140ms)  : 1116, 1165
     .   : milestone, 1140,
    master - mean (1,137ms)  : 1113, 1161
     .   : milestone, 1137,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4957) - mean (274ms)  : 270, 277
     .   : milestone, 274,
    master - mean (273ms)  : 269, 278
     .   : milestone, 273,

    section CallTarget+Inlining+NGEN
    This PR (4957) - mean (1,100ms)  : 1077, 1122
     .   : milestone, 1100,
    master - mean (1,093ms)  : 1065, 1121
     .   : milestone, 1093,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4957) - mean (264ms)  : 259, 268
     .   : milestone, 264,
    master - mean (262ms)  : 260, 265
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (4957) - mean (1,061ms)  : 1041, 1082
     .   : milestone, 1061,
    master - mean (1,059ms)  : 1032, 1087
     .   : milestone, 1059,

andrewlock avatar Dec 05 '23 22:12 andrewlock

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4957) (11.211M)   : 0, 11210660
    master (11.040M)   : 0, 11039705
    benchmarks/2.9.0 (11.109M)   : 0, 11108980

    section Automatic
    This PR (4957) (7.710M)   : 0, 7710134
    master (7.552M)   : 0, 7551623
    benchmarks/2.9.0 (8.083M)   : 0, 8082605

    section Trace stats
    This PR (4957) (8.011M)   : 0, 8011236
    master (7.982M)   : 0, 7981851

    section Manual
    This PR (4957) (9.797M)   : 0, 9796546
    master (9.754M)   : 0, 9753918

    section Manual + Automatic
    This PR (4957) (7.189M)   : 0, 7188771
    master (7.350M)   : 0, 7350127

    section Version Conflict
    This PR (4957) (6.586M)   : 0, 6586286
    master (6.701M)   : 0, 6700963


gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4957) (9.483M)   : 0, 9482527
    master (9.599M)   : 0, 9598918
    benchmarks/2.9.0 (9.303M)   : 0, 9302781

    section Automatic
    This PR (4957) (6.415M)   : 0, 6414723
    master (6.506M)   : 0, 6506088

    section Trace stats
    This PR (4957) (6.852M)   : 0, 6851857
    master (6.900M)   : 0, 6899626

    section Manual
    This PR (4957) (8.376M)   : 0, 8375728
    master (8.261M)   : 0, 8261055

    section Manual + Automatic
    This PR (4957) (6.127M)   : 0, 6126867
    master (6.225M)   : 0, 6224665

    section Version Conflict
    This PR (4957) (5.681M)   : 0, 5680737
    master (5.781M)   : 0, 5780607


gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4957) (10.363M)   : 0, 10363161
    master (10.778M)   : 0, 10777926
    benchmarks/2.9.0 (11.129M)   : 0, 11128978

    section Automatic
    This PR (4957) (7.311M)   : 0, 7311192
    master (7.531M)   : 0, 7530980
    benchmarks/2.9.0 (8.109M)   : 0, 8108685

    section Trace stats
    This PR (4957) (7.738M)   : 0, 7738405
    master (7.932M)   : 0, 7932341

    section Manual
    This PR (4957) (8.970M)   : crit ,0, 8970024
    master (9.565M)   : 0, 9564575

    section Manual + Automatic
    This PR (4957) (7.029M)   : 0, 7028950
    master (7.252M)   : 0, 7251513

    section Version Conflict
    This PR (4957) (6.522M)   : 0, 6521972
    master (6.448M)   : 0, 6448030


gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    master (7.321M)   : 0, 7320808
    benchmarks/2.9.0 (7.724M)   : 0, 7724099

    section No attack
    master (2.003M)   : 0, 2002864
    benchmarks/2.9.0 (3.167M)   : 0, 3166743

    section Attack
    master (1.572M)   : 0, 1571902
    benchmarks/2.9.0 (2.437M)   : 0, 2436746

    section Blocking
    master (3.252M)   : 0, 3252414

    section IAST default
    master (6.482M)   : 0, 6481864

    section IAST full
    master (5.631M)   : 0, 5630684

    section Base vuln
    master (0.960M)   : 0, 960128

    section IAST vuln
    master (0.878M)   : 0, 877982

andrewlock avatar Dec 05 '23 22:12 andrewlock

Benchmarks Report :snail:

Benchmarks for #4957 compared to master:

  • 2 benchmarks are faster, with geometric mean 1.143
  • 2 benchmarks are slower, with geometric mean 1.149
  • 1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.78μs 48.3ns 282ns 0.0214 0.00856 0 7.48 KB
master StartStopWithChild netcoreapp3.1 10.9μs 56.9ns 273ns 0.0215 0.00538 0 7.58 KB
master StartStopWithChild net472 17.1μs 64.6ns 250ns 1.35 0.349 0.111 7.95 KB
#4957 StartStopWithChild net6.0 8.62μs 45.9ns 229ns 0.0226 0.00905 0 7.48 KB
#4957 StartStopWithChild netcoreapp3.1 11μs 27.7ns 96ns 0.0266 0.0106 0 7.58 KB
#4957 StartStopWithChild net472 16.9μs 37.7ns 146ns 1.34 0.344 0.101 7.95 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 476μs 293ns 1.13μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 634μs 258ns 1μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 789μs 240ns 929ns 0.398 0 0 3.3 KB
#4957 WriteAndFlushEnrichedTraces net6.0 455μs 290ns 1.12μs 0 0 0 2.7 KB
#4957 WriteAndFlushEnrichedTraces netcoreapp3.1 623μs 161ns 602ns 0 0 0 2.7 KB
#4957 WriteAndFlushEnrichedTraces net472 782μs 422ns 1.63μs 0.388 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 43.9μs 35ns 136ns 0.0221 0 0 1.77 KB
master AllCycleSimpleBody netcoreapp3.1 46.8μs 38.2ns 138ns 0.0233 0 0 1.74 KB
master AllCycleSimpleBody net472 49.3μs 121ns 468ns 0.269 0 0 1.81 KB
master AllCycleMoreComplexBody net6.0 215μs 171ns 661ns 0.106 0 0 9.25 KB
master AllCycleMoreComplexBody netcoreapp3.1 226μs 289ns 1.26μs 0.113 0 0 9.14 KB
master AllCycleMoreComplexBody net472 237μs 94.1ns 365ns 1.42 0 0 9.32 KB
master ObjectExtractorSimpleBody net6.0 134ns 0.0414ns 0.155ns 0.00393 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 194ns 0.228ns 0.852ns 0.00373 0 0 272 B
master ObjectExtractorSimpleBody net472 153ns 0.0718ns 0.269ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 2.86μs 1.65ns 6.17ns 0.053 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.62μs 1.45ns 5.42ns 0.0506 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 3.51μs 2.78ns 10.8ns 0.602 0.00525 0 3.8 KB
#4957 AllCycleSimpleBody net6.0 43.9μs 31.1ns 116ns 0.0213 0 0 1.77 KB
#4957 AllCycleSimpleBody netcoreapp3.1 46μs 179ns 692ns 0 0 0 1.74 KB
#4957 AllCycleSimpleBody net472 48.9μs 13.5ns 50.6ns 0.288 0 0 1.81 KB
#4957 AllCycleMoreComplexBody net6.0 213μs 322ns 1.2μs 0.106 0 0 9.25 KB
#4957 AllCycleMoreComplexBody netcoreapp3.1 225μs 142ns 961ns 0.112 0 0 9.14 KB
#4957 AllCycleMoreComplexBody net472 238μs 267ns 1.03μs 1.43 0 0 9.32 KB
#4957 ObjectExtractorSimpleBody net6.0 128ns 0.0598ns 0.232ns 0.00395 0 0 280 B
#4957 ObjectExtractorSimpleBody netcoreapp3.1 188ns 0.107ns 0.414ns 0.0036 0 0 272 B
#4957 ObjectExtractorSimpleBody net472 151ns 0.0861ns 0.31ns 0.0446 0 0 281 B
#4957 ObjectExtractorMoreComplexBody net6.0 2.85μs 2.67ns 10ns 0.0527 0 0 3.78 KB
#4957 ObjectExtractorMoreComplexBody netcoreapp3.1 3.7μs 1.78ns 6.9ns 0.0501 0 0 3.69 KB
#4957 ObjectExtractorMoreComplexBody net472 3.46μs 2.1ns 8.15ns 0.601 0.00523 0 3.8 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 53.7μs 81.1ns 303ns 0.214 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 68.9μs 371ns 1.96μs 0.202 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) net472 93.8μs 53.6ns 208ns 2.53 0.0938 0 16.14 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 115μs 97ns 423ns 0.304 0 0 22.41 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 137μs 254ns 985ns 0.281 0 0 22.36 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 165μs 97.8ns 353ns 3.54 0.165 0 22.7 KB
master RunWaf(args=NestedMap (100)) net6.0 104μs 472ns 1.83μs 0.428 0 0 32.76 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 131μs 463ns 1.73μs 0.395 0 0 33.33 KB
master RunWaf(args=NestedMap (100)) net472 189μs 911ns 3.64μs 5.28 0.364 0 33.67 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 175μs 618ns 2.39μs 0.49 0 0 39.1 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 206μs 811ns 2.92μs 0.511 0 0 39.63 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 266μs 1.29μs 5.17μs 6.31 0.515 0 40.23 KB
master RunWaf(args=NestedMap (20)) net6.0 93.9μs 34.6ns 120ns 0.423 0 0 32.18 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 129μs 708ns 4.25μs 0.398 0 0 32.3 KB
master RunWaf(args=NestedMap (20)) net472 192μs 581ns 2.25μs 5.18 0.363 0 32.63 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 172μs 85.9ns 333ns 0.517 0 0 38.53 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 199μs 113ns 391ns 0.52 0 0 38.6 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 266μs 738ns 2.86μs 6.17 0.514 0 39.2 KB
#4957 RunWaf(args=NestedMap (10)) net6.0 48.5μs 9.29ns 34.7ns 0.219 0 0 16.06 KB
#4957 RunWaf(args=NestedMap (10)) netcoreapp3.1 67.7μs 318ns 1.23μs 0.202 0 0 16.06 KB
#4957 RunWaf(args=NestedMap (10)) net472 98.9μs 40.7ns 158ns 2.52 0.0934 0 16.14 KB
#4957 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 114μs 26.3ns 94.9ns 0.284 0 0 22.41 KB
#4957 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 138μs 563ns 2.58μs 0.28 0 0 22.36 KB
#4957 RunWafWithAttack(args=Neste(...)tack) [22]) net472 172μs 423ns 1.58μs 3.6 0.164 0 22.7 KB
#4957 RunWaf(args=NestedMap (100)) net6.0 103μs 50.3ns 174ns 0.463 0 0 32.76 KB
#4957 RunWaf(args=NestedMap (100)) netcoreapp3.1 126μs 295ns 1.02μs 0.446 0 0 33.33 KB
#4957 RunWaf(args=NestedMap (100)) net472 191μs 63ns 218ns 5.29 0.371 0 33.67 KB
#4957 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 169μs 89.4ns 346ns 0.484 0 0 39.1 KB
#4957 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 210μs 993ns 4.1μs 0.532 0 0 39.63 KB
#4957 RunWafWithAttack(args=Neste(...)tack) [23]) net472 261μs 272ns 980ns 6.34 0.518 0 40.23 KB
#4957 RunWaf(args=NestedMap (20)) net6.0 94.9μs 24.3ns 94.3ns 0.426 0 0 32.18 KB
#4957 RunWaf(args=NestedMap (20)) netcoreapp3.1 131μs 647ns 2.67μs 0.394 0 0 32.3 KB
#4957 RunWaf(args=NestedMap (20)) net472 182μs 183ns 708ns 5.19 0.364 0 32.63 KB
#4957 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 165μs 57.1ns 221ns 0.495 0 0 38.53 KB
#4957 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 209μs 986ns 3.94μs 0.495 0 0 38.6 KB
#4957 RunWafWithAttack(args=Neste(...)tack) [22]) net472 266μs 713ns 2.76μs 6.13 0.522 0 39.2 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 174μs 181ns 676ns 0.173 0 0 18.25 KB
master SendRequest netcoreapp3.1 195μs 404ns 1.56μs 0.195 0 0 20.41 KB
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#4957 SendRequest net6.0 172μs 145ns 544ns 0.173 0 0 18.25 KB
#4957 SendRequest netcoreapp3.1 191μs 400ns 1.55μs 0.192 0 0 20.41 KB
#4957 SendRequest net472 0.00104ns 0.000363ns 0.00141ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 547μs 828ns 3.21μs 0.553 0 0 41.41 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 651μs 530ns 2.05μs 0.324 0 0 41.75 KB
master WriteAndFlushEnrichedTraces net472 810μs 3.33μs 12.9μs 8.31 2.37 0.396 53.24 KB
#4957 WriteAndFlushEnrichedTraces net6.0 546μs 842ns 3.26μs 0.541 0 0 41.54 KB
#4957 WriteAndFlushEnrichedTraces netcoreapp3.1 663μs 789ns 3.06μs 0.322 0 0 41.75 KB
#4957 WriteAndFlushEnrichedTraces net472 825μs 3.23μs 12.5μs 8.17 2.45 0.408 53.22 KB
Benchmarks.Trace.DbCommandBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #4957

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑netcoreapp3.1 1.116 1,549.68 1,388.75

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.11μs 0.785ns 3.04ns 0.0105 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.55μs 0.632ns 2.45ns 0.0101 0 0 768 B
master ExecuteNonQuery net472 1.81μs 0.748ns 2.9ns 0.116 0 0 730 B
#4957 ExecuteNonQuery net6.0 1.04μs 0.295ns 1.11ns 0.0105 0 0 768 B
#4957 ExecuteNonQuery netcoreapp3.1 1.39μs 0.363ns 1.36ns 0.00993 0 0 768 B
#4957 ExecuteNonQuery net472 1.77μs 4.48ns 17.4ns 0.115 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.17μs 0.427ns 1.6ns 0.013 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.58μs 1.14ns 4.28ns 0.0124 0 0 936 B
master CallElasticsearch net472 2.51μs 1.07ns 4.16ns 0.152 0 0 955 B
master CallElasticsearchAsync net6.0 1.35μs 0.499ns 1.93ns 0.0128 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.67μs 2.04ns 7.9ns 0.0132 0 0 984 B
master CallElasticsearchAsync net472 2.58μs 0.779ns 3.02ns 0.16 0 0 1.01 KB
#4957 CallElasticsearch net6.0 1.15μs 0.329ns 1.28ns 0.0133 0 0 936 B
#4957 CallElasticsearch netcoreapp3.1 1.52μs 1.83ns 7.1ns 0.0122 0 0 936 B
#4957 CallElasticsearch net472 2.33μs 0.562ns 2.18ns 0.152 0.00117 0 955 B
#4957 CallElasticsearchAsync net6.0 1.26μs 0.536ns 2.01ns 0.0126 0 0 912 B
#4957 CallElasticsearchAsync netcoreapp3.1 1.62μs 2.83ns 11ns 0.0128 0 0 984 B
#4957 CallElasticsearchAsync net472 2.55μs 0.822ns 3.07ns 0.16 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.36μs 1.4ns 5.42ns 0.0128 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.67μs 0.716ns 2.58ns 0.0126 0 0 912 B
master ExecuteAsync net472 1.86μs 0.736ns 2.76ns 0.139 0 0 875 B
#4957 ExecuteAsync net6.0 1.29μs 0.591ns 2.21ns 0.013 0 0 912 B
#4957 ExecuteAsync netcoreapp3.1 1.6μs 1.19ns 4.6ns 0.012 0 0 912 B
#4957 ExecuteAsync net472 1.85μs 0.728ns 2.72ns 0.139 0 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.12μs 1.98ns 7.4ns 0.0288 0 0 2.1 KB
master SendAsync netcoreapp3.1 4.9μs 3.35ns 13ns 0.0344 0 0 2.63 KB
master SendAsync net472 7.75μs 4.8ns 18.6ns 0.523 0 0 3.31 KB
#4957 SendAsync net6.0 4.05μs 2.53ns 9.8ns 0.0284 0 0 2.1 KB
#4957 SendAsync netcoreapp3.1 4.79μs 1.34ns 5.2ns 0.0361 0 0 2.63 KB
#4957 SendAsync net472 7.69μs 4.01ns 14.5ns 0.525 0 0 3.31 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Faster :tada: Fewer allocations :tada:

Faster :tada: in #4957

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net6.0 1.170 60,600.00 51,800.00

Fewer allocations :tada: in #4957

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472 60.02 KB 59.66 KB -352 B -0.59%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 59μs 485ns 4.58μs 0 0 0 43.44 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 52.8μs 258ns 1.1μs 0 0 0 42.64 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 38.4μs 65.7ns 228ns 0 0 0 60.02 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 66.8μs 263ns 983ns 0 0 0 43.29 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 95.6μs 3.94μs 39.2μs 0 0 0 42.64 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 63.7μs 316ns 1.45μs 0 0 0 57.34 KB
#4957 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 51.8μs 272ns 1.41μs 0 0 0 43.44 KB
#4957 StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 53.8μs 253ns 981ns 0 0 0 42.64 KB
#4957 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 37.9μs 144ns 540ns 0 0 0 59.66 KB
#4957 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 86.4μs 3.16μs 31.5μs 0 0 0 43.29 KB
#4957 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 97.9μs 3.83μs 37.9μs 0 0 0 42.64 KB
#4957 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 63.5μs 330ns 1.58μs 0 0 0 57.34 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.63μs 0.827ns 2.98ns 0.022 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 2.07μs 0.954ns 3.57ns 0.0208 0 0 1.57 KB
master EnrichedLog net472 2.75μs 2.92ns 11.3ns 0.237 0 0 1.5 KB
#4957 EnrichedLog net6.0 1.51μs 0.564ns 2.11ns 0.022 0 0 1.57 KB
#4957 EnrichedLog netcoreapp3.1 2.27μs 0.782ns 2.93ns 0.0216 0 0 1.57 KB
#4957 EnrichedLog net472 2.52μs 2.7ns 10.1ns 0.237 0 0 1.5 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 116μs 133ns 514ns 0.0583 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 119μs 149ns 577ns 0.0591 0 0 4.21 KB
master EnrichedLog net472 146μs 49.3ns 171ns 0.658 0.219 0 4.39 KB
#4957 EnrichedLog net6.0 113μs 208ns 807ns 0.0562 0 0 4.21 KB
#4957 EnrichedLog netcoreapp3.1 119μs 185ns 717ns 0 0 0 4.21 KB
#4957 EnrichedLog net472 150μs 237ns 917ns 0.667 0.222 0 4.39 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.99μs 1.44ns 5.57ns 0.0299 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 4.34μs 1.48ns 5.73ns 0.0282 0 0 2.13 KB
master EnrichedLog net472 4.94μs 3.34ns 12.9ns 0.307 0 0 1.95 KB
#4957 EnrichedLog net6.0 3.11μs 1.3ns 4.68ns 0.0298 0 0 2.13 KB
#4957 EnrichedLog netcoreapp3.1 4.2μs 1.03ns 4.01ns 0.0294 0 0 2.13 KB
#4957 EnrichedLog net472 4.8μs 5.38ns 20.8ns 0.309 0 0 1.95 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.27μs 0.735ns 2.85ns 0.0151 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.77μs 3.93ns 15.2ns 0.0147 0 0 1.1 KB
master SendReceive net472 2.14μs 1.29ns 4.82ns 0.177 0 0 1.12 KB
#4957 SendReceive net6.0 1.26μs 0.881ns 3.41ns 0.0157 0 0 1.1 KB
#4957 SendReceive netcoreapp3.1 1.84μs 0.881ns 3.3ns 0.0147 0 0 1.1 KB
#4957 SendReceive net472 2.07μs 1.34ns 5.18ns 0.177 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.76μs 1.15ns 4.3ns 0.0207 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.99μs 0.937ns 3.63ns 0.02 0 0 1.58 KB
master EnrichedLog net472 4.28μs 1.71ns 6.64ns 0.311 0 0 1.97 KB
#4957 EnrichedLog net6.0 2.7μs 0.914ns 3.54ns 0.0216 0 0 1.53 KB
#4957 EnrichedLog netcoreapp3.1 3.93μs 1.65ns 6.18ns 0.0199 0 0 1.58 KB
#4957 EnrichedLog net472 4.3μs 1.23ns 4.77ns 0.311 0 0 1.97 KB
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #4957

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.175 514.49 604.77
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 1.123 450.42 505.98

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 450ns 0.129ns 0.483ns 0.00759 0 0 536 B
master StartFinishSpan netcoreapp3.1 781ns 1.64ns 6.37ns 0.0073 0 0 536 B
master StartFinishSpan net472 765ns 0.165ns 0.616ns 0.0853 0 0 538 B
master StartFinishScope net6.0 515ns 0.145ns 0.543ns 0.00908 0 0 656 B
master StartFinishScope netcoreapp3.1 818ns 0.436ns 1.57ns 0.00896 0 0 656 B
master StartFinishScope net472 928ns 0.258ns 1ns 0.0982 0 0 618 B
#4957 StartFinishSpan net6.0 506ns 0.231ns 0.833ns 0.00738 0 0 536 B
#4957 StartFinishSpan netcoreapp3.1 718ns 0.139ns 0.518ns 0.00703 0 0 536 B
#4957 StartFinishSpan net472 713ns 0.624ns 2.41ns 0.0851 0 0 538 B
#4957 StartFinishScope net6.0 605ns 0.348ns 1.35ns 0.00909 0 0 656 B
#4957 StartFinishScope netcoreapp3.1 847ns 0.376ns 1.45ns 0.00888 0 0 656 B
#4957 StartFinishScope net472 976ns 0.409ns 1.58ns 0.0978 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 590ns 0.645ns 2.5ns 0.00911 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 931ns 1.02ns 3.93ns 0.00873 0 0 656 B
master RunOnMethodBegin net472 1.08μs 0.321ns 1.24ns 0.0981 0 0 618 B
#4957 RunOnMethodBegin net6.0 592ns 0.442ns 1.71ns 0.00916 0 0 656 B
#4957 RunOnMethodBegin netcoreapp3.1 962ns 0.736ns 2.85ns 0.00868 0 0 656 B
#4957 RunOnMethodBegin net472 1.14μs 0.352ns 1.32ns 0.0978 0 0 618 B

andrewlock avatar Dec 05 '23 23:12 andrewlock

Closing the pull request since it's become stale. We can always pick this back up if desired

zacharycmontoya avatar May 09 '24 23:05 zacharycmontoya