dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard
verified publisher icon DataDog

Activate runtime metrics by default on CI and use scoped tracer

Open anna-git opened this issue 7 months ago • 3 comments

Summary of changes

Activate runtime metrics by default on CI.

Reason for change

Implementation details

Test coverage

Change tests accordingly

Other details

anna-git avatar May 27 '25 16:05 anna-git

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7017) - mean (81ms)  : 77, 84
     .   : milestone, 81,
    master - mean (80ms)  : 76, 84
     .   : milestone, 80,

    section Baseline
    This PR (7017) - mean (76ms)  : 71, 81
     .   : milestone, 76,
    master - mean (76ms)  : 71, 82
     .   : milestone, 76,

    section CallTarget+Inlining+NGEN
    This PR (7017) - mean (1,078ms)  : 1051, 1106
     .   : milestone, 1078,
    master - mean (1,078ms)  : 1049, 1108
     .   : milestone, 1078,


gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7017) - mean (120ms)  : 115, 125
     .   : milestone, 120,
    master - mean (119ms)  : 115, 123
     .   : milestone, 119,

    section Baseline
    This PR (7017) - mean (119ms)  : 114, 124
     .   : milestone, 119,
    master - mean (118ms)  : 114, 122
     .   : milestone, 118,

    section CallTarget+Inlining+NGEN
    This PR (7017) - mean (753ms)  : 732, 775
     .   : milestone, 753,
    master - mean (753ms)  : 731, 775
     .   : milestone, 753,


gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7017) - mean (107ms)  : 102, 111
     .   : milestone, 107,
    master - mean (107ms)  : 102, 112
     .   : milestone, 107,

    section Baseline
    This PR (7017) - mean (106ms)  : 100, 111
     .   : milestone, 106,
    master - mean (105ms)  : 100, 111
     .   : milestone, 105,

    section CallTarget+Inlining+NGEN
    This PR (7017) - mean (711ms)  : 684, 737
     .   : milestone, 711,
    master - mean (719ms)  : 690, 748
     .   : milestone, 719,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7017) - mean (217ms)  : 211, 223
     .   : milestone, 217,
    master - mean (217ms)  : 209, 224
     .   : milestone, 217,

    section Baseline
    This PR (7017) - mean (214ms)  : 206, 223
     .   : milestone, 214,
    master - mean (212ms)  : 203, 221
     .   : milestone, 212,

    section CallTarget+Inlining+NGEN
    This PR (7017) - mean (1,215ms)  : 1179, 1252
     .   : milestone, 1215,
    master - mean (1,219ms)  : 1188, 1249
     .   : milestone, 1219,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7017) - mean (306ms)  : 299, 313
     .   : milestone, 306,
    master - mean (308ms)  : 299, 316
     .   : milestone, 308,

    section Baseline
    This PR (7017) - mean (303ms)  : 294, 313
     .   : milestone, 303,
    master - mean (308ms)  : 299, 316
     .   : milestone, 308,

    section CallTarget+Inlining+NGEN
    This PR (7017) - mean (976ms)  : 939, 1013
     .   : milestone, 976,
    master - mean (980ms)  : 951, 1009
     .   : milestone, 980,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7017) - mean (300ms)  : 292, 309
     .   : milestone, 300,
    master - mean (301ms)  : 290, 311
     .   : milestone, 301,

    section Baseline
    This PR (7017) - mean (299ms)  : 291, 307
     .   : milestone, 299,
    master - mean (299ms)  : 290, 307
     .   : milestone, 299,

    section CallTarget+Inlining+NGEN
    This PR (7017) - mean (964ms)  : 931, 998
     .   : milestone, 964,
    master - mean (970ms)  : 939, 1002
     .   : milestone, 970,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7017) - mean (298ms)  : 286, 310
     .   : milestone, 298,
    master - mean (300ms)  : 292, 308
     .   : milestone, 300,

    section Baseline
    This PR (7017) - mean (297ms)  : 288, 306
     .   : milestone, 297,
    master - mean (298ms)  : 287, 308
     .   : milestone, 298,

    section CallTarget+Inlining+NGEN
    This PR (7017) - mean (870ms)  : 815, 924
     .   : milestone, 870,
    master - mean (872ms)  : 812, 932
     .   : milestone, 872,

andrewlock avatar May 27 '25 17:05 andrewlock

Datadog Report

All test runs 985d526 :link:

:white_check_mark: 2 Total Test Services: 0 Failed, 2 Passed

Test Services
Service Name Failed Known Flaky New Flaky Passed Skipped Total Time Test Service View
dd-trace-dotnet 0 0 0 255940 2896 20h 57m 13.33s Link
exploration_tests 0 0 0 22085 3 5m 4.23s Link

datadog-datadog-prod-us1[bot] avatar May 27 '25 17:05 datadog-datadog-prod-us1[bot]

Benchmarks

Benchmarks Report for benchmark platform :snail:

Benchmarks for #7017 compared to master:

  • 7 benchmarks are slower, with geometric mean 1.429
  • 41 benchmarks have fewer allocations
  • 4 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1 5.75 KB 5.71 KB -48 B -0.83%
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net6.0 5.58 KB 5.49 KB -85 B -1.52%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 11.2μs 59.7ns 322ns 0 0 0 5.58 KB
master StartStopWithChild netcoreapp3.1 14.1μs 67.5ns 286ns 0 0 0 5.75 KB
master StartStopWithChild net472 22.1μs 119ns 651ns 1.03 0.411 0.103 6.09 KB
#7017 StartStopWithChild net6.0 10.5μs 58.7ns 385ns 0 0 0 5.49 KB
#7017 StartStopWithChild netcoreapp3.1 13.7μs 70.7ns 339ns 0 0 0 5.71 KB
#7017 StartStopWithChild net472 22μs 106ns 423ns 0.963 0.214 0 6.06 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 3.33 KB 3.31 KB -23 B -0.69%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 927μs 26.6ns 103ns 0 0 0 2.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.04ms 404ns 1.56μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.25ms 357ns 1.38μs 0 0 0 3.33 KB
#7017 WriteAndFlushEnrichedTraces net6.0 921μs 103ns 370ns 0 0 0 2.71 KB
#7017 WriteAndFlushEnrichedTraces netcoreapp3.1 1.02ms 326ns 1.26μs 0 0 0 2.7 KB
#7017 WriteAndFlushEnrichedTraces net472 1.19ms 26.9ns 97.1ns 0 0 0 3.31 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed :heavy_check_mark: More allocations :warning:

More allocations :warning: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net472 236.35 KB 239.66 KB 3.3 KB 1.40%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net472 239.87 KB 243.17 KB 3.3 KB 1.38%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 331μs 1.76μs 8.42μs 0 0 0 197.06 KB
master AllCycleSimpleBody netcoreapp3.1 510μs 1.45μs 5.61μs 0 0 0 204.77 KB
master AllCycleSimpleBody net472 436μs 119ns 460ns 36.6 2.16 0 236.35 KB
master AllCycleMoreComplexBody net6.0 338μs 1.76μs 8.8μs 0 0 0 200.56 KB
master AllCycleMoreComplexBody netcoreapp3.1 495μs 987ns 3.56μs 0 0 0 208.18 KB
master AllCycleMoreComplexBody net472 446μs 106ns 412ns 36.6 2.16 0 239.87 KB
master ObjectExtractorSimpleBody net6.0 311ns 1.77ns 12.3ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 409ns 1.93ns 8.2ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 303ns 0.175ns 0.676ns 0.0442 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.52μs 29.5ns 110ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.76μs 36.2ns 140ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.66μs 0.89ns 3.33ns 0.599 0 0 3.8 KB
#7017 AllCycleSimpleBody net6.0 332μs 1.67μs 7.85μs 0 0 0 197.61 KB
#7017 AllCycleSimpleBody netcoreapp3.1 508μs 185ns 715ns 0 0 0 205.37 KB
#7017 AllCycleSimpleBody net472 439μs 94.5ns 366ns 36.6 2.16 0 239.66 KB
#7017 AllCycleMoreComplexBody net6.0 341μs 215ns 834ns 0 0 0 201.12 KB
#7017 AllCycleMoreComplexBody netcoreapp3.1 507μs 371ns 1.44μs 0 0 0 208.78 KB
#7017 AllCycleMoreComplexBody net472 449μs 82.8ns 310ns 37.9 2.23 0 243.17 KB
#7017 ObjectExtractorSimpleBody net6.0 317ns 0.166ns 0.641ns 0 0 0 280 B
#7017 ObjectExtractorSimpleBody netcoreapp3.1 427ns 2.24ns 10.8ns 0 0 0 272 B
#7017 ObjectExtractorSimpleBody net472 306ns 0.0343ns 0.133ns 0.0432 0 0 281 B
#7017 ObjectExtractorMoreComplexBody net6.0 6.38μs 4.28ns 16.6ns 0 0 0 3.78 KB
#7017 ObjectExtractorMoreComplexBody netcoreapp3.1 8.02μs 40ns 174ns 0 0 0 3.69 KB
#7017 ObjectExtractorMoreComplexBody net472 6.69μs 3.46ns 13.4ns 0.588 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs‑net6.0 2.16 KB 2.15 KB -11 B -0.51%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 73.3μs 263ns 984ns 0 0 0 32.41 KB
master EncodeArgs netcoreapp3.1 95.7μs 36.2ns 130ns 0 0 0 32.4 KB
master EncodeArgs net472 107μs 17.6ns 65.8ns 4.82 0 0 32.51 KB
master EncodeLegacyArgs net6.0 143μs 122ns 472ns 0 0 0 2.16 KB
master EncodeLegacyArgs netcoreapp3.1 197μs 42.5ns 147ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 261μs 66.8ns 250ns 0 0 0 2.16 KB
#7017 EncodeArgs net6.0 77.1μs 246ns 954ns 0 0 0 32.4 KB
#7017 EncodeArgs netcoreapp3.1 95.3μs 345ns 1.34μs 0 0 0 32.4 KB
#7017 EncodeArgs net472 109μs 28.6ns 111ns 4.93 0 0 32.5 KB
#7017 EncodeLegacyArgs net6.0 140μs 53.1ns 206ns 0 0 0 2.15 KB
#7017 EncodeLegacyArgs netcoreapp3.1 198μs 516ns 2μs 0 0 0 2.14 KB
#7017 EncodeLegacyArgs net472 262μs 24.8ns 96ns 0 0 0 2.16 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #7017

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑net6.0 1.564 181,119.78 283,329.55
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑net472 1.514 206,733.04 312,964.06
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑netcoreapp3.1 1.509 198,578.46 299,576.26
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark‑net6.0 1.442 271,505.69 391,422.18
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark‑net472 1.398 306,857.62 428,977.29
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark‑netcoreapp3.1 1.397 293,157.49 409,523.54

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 271μs 133ns 499ns 0 0 0 4.55 KB
master RunWafRealisticBenchmark netcoreapp3.1 294μs 264ns 989ns 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 307μs 38.1ns 147ns 0 0 0 4.66 KB
master RunWafRealisticBenchmarkWithAttack net6.0 181μs 80.1ns 300ns 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 198μs 104ns 404ns 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 207μs 45.3ns 169ns 0 0 0 2.28 KB
#7017 RunWafRealisticBenchmark net6.0 391μs 37.1ns 139ns 0 0 0 4.55 KB
#7017 RunWafRealisticBenchmark netcoreapp3.1 410μs 363ns 1.36μs 0 0 0 4.48 KB
#7017 RunWafRealisticBenchmark net472 429μs 28.2ns 109ns 0 0 0 4.66 KB
#7017 RunWafRealisticBenchmarkWithAttack net6.0 283μs 34.2ns 128ns 0 0 0 2.24 KB
#7017 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 300μs 30.8ns 115ns 0 0 0 2.22 KB
#7017 RunWafRealisticBenchmarkWithAttack net472 313μs 23.1ns 89.7ns 0 0 0 2.29 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 61.1μs 29ns 104ns 0 0 0 14.53 KB
master SendRequest netcoreapp3.1 69.9μs 118ns 440ns 0 0 0 17.42 KB
master SendRequest net472 0.0171ns 0.00147ns 0.00551ns 0 0 0 0 b
#7017 SendRequest net6.0 61.7μs 35.5ns 128ns 0 0 0 14.52 KB
#7017 SendRequest netcoreapp3.1 71.2μs 108ns 420ns 0 0 0 17.42 KB
#7017 SendRequest net472 0.0118ns 0.00177ns 0.00687ns 0 0 0 0 b
Benchmarks.Trace.CharSliceBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
#7017 OriginalCharSlice net6.0 1.89ms 4.79μs 17.9μs 0 0 0 640.01 KB
#7017 OriginalCharSlice netcoreapp3.1 2.15ms 10.2μs 41.9μs 0 0 0 640 KB
#7017 OriginalCharSlice net472 2.72ms 168ns 627ns 100 0 0 641.95 KB
#7017 OptimizedCharSlice net6.0 1.43ms 133ns 515ns 0 0 0 4 B
#7017 OptimizedCharSlice netcoreapp3.1 1.68ms 148ns 555ns 0 0 0 1 B
#7017 OptimizedCharSlice net472 2.02ms 1.3μs 5.03μs 0 0 0 73 B
#7017 OptimizedCharSliceWithPool net6.0 830μs 152ns 591ns 0 0 0 3 B
#7017 OptimizedCharSliceWithPool netcoreapp3.1 880μs 64.5ns 250ns 0 0 0 0 b
#7017 OptimizedCharSliceWithPool net472 1.14ms 30.7ns 111ns 0 0 0 47 B
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 645μs 512ns 1.98μs 0 0 0 41.73 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 631μs 1.67μs 6.67μs 0 0 0 41.89 KB
master WriteAndFlushEnrichedTraces net472 923μs 1.93μs 7.2μs 4.46 0 0 55.75 KB
#7017 WriteAndFlushEnrichedTraces net6.0 650μs 902ns 3.49μs 0 0 0 41.73 KB
#7017 WriteAndFlushEnrichedTraces netcoreapp3.1 661μs 2.16μs 8.36μs 0 0 0 41.98 KB
#7017 WriteAndFlushEnrichedTraces net472 880μs 2.18μs 8.45μs 8.33 0 0 55.61 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net6.0 1.03 KB 1.02 KB -8 B -0.78%
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑netcoreapp3.1 1.02 KB 1.02 KB -8 B -0.78%
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net472 995 B 987 B -8 B -0.80%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.99μs 10.2ns 46.8ns 0 0 0 1.03 KB
master ExecuteNonQuery netcoreapp3.1 2.53μs 3.43ns 13.3ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.7μs 2.41ns 9.34ns 0.147 0.0134 0 995 B
#7017 ExecuteNonQuery net6.0 1.94μs 8.69ns 33.6ns 0 0 0 1.02 KB
#7017 ExecuteNonQuery netcoreapp3.1 2.56μs 6ns 23.2ns 0 0 0 1.02 KB
#7017 ExecuteNonQuery net472 2.76μs 3.28ns 12.7ns 0.154 0.014 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net472 1.11 KB 1.1 KB -8 B -0.72%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑netcoreapp3.1 1.09 KB 1.08 KB -8 B -0.74%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net472 1.05 KB 1.04 KB -8 B -0.76%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0 1.04 KB 1.03 KB -8 B -0.77%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑netcoreapp3.1 1.04 KB 1.03 KB -8 B -0.77%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net6.0 1.02 KB 1.01 KB -8 B -0.79%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.83μs 5.85ns 22.7ns 0 0 0 1.04 KB
master CallElasticsearch netcoreapp3.1 2.3μs 11.8ns 56.6ns 0 0 0 1.04 KB
master CallElasticsearch net472 3.55μs 3.05ns 11.8ns 0.159 0 0 1.05 KB
master CallElasticsearchAsync net6.0 1.81μs 3.89ns 14.5ns 0 0 0 1.02 KB
master CallElasticsearchAsync netcoreapp3.1 2.35μs 7.26ns 28.1ns 0 0 0 1.09 KB
master CallElasticsearchAsync net472 3.84μs 3.19ns 12.4ns 0.169 0 0 1.11 KB
#7017 CallElasticsearch net6.0 1.76μs 9.19ns 44.1ns 0 0 0 1.03 KB
#7017 CallElasticsearch netcoreapp3.1 2.36μs 0.779ns 2.81ns 0 0 0 1.03 KB
#7017 CallElasticsearch net472 3.51μs 4.06ns 15.2ns 0.158 0 0 1.04 KB
#7017 CallElasticsearchAsync net6.0 1.79μs 8.11ns 30.3ns 0 0 0 1.01 KB
#7017 CallElasticsearchAsync netcoreapp3.1 2.41μs 11.2ns 46.2ns 0 0 0 1.08 KB
#7017 CallElasticsearchAsync net472 3.67μs 1.92ns 7.44ns 0.165 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0 960 B 952 B -8 B -0.83%
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑netcoreapp3.1 960 B 952 B -8 B -0.83%
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net472 923 B 915 B -8 B -0.87%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.78μs 9.27ns 43.5ns 0 0 0 960 B
master ExecuteAsync netcoreapp3.1 2.29μs 7.45ns 25.8ns 0 0 0 960 B
master ExecuteAsync net472 2.58μs 1.78ns 6.88ns 0.143 0 0 923 B
#7017 ExecuteAsync net6.0 2μs 9.12ns 36.5ns 0 0 0 952 B
#7017 ExecuteAsync netcoreapp3.1 2.44μs 11.6ns 44.9ns 0 0 0 952 B
#7017 ExecuteAsync net472 2.74μs 1.68ns 6.52ns 0.137 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 7.1μs 10.4ns 39ns 0 0 0 2.37 KB
master SendAsync netcoreapp3.1 8.69μs 14.1ns 54.6ns 0 0 0 2.9 KB
master SendAsync net472 12.5μs 10.3ns 38.5ns 0.498 0 0 3.19 KB
#7017 SendAsync net6.0 6.65μs 3.63ns 12.6ns 0 0 0 2.36 KB
#7017 SendAsync netcoreapp3.1 8.48μs 28.4ns 110ns 0 0 0 2.9 KB
#7017 SendAsync net472 12.5μs 9.97ns 38.6ns 0.499 0 0 3.18 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower :warning: More allocations :warning:

Slower :warning: in #7017

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 1.208 413,700.00 499,900.00

More allocations :warning: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 57.34 KB 73.73 KB 16.38 KB 28.57%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 42.64 KB 44.02 KB 1.38 KB 3.23%

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 274.93 KB 272.19 KB -2.74 KB -1.00%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 44.6μs 212ns 1.6μs 0 0 0 43.83 KB
master StringConcatBenchmark netcoreapp3.1 47.2μs 224ns 838ns 0 0 0 42.64 KB
master StringConcatBenchmark net472 56.8μs 259ns 968ns 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 458μs 1.08μs 3.89μs 0 0 0 259.96 KB
master StringConcatAspectBenchmark netcoreapp3.1 447μs 6.44μs 63.7μs 0 0 0 274.93 KB
master StringConcatAspectBenchmark net472 410μs 2.07μs 9.27μs 0 0 0 286.72 KB
#7017 StringConcatBenchmark net6.0 45.7μs 280ns 2.68μs 0 0 0 43.86 KB
#7017 StringConcatBenchmark netcoreapp3.1 46.3μs 228ns 992ns 0 0 0 44.02 KB
#7017 StringConcatBenchmark net472 57.3μs 172ns 642ns 0 0 0 73.73 KB
#7017 StringConcatAspectBenchmark net6.0 480μs 1.8μs 6.49μs 0 0 0 259.38 KB
#7017 StringConcatAspectBenchmark netcoreapp3.1 502μs 2.37μs 9.18μs 0 0 0 272.19 KB
#7017 StringConcatAspectBenchmark net472 404μs 1.99μs 8.69μs 0 0 0 286.72 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net6.0 1.76 KB 1.7 KB -56 B -3.18%
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1 1.76 KB 1.7 KB -56 B -3.18%
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net472 1.69 KB 1.64 KB -56 B -3.31%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.63μs 2.6ns 10.1ns 0 0 0 1.76 KB
master EnrichedLog netcoreapp3.1 3.44μs 4.38ns 17ns 0 0 0 1.76 KB
master EnrichedLog net472 4.05μs 4.83ns 18.7ns 0.265 0 0 1.69 KB
#7017 EnrichedLog net6.0 2.65μs 3.72ns 14.4ns 0 0 0 1.7 KB
#7017 EnrichedLog netcoreapp3.1 3.43μs 16ns 61.8ns 0 0 0 1.7 KB
#7017 EnrichedLog net472 3.91μs 2.81ns 10.9ns 0.253 0 0 1.64 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑net6.0 4.37 KB 4.31 KB -56 B -1.28%
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑netcoreapp3.1 4.37 KB 4.31 KB -57 B -1.30%
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑net472 4.57 KB 4.51 KB -60 B -1.31%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 123μs 145ns 563ns 0 0 0 4.37 KB
master EnrichedLog netcoreapp3.1 126μs 330ns 1.24μs 0 0 0 4.37 KB
master EnrichedLog net472 167μs 161ns 603ns 0 0 0 4.57 KB
#7017 EnrichedLog net6.0 122μs 63ns 218ns 0 0 0 4.31 KB
#7017 EnrichedLog netcoreapp3.1 126μs 65ns 252ns 0 0 0 4.31 KB
#7017 EnrichedLog net472 166μs 135ns 521ns 0 0 0 4.51 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑net6.0 2.32 KB 2.26 KB -56 B -2.41%
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑netcoreapp3.1 2.32 KB 2.26 KB -56 B -2.41%
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑net472 2.14 KB 2.08 KB -56 B -2.62%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.94μs 19.3ns 74.6ns 0 0 0 2.32 KB
master EnrichedLog netcoreapp3.1 6.77μs 22.2ns 79.9ns 0 0 0 2.32 KB
master EnrichedLog net472 7.45μs 8.04ns 31.1ns 0.335 0 0 2.14 KB
#7017 EnrichedLog net6.0 4.97μs 9.43ns 36.5ns 0 0 0 2.26 KB
#7017 EnrichedLog netcoreapp3.1 7μs 30.5ns 114ns 0 0 0 2.26 KB
#7017 EnrichedLog net472 7.45μs 7.86ns 30.4ns 0.297 0 0 2.08 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.RedisBenchmark.SendReceive‑net472 1.21 KB 1.2 KB -8 B -0.66%
Benchmarks.Trace.RedisBenchmark.SendReceive‑net6.0 1.21 KB 1.2 KB -8 B -0.66%
Benchmarks.Trace.RedisBenchmark.SendReceive‑netcoreapp3.1 1.21 KB 1.2 KB -8 B -0.66%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2.04μs 10.5ns 49.4ns 0 0 0 1.21 KB
master SendReceive netcoreapp3.1 2.53μs 11.8ns 47.4ns 0 0 0 1.21 KB
master SendReceive net472 3.28μs 2.71ns 10.5ns 0.178 0 0 1.21 KB
#7017 SendReceive net6.0 2.03μs 10.4ns 49.9ns 0 0 0 1.2 KB
#7017 SendReceive netcoreapp3.1 2.59μs 13.3ns 62.5ns 0 0 0 1.2 KB
#7017 SendReceive net472 3.19μs 5.3ns 20.5ns 0.189 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑net472 2.08 KB 2.03 KB -56 B -2.69%
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑netcoreapp3.1 1.69 KB 1.63 KB -56 B -3.32%
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑net6.0 1.64 KB 1.58 KB -56 B -3.41%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.18μs 0.92ns 3.44ns 0 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 5.64μs 23ns 89.3ns 0 0 0 1.69 KB
master EnrichedLog net472 6.67μs 7.28ns 27.3ns 0.298 0 0 2.08 KB
#7017 EnrichedLog net6.0 4.28μs 4.89ns 18.9ns 0 0 0 1.58 KB
#7017 EnrichedLog netcoreapp3.1 5.57μs 10.1ns 39ns 0 0 0 1.63 KB
#7017 EnrichedLog net472 6.53μs 7.85ns 30.4ns 0.292 0 0 2.03 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 704 B 696 B -8 B -1.14%
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 704 B 696 B -8 B -1.14%
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 666 B 658 B -8 B -1.20%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472 586 B 578 B -8 B -1.37%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 584 B 576 B -8 B -1.37%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 584 B 576 B -8 B -1.37%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 751ns 3.97ns 22.4ns 0 0 0 584 B
master StartFinishSpan netcoreapp3.1 953ns 4.42ns 17.7ns 0 0 0 584 B
master StartFinishSpan net472 914ns 0.809ns 3.13ns 0.0912 0 0 586 B
master StartFinishScope net6.0 918ns 0.479ns 1.73ns 0 0 0 704 B
master StartFinishScope netcoreapp3.1 1.15μs 6.24ns 34.2ns 0 0 0 704 B
master StartFinishScope net472 1.09μs 0.174ns 0.652ns 0.104 0 0 666 B
#7017 StartFinishSpan net6.0 763ns 3.7ns 14.8ns 0 0 0 576 B
#7017 StartFinishSpan netcoreapp3.1 943ns 0.341ns 1.23ns 0 0 0 576 B
#7017 StartFinishSpan net472 898ns 0.622ns 2.41ns 0.09 0 0 578 B
#7017 StartFinishScope net6.0 886ns 4.75ns 25.1ns 0 0 0 696 B
#7017 StartFinishScope netcoreapp3.1 1.18μs 1.89ns 7.09ns 0 0 0 696 B
#7017 StartFinishScope net472 1.13μs 0.367ns 1.42ns 0.101 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7017

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 704 B 696 B -8 B -1.14%
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1 704 B 696 B -8 B -1.14%
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472 666 B 658 B -8 B -1.20%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.02μs 5.36ns 26.8ns 0 0 0 704 B
master RunOnMethodBegin netcoreapp3.1 1.38μs 2.28ns 8.82ns 0 0 0 704 B
master RunOnMethodBegin net472 1.36μs 0.141ns 0.544ns 0.102 0 0 666 B
#7017 RunOnMethodBegin net6.0 1.04μs 1.8ns 6.73ns 0 0 0 696 B
#7017 RunOnMethodBegin netcoreapp3.1 1.43μs 0.879ns 3.4ns 0 0 0 696 B
#7017 RunOnMethodBegin net472 1.39μs 1.22ns 4.71ns 0.104 0 0 658 B

pr-commenter[bot] avatar Jun 03 '25 09:06 pr-commenter[bot]

The big change in execution-time benchmarks is surprising, and I can't explain it 😕 There's nothing in this PR that should have impacted them right?

andrewlock avatar Jul 11 '25 12:07 andrewlock

The big change in execution-time benchmarks is surprising, and I can't explain it 😕 There's nothing in this PR that should have impacted them right?

I'm not sure why... I havent changed the StringAspects in this PR 😅

anna-git avatar Jul 11 '25 18:07 anna-git

I extracted all the scoped tracer changes into a separate PR, to make this one more focused, and rebased 🙂 https://github.com/DataDog/dd-trace-dotnet/pull/7220

andrewlock avatar Jul 15 '25 08:07 andrewlock