dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard
verified publisher icon DataDog

Remove some allocations from DatadogHttpClient

Open tonyredondo opened this issue 1 year ago • 4 comments

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

tonyredondo avatar Jun 07 '24 12:06 tonyredondo

Datadog Report

Branch report: tony/datadoghttpclient-allocations Commit report: 81fc078 Test service: dd-trace-dotnet

:white_check_mark: 0 Failed, 337011 Passed, 1655 Skipped, 14h 8m 59.32s Total Time

datadog-ddstaging[bot] avatar Jun 07 '24 14:06 datadog-ddstaging[bot]

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5663) - mean (73ms)  : 65, 82
     .   : milestone, 73,
    master - mean (75ms)  : 61, 90
     .   : milestone, 75,

    section CallTarget+Inlining+NGEN
    This PR (5663) - mean (978ms)  : 960, 995
     .   : milestone, 978,
    master - mean (978ms)  : 947, 1010
     .   : milestone, 978,


gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5663) - mean (108ms)  : 106, 111
     .   : milestone, 108,
    master - mean (109ms)  : 107, 112
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (5663) - mean (689ms)  : 667, 712
     .   : milestone, 689,
    master - mean (688ms)  : 669, 707
     .   : milestone, 688,


gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5663) - mean (93ms)  : 90, 96
     .   : milestone, 93,
    master - mean (93ms)  : 90, 96
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (5663) - mean (643ms)  : 624, 661
     .   : milestone, 643,
    master - mean (643ms)  : 622, 663
     .   : milestone, 643,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5663) - mean (198ms)  : 191, 206
     .   : milestone, 198,
    master - mean (192ms)  : 188, 196
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (5663) - mean (1,102ms)  : 1078, 1125
     .   : milestone, 1102,
    master - mean (1,066ms)  : 1041, 1090
     .   : milestone, 1066,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5663) - mean (285ms)  : 279, 291
     .   : milestone, 285,
    master - mean (276ms)  : 272, 280
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (5663) - mean (868ms)  : 847, 890
     .   : milestone, 868,
    master - mean (864ms)  : 840, 888
     .   : milestone, 864,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5663) - mean (269ms)  : 264, 273
     .   : milestone, 269,
    master - mean (266ms)  : 262, 270
     .   : milestone, 266,

    section CallTarget+Inlining+NGEN
    This PR (5663) - mean (863ms)  : 830, 896
     .   : milestone, 863,
    master - mean (855ms)  : 828, 881
     .   : milestone, 855,

andrewlock avatar Jun 07 '24 14:06 andrewlock

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5663) (11.953M)   : 0, 11952710
    master (11.869M)   : 0, 11868538
    benchmarks/2.9.0 (11.919M)   : 0, 11919161

    section Automatic
    This PR (5663) (7.976M)   : 0, 7976434
    master (7.940M)   : 0, 7939651
    benchmarks/2.9.0 (8.337M)   : 0, 8336508

    section Trace stats
    master (8.273M)   : 0, 8273017

    section Manual
    This PR (5663) (10.191M)   : 0, 10191119
    master (10.092M)   : 0, 10092099

    section Manual + Automatic
    This PR (5663) (7.598M)   : 0, 7598204
    master (7.436M)   : 0, 7436191

    section Version Conflict
    master (6.662M)   : 0, 6662389


gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5663) (9.490M)   : 0, 9490088
    master (9.414M)   : 0, 9414089
    benchmarks/2.9.0 (9.646M)   : 0, 9645709

    section Automatic
    This PR (5663) (6.576M)   : 0, 6575892
    master (6.747M)   : 0, 6746671

    section Trace stats
    master (6.904M)   : 0, 6904262

    section Manual
    This PR (5663) (8.292M)   : 0, 8291547
    master (8.232M)   : 0, 8232186

    section Manual + Automatic
    This PR (5663) (6.113M)   : 0, 6113126
    master (6.103M)   : 0, 6102543

    section Version Conflict
    master (5.737M)   : 0, 5737013


gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5663) (9.988M)   : 0, 9988060
    master (9.973M)   : 0, 9973466
    benchmarks/2.9.0 (10.036M)   : 0, 10035695

    section Automatic
    This PR (5663) (7.127M)   : 0, 7126940
    master (7.080M)   : 0, 7080470
    benchmarks/2.9.0 (7.475M)   : 0, 7475419

    section Trace stats
    master (7.397M)   : 0, 7396581

    section Manual
    This PR (5663) (8.931M)   : 0, 8930812
    master (8.941M)   : 0, 8941201

    section Manual + Automatic
    This PR (5663) (6.863M)   : 0, 6863432
    master (6.906M)   : 0, 6905882

    section Version Conflict
    master (6.247M)   : 0, 6247204

andrewlock avatar Jun 07 '24 21:06 andrewlock

Benchmarks Report for tracer :snail:

Benchmarks for #5663 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.138
  • 2 benchmarks are slower, with geometric mean 1.131
  • 1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.83μs 48.9ns 294ns 0.0215 0.0086 0 7.68 KB
master StartStopWithChild netcoreapp3.1 10.9μs 57.6ns 294ns 0.0316 0.0158 0 7.77 KB
master StartStopWithChild net472 17.4μs 50.7ns 196ns 1.37 0.354 0.103 8.2 KB
#5663 StartStopWithChild net6.0 8.82μs 47.8ns 262ns 0.0269 0.0135 0 7.68 KB
#5663 StartStopWithChild netcoreapp3.1 10.9μs 59.5ns 326ns 0.0213 0.0106 0 7.77 KB
#5663 StartStopWithChild net472 17.4μs 51.2ns 198ns 1.39 0.354 0.124 8.2 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 488μs 105ns 392ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 609μs 141ns 526ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 847μs 249ns 899ns 0.422 0 0 3.3 KB
#5663 WriteAndFlushEnrichedTraces net6.0 486μs 154ns 577ns 0 0 0 2.7 KB
#5663 WriteAndFlushEnrichedTraces netcoreapp3.1 644μs 254ns 985ns 0 0 0 2.7 KB
#5663 WriteAndFlushEnrichedTraces net472 848μs 350ns 1.35μs 0.422 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 173μs 249ns 934ns 0.172 0 0 18.44 KB
master SendRequest netcoreapp3.1 195μs 404ns 1.57μs 0.193 0 0 20.6 KB
master SendRequest net472 0.000823ns 0.000359ns 0.00134ns 0 0 0 0 b
#5663 SendRequest net6.0 171μs 185ns 691ns 0.256 0 0 18.44 KB
#5663 SendRequest netcoreapp3.1 193μs 370ns 1.43μs 0.195 0 0 20.6 KB
#5663 SendRequest net472 0.00013ns 8.67E‑05ns 0.000336ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #5663

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.91 KB 41.52 KB -392 B -0.94%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 535μs 466ns 1.68μs 0.536 0 0 41.53 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 661μs 646ns 2.24μs 0.334 0 0 41.91 KB
master WriteAndFlushEnrichedTraces net472 827μs 3.01μs 11.7μs 8.17 2.45 0.408 53.23 KB
#5663 WriteAndFlushEnrichedTraces net6.0 537μs 299ns 1.04μs 0.546 0 0 41.61 KB
#5663 WriteAndFlushEnrichedTraces netcoreapp3.1 657μs 1.51μs 5.83μs 0.34 0 0 41.52 KB
#5663 WriteAndFlushEnrichedTraces net472 843μs 3.4μs 13.2μs 8.45 2.53 0.422 53.24 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.18μs 0.829ns 3.21ns 0.0112 0 0 808 B
master ExecuteNonQuery netcoreapp3.1 1.43μs 0.817ns 3.17ns 0.0109 0 0 808 B
master ExecuteNonQuery net472 1.83μs 1.78ns 6.88ns 0.122 0 0 770 B
#5663 ExecuteNonQuery net6.0 1.11μs 0.74ns 2.87ns 0.0112 0 0 808 B
#5663 ExecuteNonQuery netcoreapp3.1 1.45μs 0.775ns 2.9ns 0.0109 0 0 808 B
#5663 ExecuteNonQuery net472 1.75μs 1.74ns 6.75ns 0.122 0 0 770 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.24μs 3.18ns 11.9ns 0.0135 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.52μs 1.85ns 6.68ns 0.0129 0 0 976 B
master CallElasticsearch net472 2.55μs 1.61ns 6.02ns 0.158 0 0 995 B
master CallElasticsearchAsync net6.0 1.34μs 1.23ns 4.78ns 0.0132 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.62μs 0.832ns 3.11ns 0.0137 0 0 1.02 KB
master CallElasticsearchAsync net472 2.59μs 2.44ns 9.46ns 0.166 0 0 1.05 KB
#5663 CallElasticsearch net6.0 1.2μs 0.79ns 3.06ns 0.0133 0 0 976 B
#5663 CallElasticsearch netcoreapp3.1 1.53μs 0.695ns 2.6ns 0.0131 0 0 976 B
#5663 CallElasticsearch net472 2.67μs 1.85ns 7.15ns 0.157 0 0 995 B
#5663 CallElasticsearchAsync net6.0 1.26μs 0.945ns 3.54ns 0.0132 0 0 952 B
#5663 CallElasticsearchAsync netcoreapp3.1 1.62μs 1.86ns 7.22ns 0.0138 0 0 1.02 KB
#5663 CallElasticsearchAsync net472 2.68μs 1.11ns 4.3ns 0.167 0.00134 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #5663

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0 1.121 1,188.79 1,332.98

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.19μs 0.68ns 2.55ns 0.0137 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.53μs 0.95ns 3.55ns 0.0127 0 0 952 B
master ExecuteAsync net472 1.75μs 0.358ns 1.39ns 0.145 0 0 915 B
#5663 ExecuteAsync net6.0 1.33μs 0.409ns 1.53ns 0.0133 0 0 952 B
#5663 ExecuteAsync netcoreapp3.1 1.64μs 2.26ns 8.45ns 0.0123 0 0 952 B
#5663 ExecuteAsync net472 1.74μs 1.12ns 4.18ns 0.145 0.000873 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.11μs 2.65ns 10.3ns 0.0307 0 0 2.22 KB
master SendAsync netcoreapp3.1 5.1μs 3.42ns 12.8ns 0.0357 0 0 2.76 KB
master SendAsync net472 7.67μs 2.26ns 8.45ns 0.494 0 0 3.12 KB
#5663 SendAsync net6.0 4.17μs 2.27ns 8.17ns 0.0312 0 0 2.22 KB
#5663 SendAsync netcoreapp3.1 5.06μs 2.15ns 8.03ns 0.0378 0 0 2.76 KB
#5663 SendAsync net472 7.5μs 2.67ns 10.4ns 0.494 0 0 3.12 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.42μs 0.47ns 1.63ns 0.0228 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.3μs 3.78ns 14.6ns 0.0216 0 0 1.64 KB
master EnrichedLog net472 2.51μs 2.47ns 9.59ns 0.249 0 0 1.57 KB
#5663 EnrichedLog net6.0 1.44μs 0.796ns 2.87ns 0.0229 0 0 1.64 KB
#5663 EnrichedLog netcoreapp3.1 2.12μs 1.83ns 6.61ns 0.0218 0 0 1.64 KB
#5663 EnrichedLog net472 2.58μs 1.08ns 4.18ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 116μs 115ns 431ns 0 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 121μs 195ns 755ns 0 0 0 4.28 KB
master EnrichedLog net472 150μs 91.5ns 354ns 0.678 0.226 0 4.46 KB
#5663 EnrichedLog net6.0 117μs 129ns 499ns 0 0 0 4.28 KB
#5663 EnrichedLog netcoreapp3.1 119μs 322ns 1.25μs 0.0594 0 0 4.28 KB
#5663 EnrichedLog net472 151μs 311ns 1.2μs 0.679 0.226 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3μs 0.82ns 3.07ns 0.0314 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.24μs 15.7ns 60.9ns 0.0292 0 0 2.2 KB
master EnrichedLog net472 4.76μs 1.69ns 6.54ns 0.32 0 0 2.02 KB
#5663 EnrichedLog net6.0 3.05μs 0.855ns 3.31ns 0.0309 0 0 2.2 KB
#5663 EnrichedLog netcoreapp3.1 4.07μs 0.793ns 2.97ns 0.0285 0 0 2.2 KB
#5663 EnrichedLog net472 4.76μs 1.44ns 5.59ns 0.319 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #5663

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.RedisBenchmark.SendReceive‑net6.0 1.141 1,236.68 1,411.28

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.24μs 2.92ns 11.3ns 0.0159 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.77μs 0.815ns 3.16ns 0.015 0 0 1.14 KB
master SendReceive net472 2.13μs 1.31ns 4.71ns 0.183 0 0 1.16 KB
#5663 SendReceive net6.0 1.41μs 0.84ns 3.14ns 0.0162 0 0 1.14 KB
#5663 SendReceive netcoreapp3.1 1.78μs 0.759ns 2.94ns 0.0151 0 0 1.14 KB
#5663 SendReceive net472 2.05μs 0.95ns 3.68ns 0.183 0.00102 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.77μs 0.545ns 2.04ns 0.0221 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 3.92μs 2.33ns 9.03ns 0.0215 0 0 1.65 KB
master EnrichedLog net472 4.33μs 2.61ns 10.1ns 0.322 0 0 2.04 KB
#5663 EnrichedLog net6.0 2.67μs 1.04ns 3.89ns 0.0227 0 0 1.6 KB
#5663 EnrichedLog netcoreapp3.1 3.89μs 3.32ns 12.9ns 0.0217 0 0 1.65 KB
#5663 EnrichedLog net472 4.41μs 1.55ns 5.8ns 0.323 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #5663

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 1.138 448.99 394.46

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 449ns 0.114ns 0.44ns 0.00807 0 0 576 B
master StartFinishSpan netcoreapp3.1 561ns 0.11ns 0.397ns 0.00774 0 0 576 B
master StartFinishSpan net472 614ns 0.828ns 3.21ns 0.0917 0 0 578 B
master StartFinishScope net6.0 481ns 0.308ns 1.19ns 0.0099 0 0 696 B
master StartFinishScope netcoreapp3.1 779ns 0.474ns 1.84ns 0.00934 0 0 696 B
master StartFinishScope net472 856ns 0.932ns 3.61ns 0.104 0 0 658 B
#5663 StartFinishSpan net6.0 394ns 0.146ns 0.566ns 0.00815 0 0 576 B
#5663 StartFinishSpan netcoreapp3.1 565ns 1.73ns 6.68ns 0.00781 0 0 576 B
#5663 StartFinishSpan net472 587ns 0.811ns 3.14ns 0.0918 0 0 578 B
#5663 StartFinishScope net6.0 472ns 0.176ns 0.681ns 0.00993 0 0 696 B
#5663 StartFinishScope netcoreapp3.1 707ns 0.454ns 1.76ns 0.00939 0 0 696 B
#5663 StartFinishScope net472 803ns 0.353ns 1.32ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 648ns 0.396ns 1.53ns 0.00973 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 884ns 0.532ns 2.06ns 0.00933 0 0 696 B
master RunOnMethodBegin net472 1.05μs 0.56ns 2.17ns 0.104 0 0 658 B
#5663 RunOnMethodBegin net6.0 674ns 0.212ns 0.823ns 0.0095 0 0 696 B
#5663 RunOnMethodBegin netcoreapp3.1 948ns 0.274ns 1.06ns 0.00955 0 0 696 B
#5663 RunOnMethodBegin net472 1.03μs 0.579ns 2.24ns 0.104 0 0 658 B

andrewlock avatar Jun 07 '24 22:06 andrewlock