dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard

Published 20 hours ago • verified publisher icon DataDog

[Tracer] Add tests for stats computation feature

Open zacharycmontoya opened this issue 3 years ago • 5 comments

Summary of changes

Adds testing for various requirements of the stats computation feature, as agreed upon by other tracers.

This is PR 1/3 in an attempt to break down the massive PR #2988

Reason for change

Implementation details

Test coverage

Adds new unit tests (StatsAggregatorTests) and integration tests (StatsTests)

Other details

zacharycmontoya avatar Aug 03 '22 21:08 zacharycmontoya

Benchmarks Report :snail:

Benchmarks for #3047 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 719μs 673ns 2.52μs 0.359 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 458μs 220ns 824ns 0 0 0 2.58 KB
#3047 WriteAndFlushEnrichedTraces net472 721μs 204ns 789ns 0.357 0 0 3.18 KB
#3047 WriteAndFlushEnrichedTraces netcoreapp3.1 457μs 255ns 986ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.71μs 4.24ns 16.4ns 0.236 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.87μs 1.95ns 7.3ns 0.0187 0 0 1.37 KB
master AllCycleMoreComplexBody net472 16.8μs 56.3ns 218ns 1.39 0.0249 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.2μs 18.3ns 70.9ns 0.107 0 0 7.85 KB
master BodyExtractorSimpleBody net472 251ns 0.19ns 0.737ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 226ns 0.427ns 1.6ns 0.0038 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.6μs 9.35ns 33.7ns 1.21 0.0147 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 12.1ns 46.8ns 0.0905 0 0 6.75 KB
#3047 AllCycleSimpleBody net472 1.67μs 1.61ns 6.22ns 0.237 0 0 1.49 KB
#3047 AllCycleSimpleBody netcoreapp3.1 1.78μs 1.71ns 6.38ns 0.0189 0 0 1.37 KB
#3047 AllCycleMoreComplexBody net472 17μs 13.6ns 52.6ns 1.38 0.0255 0 8.75 KB
#3047 AllCycleMoreComplexBody netcoreapp3.1 14.2μs 26.4ns 102ns 0.107 0 0 7.85 KB
#3047 BodyExtractorSimpleBody net472 257ns 0.307ns 1.19ns 0.0574 0 0 361 B
#3047 BodyExtractorSimpleBody netcoreapp3.1 225ns 0.161ns 0.581ns 0.00371 0 0 272 B
#3047 BodyExtractorMoreComplexBody net472 14.7μs 11.1ns 42.9ns 1.21 0.0147 0 7.62 KB
#3047 BodyExtractorMoreComplexBody netcoreapp3.1 11.6μs 12.5ns 46.7ns 0.0925 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 181μs 166ns 642ns 0.181 0 0 20.33 KB
#3047 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3047 SendRequest netcoreapp3.1 179μs 65.5ns 236ns 0.269 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.51μs 0.67ns 2.59ns 0.126 0.000761 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.27μs 0.378ns 1.47ns 0.0114 0 0 824 B
#3047 ExecuteNonQuery net472 1.55μs 0.351ns 1.36ns 0.126 0.000785 0 794 B
#3047 ExecuteNonQuery netcoreapp3.1 1.25μs 0.301ns 1.09ns 0.0113 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.12μs 1.05ns 4.06ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.41μs 0.982ns 3.67ns 0.0134 0 0 984 B
master CallElasticsearchAsync net472 2.45μs 0.989ns 3.83ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.48μs 0.503ns 1.88ns 0.0148 0 0 1.1 KB
#3047 CallElasticsearch net472 2.22μs 0.72ns 2.69ns 0.159 0 0 1 KB
#3047 CallElasticsearch netcoreapp3.1 1.38μs 0.538ns 2.01ns 0.0136 0 0 984 B
#3047 CallElasticsearchAsync net472 2.33μs 1.2ns 4.48ns 0.181 0 0 1.14 KB
#3047 CallElasticsearchAsync netcoreapp3.1 1.49μs 0.528ns 2.04ns 0.0149 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.39μs 5.36ns 20ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.63μs 2.21ns 8.28ns 0.0164 0 0 1.22 KB
#3047 ExecuteAsync net472 2.32μs 2.24ns 8.66ns 0.2 0 0 1.26 KB
#3047 ExecuteAsync netcoreapp3.1 1.63μs 0.99ns 3.84ns 0.0164 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.92μs 5.04ns 19.5ns 0.392 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.22μs 2.52ns 9.43ns 0.032 0 0 2.36 KB
#3047 SendAsync net472 5.09μs 3.37ns 13ns 0.392 0 0 2.48 KB
#3047 SendAsync netcoreapp3.1 3.23μs 0.698ns 2.52ns 0.0323 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.73μs 0.783ns 2.82ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.36μs 0.97ns 3.76ns 0.0238 0 0 1.73 KB
#3047 EnrichedLog net472 2.8μs 1.56ns 6.05ns 0.264 0 0 1.66 KB
#3047 EnrichedLog netcoreapp3.1 2.32μs 0.593ns 2.14ns 0.0233 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 146μs 114ns 440ns 0.659 0.22 0 4.5 KB
master EnrichedLog netcoreapp3.1 112μs 144ns 557ns 0.0562 0 0 4.38 KB
#3047 EnrichedLog net472 148μs 100ns 388ns 0.659 0.22 0 4.5 KB
#3047 EnrichedLog netcoreapp3.1 113μs 98.1ns 367ns 0 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.42μs 5.29ns 19.8ns 0.543 0.0027 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.39μs 2.26ns 8.77ns 0.0525 0 0 3.8 KB
#3047 EnrichedLog net472 5.44μs 2.53ns 9.46ns 0.544 0.00272 0 3.43 KB
#3047 EnrichedLog netcoreapp3.1 4.49μs 2ns 7.21ns 0.0511 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.02μs 1.61ns 6.25ns 0.194 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.72μs 0.607ns 2.27ns 0.0163 0 0 1.21 KB
#3047 SendReceive net472 1.99μs 0.382ns 1.48ns 0.193 0 0 1.22 KB
#3047 SendReceive netcoreapp3.1 1.68μs 0.487ns 1.82ns 0.016 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.63μs 1.88ns 7.05ns 0.328 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.22μs 1.37ns 5.32ns 0.0232 0 0 1.69 KB
#3047 EnrichedLog net472 4.7μs 2.94ns 11.4ns 0.329 0 0 2.08 KB
#3047 EnrichedLog netcoreapp3.1 4.23μs 1.15ns 4.46ns 0.0211 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 849ns 0.263ns 0.985ns 0.105 0 0 658 B
master StartFinishSpan netcoreapp3.1 762ns 0.226ns 0.875ns 0.00877 0 0 648 B
master StartFinishScope net472 1.09μs 0.281ns 1.05ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 888ns 0.66ns 2.47ns 0.0101 0 0 768 B
#3047 StartFinishSpan net472 871ns 0.209ns 0.754ns 0.104 0 0 658 B
#3047 StartFinishSpan netcoreapp3.1 794ns 0.324ns 1.21ns 0.00862 0 0 648 B
#3047 StartFinishScope net472 1.1μs 0.416ns 1.56ns 0.117 0 0 738 B
#3047 StartFinishScope netcoreapp3.1 862ns 0.661ns 2.56ns 0.0103 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.19μs 0.319ns 1.23ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.01μs 0.491ns 1.84ns 0.0101 0 0 768 B
#3047 RunOnMethodBegin net472 1.22μs 0.459ns 1.72ns 0.117 0 0 738 B
#3047 RunOnMethodBegin netcoreapp3.1 1.04μs 0.369ns 1.43ns 0.0103 0 0 768 B

andrewlock avatar Aug 03 '22 22:08 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #3047 into master will not change line coverage :heavy_check_mark: Merging #3047 into master will not change branch coverage :heavy_check_mark: Merging #3047 into master will not change complexity

master #3047 Change
Lines 16775 / 22859 16750 / 22859
Lines % 73% 73% 0% :heavy_check_mark:
Branches 9996 / 14184 9998 / 14184
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15245 15245 0 :heavy_check_mark:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #3047 Change
Lines % 73% 73% 0% :heavy_check_mark:
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15245 15245 0 :heavy_check_mark:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.GitInfo -17% :no_entry: -11% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:

View the full reports for further details:

andrewlock avatar Aug 03 '22 23:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3047 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.328
  • 1 benchmarks are slower, with geometric mean 1.116
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 722μs 326ns 1.26μs 0.359 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 478μs 344ns 1.33μs 0 0 0 2.58 KB
#3047 WriteAndFlushEnrichedTraces net472 717μs 583ns 2.26μs 0.357 0 0 3.18 KB
#3047 WriteAndFlushEnrichedTraces netcoreapp3.1 473μs 329ns 1.27μs 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #3047

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AppSecBodyBenchmark.AllCycleSimpleBody‑net472 1.328 2,208.74 1,663.02

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 2.21μs 2.52ns 9.44ns 0.237 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.8μs 2.11ns 8.16ns 0.0186 0 0 1.37 KB
master AllCycleMoreComplexBody net472 17.1μs 14.3ns 53.6ns 1.38 0.0171 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.2μs 17.1ns 63.8ns 0.107 0 0 7.85 KB
master BodyExtractorSimpleBody net472 277ns 0.284ns 1.02ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 227ns 0.136ns 0.509ns 0.00376 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.1μs 6.97ns 26.1ns 1.21 0.0151 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12μs 16.6ns 62.2ns 0.0902 0 0 6.75 KB
#3047 AllCycleSimpleBody net472 1.66μs 1.47ns 5.69ns 0.237 0 0 1.49 KB
#3047 AllCycleSimpleBody netcoreapp3.1 1.86μs 3.3ns 12.8ns 0.0187 0 0 1.37 KB
#3047 AllCycleMoreComplexBody net472 16.7μs 9.23ns 35.7ns 1.39 0.0249 0 8.75 KB
#3047 AllCycleMoreComplexBody netcoreapp3.1 14.6μs 10.7ns 40ns 0.101 0 0 7.85 KB
#3047 BodyExtractorSimpleBody net472 274ns 0.28ns 1.05ns 0.0573 0 0 361 B
#3047 BodyExtractorSimpleBody netcoreapp3.1 213ns 0.273ns 1.02ns 0.00371 0 0 272 B
#3047 BodyExtractorMoreComplexBody net472 15μs 7.99ns 31ns 1.2 0.0158 0 7.62 KB
#3047 BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 11.9ns 41.2ns 0.0922 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 177μs 225ns 872ns 0.266 0 0 20.33 KB
#3047 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3047 SendRequest netcoreapp3.1 179μs 88.5ns 331ns 0.267 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.57μs 0.513ns 1.92ns 0.126 0.000785 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.22μs 0.346ns 1.29ns 0.011 0 0 824 B
#3047 ExecuteNonQuery net472 1.5μs 0.482ns 1.87ns 0.126 0.000752 0 794 B
#3047 ExecuteNonQuery netcoreapp3.1 1.28μs 0.35ns 1.31ns 0.0108 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.28μs 0.705ns 2.73ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.43μs 0.613ns 2.3ns 0.0136 0 0 984 B
master CallElasticsearchAsync net472 2.43μs 0.81ns 3.14ns 0.18 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.49μs 0.281ns 1.09ns 0.0149 0 0 1.1 KB
#3047 CallElasticsearch net472 2.2μs 0.615ns 2.3ns 0.159 0 0 1 KB
#3047 CallElasticsearch netcoreapp3.1 1.42μs 0.385ns 1.39ns 0.0134 0 0 984 B
#3047 CallElasticsearchAsync net472 2.25μs 1.03ns 4ns 0.181 0 0 1.14 KB
#3047 CallElasticsearchAsync netcoreapp3.1 1.5μs 0.753ns 2.82ns 0.015 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.34μs 5.11ns 19.8ns 0.199 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.59μs 2.72ns 10.2ns 0.0164 0 0 1.22 KB
#3047 ExecuteAsync net472 2.46μs 3.7ns 14.3ns 0.2 0 0 1.26 KB
#3047 ExecuteAsync netcoreapp3.1 1.57μs 0.593ns 2.22ns 0.0167 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.88μs 8.01ns 31ns 0.393 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.28μs 7.51ns 29.1ns 0.0324 0 0 2.36 KB
#3047 SendAsync net472 4.99μs 4.96ns 19.2ns 0.393 0 0 2.48 KB
#3047 SendAsync netcoreapp3.1 3.3μs 3.24ns 12.6ns 0.0312 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.89μs 0.935ns 3.5ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.29μs 0.961ns 3.72ns 0.0239 0 0 1.73 KB
#3047 EnrichedLog net472 2.83μs 0.898ns 3.48ns 0.264 0 0 1.66 KB
#3047 EnrichedLog netcoreapp3.1 2.37μs 0.892ns 3.45ns 0.0237 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 146μs 93.9ns 364ns 0.66 0.22 0 4.5 KB
master EnrichedLog netcoreapp3.1 113μs 82.4ns 308ns 0.056 0 0 4.38 KB
#3047 EnrichedLog net472 148μs 47.2ns 183ns 0.665 0.222 0 4.5 KB
#3047 EnrichedLog netcoreapp3.1 113μs 94.4ns 366ns 0.0561 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.32μs 12.7ns 49.4ns 0.545 0.00262 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.22μs 6.58ns 25.5ns 0.0512 0 0 3.8 KB
#3047 EnrichedLog net472 5.27μs 3.75ns 14.5ns 0.546 0.00262 0 3.43 KB
#3047 EnrichedLog netcoreapp3.1 4.36μs 1.52ns 5.49ns 0.0521 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.98μs 1.18ns 4.43ns 0.194 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.61μs 0.467ns 1.75ns 0.0162 0 0 1.21 KB
#3047 SendReceive net472 2μs 0.688ns 2.48ns 0.194 0 0 1.22 KB
#3047 SendReceive netcoreapp3.1 1.66μs 0.827ns 3.09ns 0.0166 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.78μs 1.93ns 7.46ns 0.329 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.12μs 1.73ns 6.48ns 0.0227 0 0 1.69 KB
#3047 EnrichedLog net472 4.63μs 1.77ns 6.86ns 0.329 0 0 2.08 KB
#3047 EnrichedLog netcoreapp3.1 4.06μs 1.34ns 4.84ns 0.0225 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 860ns 0.59ns 2.28ns 0.105 0 0 658 B
master StartFinishSpan netcoreapp3.1 733ns 0.251ns 0.906ns 0.00875 0 0 648 B
master StartFinishScope net472 1.05μs 0.702ns 2.63ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 881ns 0.501ns 1.87ns 0.0101 0 0 768 B
#3047 StartFinishSpan net472 839ns 0.402ns 1.5ns 0.105 0 0 658 B
#3047 StartFinishSpan netcoreapp3.1 772ns 0.329ns 1.23ns 0.00885 0 0 648 B
#3047 StartFinishScope net472 1.01μs 0.231ns 0.894ns 0.117 0 0 738 B
#3047 StartFinishScope netcoreapp3.1 928ns 0.258ns 0.965ns 0.0107 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #3047

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472 1.116 1,155.89 1,289.93

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.16μs 0.7ns 2.71ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.04μs 0.506ns 1.75ns 0.0104 0 0 768 B
#3047 RunOnMethodBegin net472 1.29μs 0.32ns 1.24ns 0.117 0 0 738 B
#3047 RunOnMethodBegin netcoreapp3.1 993ns 0.369ns 1.43ns 0.0104 0 0 768 B

andrewlock avatar Aug 04 '22 16:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3047 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.333
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 722μs 326ns 1.26μs 0.359 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 478μs 344ns 1.33μs 0 0 0 2.58 KB
#3047 WriteAndFlushEnrichedTraces net472 717μs 757ns 2.93μs 0.357 0 0 3.18 KB
#3047 WriteAndFlushEnrichedTraces netcoreapp3.1 463μs 217ns 814ns 0 0 0 2.59 KB
Benchmarks.Trace.AppSecBodyBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #3047

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AppSecBodyBenchmark.AllCycleSimpleBody‑net472 1.333 2,208.74 1,657.47

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 2.21μs 2.52ns 9.44ns 0.237 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.8μs 2.11ns 8.16ns 0.0186 0 0 1.37 KB
master AllCycleMoreComplexBody net472 17.1μs 14.3ns 53.6ns 1.38 0.0171 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.2μs 17.1ns 63.8ns 0.107 0 0 7.85 KB
master BodyExtractorSimpleBody net472 277ns 0.284ns 1.02ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 227ns 0.136ns 0.509ns 0.00376 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.1μs 6.97ns 26.1ns 1.21 0.0151 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12μs 16.6ns 62.2ns 0.0902 0 0 6.75 KB
#3047 AllCycleSimpleBody net472 1.66μs 1.77ns 6.87ns 0.237 0 0 1.49 KB
#3047 AllCycleSimpleBody netcoreapp3.1 1.77μs 2.26ns 8.73ns 0.0183 0 0 1.37 KB
#3047 AllCycleMoreComplexBody net472 17μs 14.1ns 52.8ns 1.38 0.0253 0 8.75 KB
#3047 AllCycleMoreComplexBody netcoreapp3.1 13.8μs 14.4ns 52ns 0.104 0 0 7.85 KB
#3047 BodyExtractorSimpleBody net472 260ns 0.803ns 3.11ns 0.0573 0 0 361 B
#3047 BodyExtractorSimpleBody netcoreapp3.1 214ns 0.132ns 0.477ns 0.00373 0 0 272 B
#3047 BodyExtractorMoreComplexBody net472 14.7μs 10.2ns 35.2ns 1.21 0.0147 0 7.62 KB
#3047 BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 10ns 36.2ns 0.0906 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 177μs 225ns 872ns 0.266 0 0 20.33 KB
#3047 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3047 SendRequest netcoreapp3.1 178μs 105ns 406ns 0.266 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.57μs 0.513ns 1.92ns 0.126 0.000785 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.22μs 0.346ns 1.29ns 0.011 0 0 824 B
#3047 ExecuteNonQuery net472 1.54μs 0.506ns 1.96ns 0.126 0.000772 0 794 B
#3047 ExecuteNonQuery netcoreapp3.1 1.28μs 0.212ns 0.765ns 0.0111 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.28μs 0.705ns 2.73ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.43μs 0.613ns 2.3ns 0.0136 0 0 984 B
master CallElasticsearchAsync net472 2.43μs 0.81ns 3.14ns 0.18 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.49μs 0.281ns 1.09ns 0.0149 0 0 1.1 KB
#3047 CallElasticsearch net472 2.21μs 0.837ns 3.24ns 0.159 0 0 1 KB
#3047 CallElasticsearch netcoreapp3.1 1.38μs 0.882ns 3.18ns 0.0131 0 0 984 B
#3047 CallElasticsearchAsync net472 2.3μs 0.698ns 2.7ns 0.181 0 0 1.14 KB
#3047 CallElasticsearchAsync netcoreapp3.1 1.5μs 0.452ns 1.63ns 0.0149 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.34μs 5.11ns 19.8ns 0.199 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.59μs 2.72ns 10.2ns 0.0164 0 0 1.22 KB
#3047 ExecuteAsync net472 2.44μs 1.98ns 7.65ns 0.2 0 0 1.26 KB
#3047 ExecuteAsync netcoreapp3.1 1.64μs 0.737ns 2.85ns 0.0163 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.88μs 8.01ns 31ns 0.393 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.28μs 7.51ns 29.1ns 0.0324 0 0 2.36 KB
#3047 SendAsync net472 5.02μs 1.62ns 6.29ns 0.393 0 0 2.48 KB
#3047 SendAsync netcoreapp3.1 3.23μs 4.65ns 18ns 0.0319 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.89μs 0.935ns 3.5ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.29μs 0.961ns 3.72ns 0.0239 0 0 1.73 KB
#3047 EnrichedLog net472 2.99μs 1.25ns 4.69ns 0.263 0 0 1.66 KB
#3047 EnrichedLog netcoreapp3.1 2.52μs 0.862ns 3.34ns 0.0238 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 146μs 93.9ns 364ns 0.66 0.22 0 4.5 KB
master EnrichedLog netcoreapp3.1 113μs 82.4ns 308ns 0.056 0 0 4.38 KB
#3047 EnrichedLog net472 147μs 78.3ns 293ns 0.666 0.222 0 4.5 KB
#3047 EnrichedLog netcoreapp3.1 113μs 70.1ns 272ns 0.0559 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.32μs 12.7ns 49.4ns 0.545 0.00262 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.22μs 6.58ns 25.5ns 0.0512 0 0 3.8 KB
#3047 EnrichedLog net472 5.36μs 4.53ns 17.6ns 0.544 0.00268 0 3.43 KB
#3047 EnrichedLog netcoreapp3.1 4.3μs 2.81ns 10.5ns 0.0515 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.98μs 1.18ns 4.43ns 0.194 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.61μs 0.467ns 1.75ns 0.0162 0 0 1.21 KB
#3047 SendReceive net472 2.03μs 2.25ns 8.71ns 0.193 0 0 1.22 KB
#3047 SendReceive netcoreapp3.1 1.65μs 0.791ns 2.96ns 0.0165 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.78μs 1.93ns 7.46ns 0.329 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.12μs 1.73ns 6.48ns 0.0227 0 0 1.69 KB
#3047 EnrichedLog net472 4.57μs 1.8ns 6.98ns 0.33 0 0 2.08 KB
#3047 EnrichedLog netcoreapp3.1 4.12μs 1.93ns 7.21ns 0.0226 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 860ns 0.59ns 2.28ns 0.105 0 0 658 B
master StartFinishSpan netcoreapp3.1 733ns 0.251ns 0.906ns 0.00875 0 0 648 B
master StartFinishScope net472 1.05μs 0.702ns 2.63ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 881ns 0.501ns 1.87ns 0.0101 0 0 768 B
#3047 StartFinishSpan net472 877ns 0.35ns 1.36ns 0.104 0 0 658 B
#3047 StartFinishSpan netcoreapp3.1 776ns 0.133ns 0.515ns 0.0089 0 0 648 B
#3047 StartFinishScope net472 1.01μs 0.253ns 0.981ns 0.117 0 0 738 B
#3047 StartFinishScope netcoreapp3.1 899ns 1.39ns 4.82ns 0.0104 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.16μs 0.7ns 2.71ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.04μs 0.506ns 1.75ns 0.0104 0 0 768 B
#3047 RunOnMethodBegin net472 1.22μs 0.519ns 2.01ns 0.117 0 0 738 B
#3047 RunOnMethodBegin netcoreapp3.1 1.06μs 0.413ns 1.55ns 0.0101 0 0 768 B

andrewlock avatar Aug 04 '22 17:08 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #3047 into master will not change line coverage :heavy_check_mark: Merging #3047 into master will not change branch coverage :heavy_check_mark: Merging #3047 into master will not change complexity

master #3047 Change
Lines 16778 / 22859 16748 / 22859
Lines % 73% 73% 0% :heavy_check_mark:
Branches 9999 / 14184 9999 / 14184
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15245 15245 0 :heavy_check_mark:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #3047 Change
Lines % 73% 73% 0% :heavy_check_mark:
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15245 15245 0 :heavy_check_mark:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.GitInfo -17% :no_entry: -11% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:

View the full reports for further details:

andrewlock avatar Aug 04 '22 17:08 andrewlock