dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard

[Debugger] Fixed serialization issues in producing snapshots

Open GreenMatan opened this issue 3 years ago • 6 comments

Summary of changes

Fixed various snapshot serialization issues that resulted in failure of producing snapshots.

Test coverage

Added a test MultidimensionalArrayTest that reproduced an issue seen in a method that resides in the Newtonsoft.Json package.

GreenMatan avatar Aug 09 '22 08:08 GreenMatan

Benchmarks Report :snail:

Benchmarks for #3064 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 717μs 191ns 741ns 0.357 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 459μs 164ns 591ns 0 0 0 2.58 KB
#3064 WriteAndFlushEnrichedTraces net472 714μs 415ns 1.61μs 0.357 0 0 3.18 KB
#3064 WriteAndFlushEnrichedTraces netcoreapp3.1 459μs 99.4ns 372ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.69μs 0.996ns 3.59ns 0.237 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.81μs 3.49ns 13.5ns 0.0189 0 0 1.37 KB
master AllCycleMoreComplexBody net472 17.2μs 16.5ns 61.9ns 1.39 0.025 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 15μs 22.4ns 83.9ns 0.105 0 0 7.85 KB
master BodyExtractorSimpleBody net472 262ns 0.433ns 1.68ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 226ns 0.303ns 1.14ns 0.00364 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.8μs 11.9ns 43.1ns 1.21 0.0148 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.1μs 13.3ns 49.9ns 0.0901 0 0 6.75 KB
#3064 AllCycleSimpleBody net472 1.73μs 2.44ns 9.12ns 0.237 0 0 1.49 KB
#3064 AllCycleSimpleBody netcoreapp3.1 1.84μs 2.52ns 9.76ns 0.019 0 0 1.37 KB
#3064 AllCycleMoreComplexBody net472 16.6μs 13.8ns 53.6ns 1.39 0.0249 0 8.75 KB
#3064 AllCycleMoreComplexBody netcoreapp3.1 14.7μs 18.9ns 70.7ns 0.102 0 0 7.85 KB
#3064 BodyExtractorSimpleBody net472 259ns 0.304ns 1.18ns 0.0573 0 0 361 B
#3064 BodyExtractorSimpleBody netcoreapp3.1 221ns 0.272ns 1.05ns 0.00368 0 0 272 B
#3064 BodyExtractorMoreComplexBody net472 15.1μs 18ns 69.7ns 1.2 0.0151 0 7.62 KB
#3064 BodyExtractorMoreComplexBody netcoreapp3.1 11.7μs 14.9ns 53.8ns 0.0933 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 180μs 140ns 525ns 0.269 0 0 20.33 KB
#3064 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3064 SendRequest netcoreapp3.1 180μs 189ns 731ns 0.269 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.59μs 0.656ns 2.54ns 0.126 0.000781 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.25μs 0.864ns 3.23ns 0.0112 0 0 824 B
#3064 ExecuteNonQuery net472 1.47μs 0.619ns 2.4ns 0.126 0.000736 0 794 B
#3064 ExecuteNonQuery netcoreapp3.1 1.25μs 0.401ns 1.45ns 0.0112 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.23μs 0.905ns 3.5ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.41μs 1.35ns 5.05ns 0.0132 0 0 984 B
master CallElasticsearchAsync net472 2.34μs 1.78ns 6.89ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.43μs 0.468ns 1.81ns 0.0149 0 0 1.1 KB
#3064 CallElasticsearch net472 2.06μs 1.02ns 3.83ns 0.159 0 0 1 KB
#3064 CallElasticsearch netcoreapp3.1 1.41μs 0.809ns 3.14ns 0.0133 0 0 984 B
#3064 CallElasticsearchAsync net472 2.33μs 1.35ns 5.23ns 0.181 0 0 1.14 KB
#3064 CallElasticsearchAsync netcoreapp3.1 1.45μs 0.544ns 2.04ns 0.0146 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.47μs 3.57ns 13.8ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.58μs 2.09ns 7.82ns 0.0166 0 0 1.22 KB
#3064 ExecuteAsync net472 2.28μs 6.58ns 24.6ns 0.199 0 0 1.26 KB
#3064 ExecuteAsync netcoreapp3.1 1.57μs 3.16ns 11.8ns 0.0162 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.88μs 7.16ns 27.7ns 0.393 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.21μs 5.27ns 20.4ns 0.032 0 0 2.36 KB
#3064 SendAsync net472 4.88μs 11ns 42.6ns 0.393 0 0 2.48 KB
#3064 SendAsync netcoreapp3.1 3.15μs 5.62ns 21.8ns 0.031 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.87μs 1.85ns 6.67ns 0.264 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.38μs 1.17ns 4.38ns 0.0238 0 0 1.73 KB
#3064 EnrichedLog net472 2.89μs 1.06ns 4.11ns 0.264 0 0 1.66 KB
#3064 EnrichedLog netcoreapp3.1 2.46μs 1.47ns 5.7ns 0.0232 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 147μs 84.7ns 317ns 0.667 0.222 0 4.5 KB
master EnrichedLog netcoreapp3.1 114μs 94.7ns 354ns 0 0 0 4.38 KB
#3064 EnrichedLog net472 143μs 143ns 554ns 0.645 0.215 0 4.5 KB
#3064 EnrichedLog netcoreapp3.1 113μs 169ns 654ns 0.0564 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.36μs 8.09ns 31.3ns 0.545 0.00269 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.21μs 5.2ns 19.4ns 0.0522 0 0 3.8 KB
#3064 EnrichedLog net472 5.33μs 16.4ns 63.4ns 0.544 0.00267 0 3.43 KB
#3064 EnrichedLog netcoreapp3.1 4.17μs 8.23ns 31.9ns 0.0524 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.97μs 0.415ns 1.61ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.68μs 0.42ns 1.57ns 0.016 0 0 1.21 KB
#3064 SendReceive net472 2.04μs 0.896ns 3.47ns 0.193 0 0 1.22 KB
#3064 SendReceive netcoreapp3.1 1.64μs 0.49ns 1.9ns 0.0164 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.69μs 0.87ns 3.25ns 0.33 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.1μs 1.22ns 4.71ns 0.0228 0 0 1.69 KB
#3064 EnrichedLog net472 4.6μs 1.78ns 6.89ns 0.33 0 0 2.08 KB
#3064 EnrichedLog netcoreapp3.1 4.13μs 1.26ns 4.55ns 0.0226 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 866ns 0.312ns 1.21ns 0.105 0 0 658 B
master StartFinishSpan netcoreapp3.1 718ns 0.197ns 0.709ns 0.00902 0 0 648 B
master StartFinishScope net472 1.08μs 0.348ns 1.3ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 848ns 0.287ns 1.07ns 0.0106 0 0 768 B
#3064 StartFinishSpan net472 842ns 0.261ns 1.01ns 0.104 0 0 658 B
#3064 StartFinishSpan netcoreapp3.1 718ns 0.309ns 1.2ns 0.00866 0 0 648 B
#3064 StartFinishScope net472 1.04μs 0.495ns 1.92ns 0.117 0 0 738 B
#3064 StartFinishScope netcoreapp3.1 891ns 0.344ns 1.29ns 0.0102 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.22μs 0.42ns 1.57ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.02μs 0.257ns 0.962ns 0.0103 0 0 768 B
#3064 RunOnMethodBegin net472 1.2μs 0.446ns 1.73ns 0.117 0 0 738 B
#3064 RunOnMethodBegin netcoreapp3.1 1.02μs 0.784ns 2.72ns 0.0102 0 0 768 B

andrewlock avatar Aug 09 '22 09:08 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #3064 into master will not change line coverage :heavy_check_mark: Merging #3064 into master will not change branch coverage :no_entry: Merging #3064 into master will will increase complexity by 10

master #3064 Change
Lines 16772 / 22876 16845 / 22883
Lines % 73% 74% 0% :heavy_check_mark:
Branches 9996 / 14196 10043 / 14206
Branches % 70% 71% 0% :heavy_check_mark:
Complexity 15264 15274 10 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #3064 Change
Lines % 73% 74% 0% :heavy_check_mark:
Branches % 70% 71% 0% :heavy_check_mark:
Complexity 15264 15274 10 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Pdb.DatadogPdbReader 1% :heavy_check_mark: 12% :heavy_check_mark: 2 :no_entry:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:

View the full reports for further details:

andrewlock avatar Aug 09 '22 10:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3064 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 716μs 304ns 1.14μs 0.359 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 483μs 205ns 768ns 0 0 0 2.58 KB
#3064 WriteAndFlushEnrichedTraces net472 717μs 229ns 888ns 0.357 0 0 3.18 KB
#3064 WriteAndFlushEnrichedTraces netcoreapp3.1 462μs 155ns 602ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.74μs 2.3ns 8.59ns 0.237 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.77μs 2.02ns 7.27ns 0.0185 0 0 1.37 KB
master AllCycleMoreComplexBody net472 17.7μs 5.09ns 17.6ns 1.39 0.0256 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.3μs 18.3ns 70.9ns 0.106 0 0 7.85 KB
master BodyExtractorSimpleBody net472 264ns 0.627ns 2.43ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 220ns 0.292ns 1.13ns 0.00375 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.6μs 11.5ns 41.5ns 1.21 0.0146 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 11.8μs 20.1ns 72.5ns 0.0886 0 0 6.75 KB
#3064 AllCycleSimpleBody net472 1.66μs 0.823ns 3.19ns 0.237 0 0 1.49 KB
#3064 AllCycleSimpleBody netcoreapp3.1 1.8μs 1.45ns 5.21ns 0.0189 0 0 1.37 KB
#3064 AllCycleMoreComplexBody net472 17μs 10.1ns 37.8ns 1.38 0.0254 0 8.75 KB
#3064 AllCycleMoreComplexBody netcoreapp3.1 13.9μs 15.2ns 56.8ns 0.105 0 0 7.85 KB
#3064 BodyExtractorSimpleBody net472 272ns 0.146ns 0.505ns 0.0573 0 0 361 B
#3064 BodyExtractorSimpleBody netcoreapp3.1 222ns 0.197ns 0.764ns 0.00369 0 0 272 B
#3064 BodyExtractorMoreComplexBody net472 15.1μs 10.2ns 39.4ns 1.21 0.015 0 7.62 KB
#3064 BodyExtractorMoreComplexBody netcoreapp3.1 12.3μs 13ns 48.5ns 0.0921 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 177μs 101ns 393ns 0.177 0 0 20.33 KB
#3064 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3064 SendRequest netcoreapp3.1 179μs 147ns 568ns 0.267 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.53μs 0.726ns 2.81ns 0.126 0.000763 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.23μs 0.429ns 1.55ns 0.0106 0 0 824 B
#3064 ExecuteNonQuery net472 1.51μs 0.38ns 1.42ns 0.126 0.00075 0 794 B
#3064 ExecuteNonQuery netcoreapp3.1 1.3μs 0.374ns 1.35ns 0.0113 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.14μs 0.514ns 1.85ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.43μs 0.919ns 3.56ns 0.0135 0 0 984 B
master CallElasticsearchAsync net472 2.32μs 1.19ns 4.46ns 0.18 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.46μs 0.542ns 2.03ns 0.0146 0 0 1.1 KB
#3064 CallElasticsearch net472 2.24μs 0.601ns 2.25ns 0.159 0 0 1 KB
#3064 CallElasticsearch netcoreapp3.1 1.39μs 0.596ns 2.31ns 0.0132 0 0 984 B
#3064 CallElasticsearchAsync net472 2.33μs 1.4ns 5.22ns 0.181 0 0 1.14 KB
#3064 CallElasticsearchAsync netcoreapp3.1 1.44μs 0.56ns 2.02ns 0.0144 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.33μs 6.28ns 23.5ns 0.199 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.6μs 4.03ns 15.1ns 0.0167 0 0 1.22 KB
#3064 ExecuteAsync net472 2.47μs 5.27ns 20.4ns 0.2 0 0 1.26 KB
#3064 ExecuteAsync netcoreapp3.1 1.56μs 2.77ns 10.4ns 0.0163 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.94μs 11.5ns 44.5ns 0.394 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.18μs 5.93ns 23ns 0.0321 0 0 2.36 KB
#3064 SendAsync net472 4.92μs 7.26ns 28.1ns 0.392 0 0 2.48 KB
#3064 SendAsync netcoreapp3.1 3.29μs 5.53ns 21.4ns 0.0326 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.88μs 0.776ns 2.69ns 0.264 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.35μs 0.907ns 3.4ns 0.0237 0 0 1.73 KB
#3064 EnrichedLog net472 2.85μs 1.4ns 5.43ns 0.264 0 0 1.66 KB
#3064 EnrichedLog netcoreapp3.1 2.26μs 1.33ns 4.96ns 0.0237 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 147μs 68.9ns 267ns 0.666 0.222 0 4.5 KB
master EnrichedLog netcoreapp3.1 111μs 47ns 163ns 0.0557 0 0 4.38 KB
#3064 EnrichedLog net472 149μs 183ns 685ns 0.659 0.22 0 4.5 KB
#3064 EnrichedLog netcoreapp3.1 112μs 61.3ns 237ns 0.0562 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.28μs 18.1ns 69.9ns 0.544 0.00264 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.07μs 8.62ns 33.4ns 0.0511 0 0 3.8 KB
#3064 EnrichedLog net472 5.3μs 16.7ns 64.8ns 0.543 0.00269 0 3.43 KB
#3064 EnrichedLog netcoreapp3.1 4.32μs 3.35ns 12.5ns 0.053 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.93μs 0.84ns 3.25ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.66μs 0.686ns 2.47ns 0.0159 0 0 1.21 KB
#3064 SendReceive net472 2.04μs 1.36ns 5.26ns 0.194 0 0 1.22 KB
#3064 SendReceive netcoreapp3.1 1.64μs 0.497ns 1.93ns 0.0164 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.74μs 1.27ns 4.74ns 0.329 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 3.99μs 1.24ns 4.48ns 0.0221 0 0 1.69 KB
#3064 EnrichedLog net472 4.77μs 1.69ns 6.53ns 0.329 0 0 2.08 KB
#3064 EnrichedLog netcoreapp3.1 3.97μs 3.26ns 12.2ns 0.0218 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 833ns 0.401ns 1.55ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 724ns 0.302ns 1.17ns 0.00871 0 0 648 B
master StartFinishScope net472 1.03μs 0.51ns 1.91ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 894ns 0.357ns 1.34ns 0.0103 0 0 768 B
#3064 StartFinishSpan net472 866ns 0.267ns 1ns 0.104 0 0 658 B
#3064 StartFinishSpan netcoreapp3.1 713ns 0.507ns 1.9ns 0.00894 0 0 648 B
#3064 StartFinishScope net472 1.06μs 0.381ns 1.43ns 0.117 0 0 738 B
#3064 StartFinishScope netcoreapp3.1 845ns 0.336ns 1.3ns 0.0102 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.2μs 0.596ns 2.31ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.06μs 0.797ns 3.09ns 0.0101 0 0 768 B
#3064 RunOnMethodBegin net472 1.26μs 0.444ns 1.72ns 0.117 0 0 738 B
#3064 RunOnMethodBegin netcoreapp3.1 1.01μs 0.457ns 1.65ns 0.0102 0 0 768 B

andrewlock avatar Aug 10 '22 12:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3064 compared to master:

  • 3 benchmarks are slower, with geometric mean 1.169
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 716μs 348ns 1.3μs 0.359 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 473μs 228ns 882ns 0 0 0 2.58 KB
#3064 WriteAndFlushEnrichedTraces net472 723μs 577ns 2.23μs 0.359 0 0 3.18 KB
#3064 WriteAndFlushEnrichedTraces netcoreapp3.1 482μs 213ns 797ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.77μs 2.6ns 9.71ns 0.237 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.84μs 1.85ns 6.9ns 0.0184 0 0 1.37 KB
master AllCycleMoreComplexBody net472 17.4μs 11.6ns 43.3ns 1.39 0.0252 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.3μs 20.8ns 80.7ns 0.1 0 0 7.85 KB
master BodyExtractorSimpleBody net472 267ns 0.497ns 1.86ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 229ns 0.271ns 1.05ns 0.00378 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.7μs 5.16ns 17.9ns 1.21 0.0146 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12μs 12.2ns 47.3ns 0.0904 0 0 6.75 KB
#3064 AllCycleSimpleBody net472 1.69μs 1.99ns 7.69ns 0.237 0 0 1.49 KB
#3064 AllCycleSimpleBody netcoreapp3.1 1.83μs 3.47ns 13.4ns 0.0187 0 0 1.37 KB
#3064 AllCycleMoreComplexBody net472 16.8μs 16.5ns 63.8ns 1.38 0.0251 0 8.75 KB
#3064 AllCycleMoreComplexBody netcoreapp3.1 14.1μs 23.5ns 88.1ns 0.105 0 0 7.85 KB
#3064 BodyExtractorSimpleBody net472 254ns 0.225ns 0.842ns 0.0574 0 0 361 B
#3064 BodyExtractorSimpleBody netcoreapp3.1 226ns 0.2ns 0.722ns 0.00367 0 0 272 B
#3064 BodyExtractorMoreComplexBody net472 15.1μs 5.95ns 20.6ns 1.21 0.015 0 7.62 KB
#3064 BodyExtractorMoreComplexBody netcoreapp3.1 11.8μs 24.3ns 90.9ns 0.0873 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 181μs 117ns 453ns 0.18 0 0 20.33 KB
#3064 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3064 SendRequest netcoreapp3.1 181μs 179ns 692ns 0.18 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.61μs 0.823ns 3.19ns 0.126 0.000812 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.21μs 0.443ns 1.6ns 0.0108 0 0 824 B
#3064 ExecuteNonQuery net472 1.58μs 0.534ns 2ns 0.126 0.000793 0 794 B
#3064 ExecuteNonQuery netcoreapp3.1 1.22μs 0.391ns 1.41ns 0.011 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.24μs 0.713ns 2.76ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.4μs 1.25ns 4.68ns 0.0131 0 0 984 B
master CallElasticsearchAsync net472 2.45μs 1.04ns 3.76ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.49μs 1.65ns 6.18ns 0.0149 0 0 1.1 KB
#3064 CallElasticsearch net472 2.28μs 0.968ns 3.75ns 0.159 0 0 1 KB
#3064 CallElasticsearch netcoreapp3.1 1.39μs 0.637ns 2.38ns 0.0131 0 0 984 B
#3064 CallElasticsearchAsync net472 2.29μs 0.585ns 2.27ns 0.18 0 0 1.14 KB
#3064 CallElasticsearchAsync netcoreapp3.1 1.49μs 0.602ns 2.25ns 0.015 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #3064

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑netcoreapp3.1 1.207 1,559.90 1,883.43

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.43μs 2.48ns 9.59ns 0.199 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.56μs 2.25ns 8.7ns 0.0162 0 0 1.22 KB
#3064 ExecuteAsync net472 2.6μs 14.5ns 93.1ns 0.199 0 0 1.26 KB
#3064 ExecuteAsync netcoreapp3.1 1.92μs 18.2ns 180ns 0.0165 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #3064

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.HttpClientBenchmark.SendAsync‑netcoreapp3.1 1.173 3,210.97 3,766.84
Benchmarks.Trace.HttpClientBenchmark.SendAsync‑net472 1.128 4,875.36 5,500.72

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.87μs 13.2ns 51.3ns 0.393 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.21μs 3.95ns 15.3ns 0.032 0 0 2.36 KB
#3064 SendAsync net472 5.49μs 27.4ns 125ns 0.392 0 0 2.48 KB
#3064 SendAsync netcoreapp3.1 3.86μs 49.9ns 491ns 0.0311 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.88μs 0.825ns 3.09ns 0.264 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.51μs 2.01ns 7.52ns 0.0224 0 0 1.73 KB
#3064 EnrichedLog net472 2.83μs 0.838ns 3.25ns 0.263 0 0 1.66 KB
#3064 EnrichedLog netcoreapp3.1 2.38μs 1.39ns 5.21ns 0.0237 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 147μs 49.6ns 179ns 0.666 0.222 0 4.5 KB
master EnrichedLog netcoreapp3.1 113μs 192ns 691ns 0 0 0 4.38 KB
#3064 EnrichedLog net472 147μs 73.9ns 276ns 0.661 0.22 0 4.5 KB
#3064 EnrichedLog netcoreapp3.1 113μs 157ns 608ns 0 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.35μs 6.87ns 26.6ns 0.545 0.00267 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.06μs 8.79ns 34ns 0.0511 0 0 3.8 KB
#3064 EnrichedLog net472 5.51μs 32ns 260ns 0.545 0.00275 0 3.43 KB
#3064 EnrichedLog netcoreapp3.1 4.36μs 5.43ns 20.3ns 0.052 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.02μs 1.99ns 7.71ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.69μs 1.95ns 7.03ns 0.0161 0 0 1.21 KB
#3064 SendReceive net472 2.04μs 1.08ns 3.89ns 0.193 0 0 1.22 KB
#3064 SendReceive netcoreapp3.1 1.63μs 0.751ns 2.81ns 0.0164 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.65μs 2.73ns 10.2ns 0.329 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.3μs 1.29ns 4.65ns 0.0214 0 0 1.69 KB
#3064 EnrichedLog net472 4.6μs 1.73ns 6.71ns 0.33 0 0 2.08 KB
#3064 EnrichedLog netcoreapp3.1 4.08μs 1.46ns 5.48ns 0.0225 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 848ns 0.358ns 1.39ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 785ns 0.355ns 1.33ns 0.00863 0 0 648 B
master StartFinishScope net472 1.06μs 0.35ns 1.36ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 943ns 0.803ns 3.11ns 0.0104 0 0 768 B
#3064 StartFinishSpan net472 834ns 0.322ns 1.21ns 0.104 0 0 658 B
#3064 StartFinishSpan netcoreapp3.1 738ns 0.355ns 1.37ns 0.00885 0 0 648 B
#3064 StartFinishScope net472 1.05μs 0.27ns 1.04ns 0.117 0 0 738 B
#3064 StartFinishScope netcoreapp3.1 921ns 0.204ns 0.765ns 0.0107 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.25μs 0.571ns 2.21ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.01μs 0.244ns 0.914ns 0.0101 0 0 768 B
#3064 RunOnMethodBegin net472 1.24μs 0.351ns 1.36ns 0.117 0 0 738 B
#3064 RunOnMethodBegin netcoreapp3.1 967ns 0.34ns 1.27ns 0.0101 0 0 768 B

andrewlock avatar Aug 10 '22 15:08 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #3064 into master will not change line coverage :heavy_check_mark: Merging #3064 into master will not change branch coverage :no_entry: Merging #3064 into master will will increase complexity by 16

master #3064 Change
Lines 16807 / 22903 16879 / 22919
Lines % 73% 74% 0% :heavy_check_mark:
Branches 10017 / 14218 10078 / 14240
Branches % 70% 71% 0% :heavy_check_mark:
Complexity 15287 15303 16 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #3064 Change
Lines % 73% 74% 0% :heavy_check_mark:
Branches % 70% 71% 0% :heavy_check_mark:
Complexity 15287 15303 16 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Agent.Transports.HttpStreamRequestFactory -12% :no_entry: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Telemetry.Transports.JsonTelemetryTransport -8% :no_entry: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Pdb.DatadogPdbReader 1% :heavy_check_mark: 12% :heavy_check_mark: 2 :no_entry:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:

View the full reports for further details:

andrewlock avatar Aug 10 '22 16:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3064 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.142
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 726μs 281ns 1.09μs 0.363 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 480μs 259ns 1μs 0 0 0 2.58 KB
#3064 WriteAndFlushEnrichedTraces net472 728μs 412ns 1.59μs 0.37 0 0 3.18 KB
#3064 WriteAndFlushEnrichedTraces netcoreapp3.1 461μs 198ns 768ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 183ns 0.172ns 0.643ns 0.0675 9.23E-05 0 425 B
master AllCycleSimpleBody netcoreapp3.1 236ns 0.375ns 1.45ns 0.00589 0 0 424 B
master AllCycleMoreComplexBody net472 206ns 0.276ns 1.07ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 235ns 0.313ns 1.21ns 0.00538 0 0 400 B
master BodyExtractorSimpleBody net472 278ns 0.226ns 0.846ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 228ns 0.239ns 0.927ns 0.00369 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.8μs 6.97ns 27ns 1.21 0.0148 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.3μs 19.9ns 74.5ns 0.0915 0 0 6.75 KB
#3064 AllCycleSimpleBody net472 182ns 0.152ns 0.568ns 0.0675 0 0 425 B
#3064 AllCycleSimpleBody netcoreapp3.1 236ns 0.246ns 0.887ns 0.00579 0 0 424 B
#3064 AllCycleMoreComplexBody net472 185ns 0.151ns 0.586ns 0.0637 0 0 401 B
#3064 AllCycleMoreComplexBody netcoreapp3.1 235ns 0.225ns 0.843ns 0.00544 0 0 400 B
#3064 BodyExtractorSimpleBody net472 252ns 0.148ns 0.552ns 0.0574 0 0 361 B
#3064 BodyExtractorSimpleBody netcoreapp3.1 215ns 0.354ns 1.37ns 0.00365 0 0 272 B
#3064 BodyExtractorMoreComplexBody net472 14.4μs 15.8ns 56.9ns 1.2 0.0215 0 7.62 KB
#3064 BodyExtractorMoreComplexBody netcoreapp3.1 13μs 24ns 89.8ns 0.0914 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 178μs 113ns 423ns 0.266 0 0 20.33 KB
#3064 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3064 SendRequest netcoreapp3.1 180μs 116ns 447ns 0.179 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.57μs 0.547ns 2.12ns 0.126 0.000785 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.31μs 0.977ns 3.78ns 0.011 0 0 824 B
#3064 ExecuteNonQuery net472 1.52μs 0.36ns 1.39ns 0.126 0.000757 0 794 B
#3064 ExecuteNonQuery netcoreapp3.1 1.31μs 1.45ns 5.63ns 0.011 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.1μs 0.823ns 3.19ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.43μs 0.525ns 2.03ns 0.0135 0 0 984 B
master CallElasticsearchAsync net472 2.29μs 0.674ns 2.61ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.49μs 0.317ns 1.19ns 0.015 0 0 1.1 KB
#3064 CallElasticsearch net472 2.25μs 0.814ns 3.05ns 0.159 0 0 1 KB
#3064 CallElasticsearch netcoreapp3.1 1.36μs 1.42ns 5.51ns 0.0129 0 0 984 B
#3064 CallElasticsearchAsync net472 2.44μs 0.681ns 2.64ns 0.181 0 0 1.14 KB
#3064 CallElasticsearchAsync netcoreapp3.1 1.52μs 0.593ns 2.3ns 0.0144 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.43μs 4.02ns 15.6ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.7μs 15.8ns 148ns 0.0168 0 0 1.22 KB
#3064 ExecuteAsync net472 2.49μs 8.25ns 31.9ns 0.2 0 0 1.26 KB
#3064 ExecuteAsync netcoreapp3.1 1.6μs 4ns 15.5ns 0.0166 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.12μs 8.82ns 34.2ns 0.393 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.27μs 2.45ns 9.5ns 0.0325 0 0 2.36 KB
#3064 SendAsync net472 4.94μs 10.9ns 42.2ns 0.392 0 0 2.48 KB
#3064 SendAsync netcoreapp3.1 3.22μs 7.02ns 27.2ns 0.032 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #3064

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1 1.142 2,266.36 2,587.41

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.8μs 1.06ns 3.98ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.27μs 0.749ns 2.9ns 0.0238 0 0 1.73 KB
#3064 EnrichedLog net472 2.88μs 1.1ns 4.25ns 0.263 0 0 1.66 KB
#3064 EnrichedLog netcoreapp3.1 2.59μs 0.911ns 3.29ns 0.0233 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 148μs 84.6ns 316ns 0.67 0.223 0 4.5 KB
master EnrichedLog netcoreapp3.1 113μs 150ns 562ns 0 0 0 4.38 KB
#3064 EnrichedLog net472 148μs 78.4ns 293ns 0.672 0.224 0 4.5 KB
#3064 EnrichedLog netcoreapp3.1 115μs 113ns 437ns 0 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.42μs 6.06ns 22.7ns 0.546 0.0027 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.39μs 3.01ns 11.6ns 0.0523 0 0 3.8 KB
#3064 EnrichedLog net472 5.43μs 10.2ns 38.3ns 0.544 0.00272 0 3.43 KB
#3064 EnrichedLog netcoreapp3.1 4.21μs 6.05ns 23.4ns 0.0529 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.97μs 1.58ns 6.13ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.68μs 1.15ns 4.44ns 0.0161 0 0 1.21 KB
#3064 SendReceive net472 1.96μs 0.61ns 2.36ns 0.194 0 0 1.22 KB
#3064 SendReceive netcoreapp3.1 1.67μs 0.478ns 1.85ns 0.0167 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.78μs 0.954ns 3.69ns 0.33 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.19μs 1.64ns 6.37ns 0.0228 0 0 1.69 KB
#3064 EnrichedLog net472 4.89μs 1.49ns 5.77ns 0.33 0 0 2.08 KB
#3064 EnrichedLog netcoreapp3.1 4μs 1.26ns 4.54ns 0.0219 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 824ns 0.289ns 1.08ns 0.105 0 0 658 B
master StartFinishSpan netcoreapp3.1 765ns 0.197ns 0.763ns 0.0088 0 0 648 B
master StartFinishScope net472 1.09μs 0.287ns 1.11ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 947ns 0.337ns 1.26ns 0.0104 0 0 768 B
#3064 StartFinishSpan net472 824ns 0.803ns 3.11ns 0.105 0 0 658 B
#3064 StartFinishSpan netcoreapp3.1 756ns 0.6ns 2.25ns 0.00874 0 0 648 B
#3064 StartFinishScope net472 1.1μs 0.907ns 3.39ns 0.117 0 0 738 B
#3064 StartFinishScope netcoreapp3.1 892ns 0.479ns 1.86ns 0.0102 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.25μs 0.588ns 2.28ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 984ns 0.394ns 1.47ns 0.0103 0 0 768 B
#3064 RunOnMethodBegin net472 1.24μs 1.25ns 4.85ns 0.117 0 0 738 B
#3064 RunOnMethodBegin netcoreapp3.1 1.04μs 0.836ns 3.24ns 0.0105 0 0 768 B

andrewlock avatar Aug 14 '22 08:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3064 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.124
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 736μs 419ns 1.62μs 0.361 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 460μs 182ns 657ns 0 0 0 2.58 KB
#3064 WriteAndFlushEnrichedTraces net472 725μs 618ns 2.39μs 0.363 0 0 3.18 KB
#3064 WriteAndFlushEnrichedTraces netcoreapp3.1 466μs 149ns 559ns 0 0 0 2.59 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 182ns 0.203ns 0.786ns 0.0676 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 237ns 0.242ns 0.935ns 0.00582 0 0 424 B
master AllCycleMoreComplexBody net472 183ns 0.262ns 1.01ns 0.0638 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 236ns 0.169ns 0.633ns 0.00546 0 0 400 B
master BodyExtractorSimpleBody net472 252ns 0.236ns 0.914ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 227ns 0.356ns 1.33ns 0.00365 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.3μs 9.69ns 37.5ns 1.2 0.0215 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 14.2ns 53.3ns 0.0927 0 0 6.75 KB
#3064 AllCycleSimpleBody net472 182ns 0.148ns 0.571ns 0.0675 0 0 425 B
#3064 AllCycleSimpleBody netcoreapp3.1 235ns 0.337ns 1.3ns 0.00578 0 0 424 B
#3064 AllCycleMoreComplexBody net472 183ns 0.174ns 0.627ns 0.0637 0 0 401 B
#3064 AllCycleMoreComplexBody netcoreapp3.1 234ns 0.34ns 1.32ns 0.00538 0 0 400 B
#3064 BodyExtractorSimpleBody net472 257ns 0.201ns 0.777ns 0.0573 0 0 361 B
#3064 BodyExtractorSimpleBody netcoreapp3.1 217ns 0.201ns 0.779ns 0.00371 0 0 272 B
#3064 BodyExtractorMoreComplexBody net472 14.9μs 9.38ns 32.5ns 1.21 0.0149 0 7.62 KB
#3064 BodyExtractorMoreComplexBody netcoreapp3.1 12.5μs 9.17ns 33.1ns 0.087 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 179μs 126ns 488ns 0.267 0 0 20.33 KB
#3064 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3064 SendRequest netcoreapp3.1 179μs 102ns 394ns 0.178 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.56μs 0.464ns 1.74ns 0.126 0.000776 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.33μs 0.425ns 1.64ns 0.0106 0 0 824 B
#3064 ExecuteNonQuery net472 1.53μs 0.536ns 2.08ns 0.126 0.000763 0 794 B
#3064 ExecuteNonQuery netcoreapp3.1 1.32μs 0.449ns 1.68ns 0.0112 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.15μs 0.526ns 2.04ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.42μs 0.626ns 2.34ns 0.0135 0 0 984 B
master CallElasticsearchAsync net472 2.4μs 1.2ns 4.63ns 0.18 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.49μs 0.711ns 2.56ns 0.0148 0 0 1.1 KB
#3064 CallElasticsearch net472 2.08μs 0.599ns 2.24ns 0.159 0 0 1 KB
#3064 CallElasticsearch netcoreapp3.1 1.38μs 0.579ns 2.24ns 0.0131 0 0 984 B
#3064 CallElasticsearchAsync net472 2.38μs 0.767ns 2.87ns 0.181 0 0 1.14 KB
#3064 CallElasticsearchAsync netcoreapp3.1 1.46μs 0.925ns 3.46ns 0.0146 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.41μs 9.49ns 36.8ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.57μs 3.83ns 14.3ns 0.0158 0 0 1.22 KB
#3064 ExecuteAsync net472 2.47μs 2.34ns 9.07ns 0.2 0 0 1.26 KB
#3064 ExecuteAsync netcoreapp3.1 1.56μs 2.08ns 7.79ns 0.0169 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.09μs 16.5ns 61.8ns 0.394 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.17μs 8.34ns 32.3ns 0.0315 0 0 2.36 KB
#3064 SendAsync net472 4.98μs 5.51ns 21.3ns 0.392 0 0 2.48 KB
#3064 SendAsync netcoreapp3.1 3.24μs 4.39ns 16.4ns 0.0316 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.77μs 2.2ns 8.52ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.39μs 0.768ns 2.97ns 0.0227 0 0 1.73 KB
#3064 EnrichedLog net472 2.85μs 5.71ns 22.1ns 0.264 0 0 1.66 KB
#3064 EnrichedLog netcoreapp3.1 2.4μs 8.88ns 33.2ns 0.0234 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 147μs 104ns 375ns 0.67 0.223 0 4.5 KB
master EnrichedLog netcoreapp3.1 116μs 316ns 1.14μs 0.0579 0 0 4.38 KB
#3064 EnrichedLog net472 152μs 109ns 423ns 0.684 0.228 0 4.5 KB
#3064 EnrichedLog netcoreapp3.1 114μs 151ns 565ns 0.0567 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.3μs 11.2ns 43.5ns 0.545 0.00265 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.04μs 8.04ns 31.1ns 0.0505 0 0 3.8 KB
#3064 EnrichedLog net472 5.45μs 5.16ns 20ns 0.543 0.00274 0 3.43 KB
#3064 EnrichedLog netcoreapp3.1 4.38μs 4.24ns 16.4ns 0.05 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.99μs 1.28ns 4.97ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.65μs 4.47ns 16.7ns 0.0163 0 0 1.21 KB
#3064 SendReceive net472 2.08μs 1.54ns 5.98ns 0.193 0 0 1.22 KB
#3064 SendReceive netcoreapp3.1 1.64μs 0.75ns 2.91ns 0.0165 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.79μs 2.24ns 8.67ns 0.33 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.16μs 2.12ns 8.23ns 0.0228 0 0 1.69 KB
#3064 EnrichedLog net472 4.86μs 1.57ns 6.06ns 0.328 0 0 2.08 KB
#3064 EnrichedLog netcoreapp3.1 4.1μs 1.65ns 6.41ns 0.0226 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #3064

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 1.124 1,039.98 1,168.65

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 878ns 0.418ns 1.62ns 0.105 0 0 658 B
master StartFinishSpan netcoreapp3.1 751ns 0.304ns 1.14ns 0.00865 0 0 648 B
master StartFinishScope net472 1.04μs 0.545ns 2.04ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 862ns 0.397ns 1.49ns 0.0103 0 0 768 B
#3064 StartFinishSpan net472 886ns 0.608ns 2.36ns 0.104 0 0 658 B
#3064 StartFinishSpan netcoreapp3.1 781ns 0.337ns 1.3ns 0.00896 0 0 648 B
#3064 StartFinishScope net472 1.17μs 0.345ns 1.34ns 0.117 0 0 738 B
#3064 StartFinishScope netcoreapp3.1 911ns 0.312ns 1.17ns 0.0105 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.29μs 0.358ns 1.38ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.09μs 0.64ns 2.39ns 0.0103 0 0 768 B
#3064 RunOnMethodBegin net472 1.26μs 0.52ns 1.95ns 0.117 0 0 738 B
#3064 RunOnMethodBegin netcoreapp3.1 1.03μs 0.413ns 1.6ns 0.0103 0 0 768 B

andrewlock avatar Aug 16 '22 15:08 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #3064 into master will not change line coverage :heavy_check_mark: Merging #3064 into master will not change branch coverage :no_entry: Merging #3064 into master will will increase complexity by 12

master #3064 Change
Lines 16889 / 23226 16938 / 23240
Lines % 73% 73% 0% :heavy_check_mark:
Branches 10068 / 14354 10104 / 14366
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15449 15461 12 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #3064 Change
Lines % 73% 73% 0% :heavy_check_mark:
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15449 15461 12 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Pdb.DatadogPdbReader 1% :heavy_check_mark: 12% :heavy_check_mark: 2 :no_entry:
Datadog.Trace.Ci.CIVisibility 5% :heavy_check_mark: 6% :heavy_check_mark: 0 :heavy_check_mark:

View the full reports for further details:

andrewlock avatar Aug 16 '22 16:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3064 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 727μs 259ns 970ns 0.363 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 475μs 749ns 2.59μs 0 0 0 2.58 KB
#3064 WriteAndFlushEnrichedTraces net472 731μs 228ns 852ns 0.363 0 0 3.18 KB
#3064 WriteAndFlushEnrichedTraces netcoreapp3.1 470μs 174ns 650ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 181ns 0.224ns 0.839ns 0.0675 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 236ns 0.282ns 1.09ns 0.00588 0 0 424 B
master AllCycleMoreComplexBody net472 184ns 0.509ns 1.97ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 243ns 0.274ns 0.987ns 0.00541 0 0 400 B
master BodyExtractorSimpleBody net472 272ns 0.36ns 1.39ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 216ns 0.2ns 0.774ns 0.00367 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.3μs 14.3ns 55.3ns 1.21 0.0215 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 14.2ns 55.1ns 0.0904 0 0 6.75 KB
#3064 AllCycleSimpleBody net472 181ns 0.105ns 0.392ns 0.0676 9.16E-05 0 425 B
#3064 AllCycleSimpleBody netcoreapp3.1 234ns 0.218ns 0.844ns 0.00584 0 0 424 B
#3064 AllCycleMoreComplexBody net472 182ns 0.165ns 0.617ns 0.0637 0 0 401 B
#3064 AllCycleMoreComplexBody netcoreapp3.1 231ns 0.24ns 0.928ns 0.00546 0 0 400 B
#3064 BodyExtractorSimpleBody net472 255ns 0.206ns 0.799ns 0.0574 0 0 361 B
#3064 BodyExtractorSimpleBody netcoreapp3.1 216ns 0.315ns 1.22ns 0.00379 0 0 272 B
#3064 BodyExtractorMoreComplexBody net472 14.2μs 10.9ns 42.3ns 1.21 0.0213 0 7.62 KB
#3064 BodyExtractorMoreComplexBody netcoreapp3.1 12.6μs 59.9ns 224ns 0.0868 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 181μs 164ns 615ns 0.181 0 0 20.33 KB
#3064 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3064 SendRequest netcoreapp3.1 181μs 214ns 830ns 0.181 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.54μs 0.423ns 1.58ns 0.126 0.000773 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.32μs 0.395ns 1.53ns 0.0113 0 0 824 B
#3064 ExecuteNonQuery net472 1.48μs 0.434ns 1.62ns 0.126 0.000749 0 794 B
#3064 ExecuteNonQuery netcoreapp3.1 1.24μs 0.614ns 2.3ns 0.0113 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.11μs 0.614ns 2.38ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.4μs 2.43ns 9.4ns 0.0132 0 0 984 B
master CallElasticsearchAsync net472 2.43μs 1.2ns 4.63ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.41μs 0.52ns 2.01ns 0.0147 0 0 1.1 KB
#3064 CallElasticsearch net472 2.2μs 1.15ns 4.3ns 0.159 0 0 1 KB
#3064 CallElasticsearch netcoreapp3.1 1.39μs 1.03ns 3.86ns 0.0131 0 0 984 B
#3064 CallElasticsearchAsync net472 2.34μs 0.534ns 2.07ns 0.181 0 0 1.14 KB
#3064 CallElasticsearchAsync netcoreapp3.1 1.48μs 1.49ns 5.79ns 0.0148 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.37μs 9.46ns 36.6ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.48μs 1.42ns 4.93ns 0.0166 0 0 1.22 KB
#3064 ExecuteAsync net472 2.33μs 5.99ns 22.4ns 0.2 0 0 1.26 KB
#3064 ExecuteAsync netcoreapp3.1 1.55μs 3.83ns 14.9ns 0.0163 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.85μs 14.5ns 56.2ns 0.394 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.18μs 6.53ns 25.3ns 0.032 0 0 2.36 KB
#3064 SendAsync net472 4.9μs 7.62ns 28.5ns 0.394 0 0 2.48 KB
#3064 SendAsync netcoreapp3.1 3.23μs 9.57ns 37.1ns 0.0316 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.79μs 1.05ns 3.94ns 0.264 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.3μs 0.877ns 3.4ns 0.023 0 0 1.73 KB
#3064 EnrichedLog net472 2.88μs 1.25ns 4.85ns 0.264 0 0 1.66 KB
#3064 EnrichedLog netcoreapp3.1 2.35μs 0.845ns 3.16ns 0.0234 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 150μs 168ns 652ns 0.674 0.225 0 4.5 KB
master EnrichedLog netcoreapp3.1 116μs 155ns 599ns 0 0 0 4.38 KB
#3064 EnrichedLog net472 147μs 185ns 694ns 0.666 0.222 0 4.5 KB
#3064 EnrichedLog netcoreapp3.1 114μs 195ns 754ns 0.0574 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.28μs 9.7ns 36.3ns 0.545 0.00266 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.18μs 8.2ns 29.6ns 0.0519 0 0 3.8 KB
#3064 EnrichedLog net472 5.35μs 7.73ns 28.9ns 0.544 0.00271 0 3.43 KB
#3064 EnrichedLog netcoreapp3.1 4.24μs 10.7ns 41.5ns 0.0524 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.03μs 0.517ns 2ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.6μs 0.884ns 3.42ns 0.0161 0 0 1.21 KB
#3064 SendReceive net472 2μs 1.09ns 4.23ns 0.194 0 0 1.22 KB
#3064 SendReceive netcoreapp3.1 1.67μs 0.55ns 2.06ns 0.0159 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.73μs 1.48ns 5.73ns 0.328 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.08μs 2.71ns 10.5ns 0.0224 0 0 1.69 KB
#3064 EnrichedLog net472 4.73μs 1.01ns 3.79ns 0.329 0 0 2.08 KB
#3064 EnrichedLog netcoreapp3.1 4.09μs 1.22ns 4.56ns 0.0224 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 855ns 0.547ns 2.12ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 764ns 0.713ns 2.76ns 0.00874 0 0 648 B
master StartFinishScope net472 1.08μs 0.865ns 3.35ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 882ns 0.549ns 2.13ns 0.0101 0 0 768 B
#3064 StartFinishSpan net472 837ns 0.855ns 3.31ns 0.104 0 0 658 B
#3064 StartFinishSpan netcoreapp3.1 772ns 1.28ns 4.79ns 0.00891 0 0 648 B
#3064 StartFinishScope net472 1E+03ns 1.08ns 4.19ns 0.117 0 0 738 B
#3064 StartFinishScope netcoreapp3.1 868ns 0.823ns 3.08ns 0.0104 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.29μs 0.811ns 3.14ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.02μs 0.626ns 2.42ns 0.0103 0 0 768 B
#3064 RunOnMethodBegin net472 1.27μs 1.32ns 5.12ns 0.117 0 0 738 B
#3064 RunOnMethodBegin netcoreapp3.1 1.02μs 0.375ns 1.35ns 0.0107 0 0 768 B

andrewlock avatar Aug 17 '22 09:08 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #3064 into master will not change line coverage :heavy_check_mark: Merging #3064 into master will not change branch coverage :no_entry: Merging #3064 into master will will increase complexity by 12

master #3064 Change
Lines 16882 / 23226 16917 / 23240
Lines % 73% 73% 0% :heavy_check_mark:
Branches 10067 / 14354 10094 / 14366
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15449 15461 12 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #3064 Change
Lines % 73% 73% 0% :heavy_check_mark:
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15449 15461 12 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Pdb.DatadogPdbReader 1% :heavy_check_mark: 12% :heavy_check_mark: 2 :no_entry:
Datadog.Trace.Ci.CIVisibility 5% :heavy_check_mark: 6% :heavy_check_mark: 0 :heavy_check_mark:

View the full reports for further details:

andrewlock avatar Aug 17 '22 09:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3064 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 727μs 259ns 970ns 0.363 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 475μs 749ns 2.59μs 0 0 0 2.58 KB
#3064 WriteAndFlushEnrichedTraces net472 731μs 315ns 1.18μs 0.363 0 0 3.18 KB
#3064 WriteAndFlushEnrichedTraces netcoreapp3.1 457μs 152ns 570ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 181ns 0.224ns 0.839ns 0.0675 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 236ns 0.282ns 1.09ns 0.00588 0 0 424 B
master AllCycleMoreComplexBody net472 184ns 0.509ns 1.97ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 243ns 0.274ns 0.987ns 0.00541 0 0 400 B
master BodyExtractorSimpleBody net472 272ns 0.36ns 1.39ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 216ns 0.2ns 0.774ns 0.00367 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.3μs 14.3ns 55.3ns 1.21 0.0215 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 14.2ns 55.1ns 0.0904 0 0 6.75 KB
#3064 AllCycleSimpleBody net472 201ns 0.315ns 1.22ns 0.0675 0 0 425 B
#3064 AllCycleSimpleBody netcoreapp3.1 239ns 0.302ns 1.17ns 0.00574 0 0 424 B
#3064 AllCycleMoreComplexBody net472 187ns 0.869ns 3.36ns 0.0637 0 0 401 B
#3064 AllCycleMoreComplexBody netcoreapp3.1 236ns 0.317ns 1.23ns 0.0055 0 0 400 B
#3064 BodyExtractorSimpleBody net472 256ns 0.287ns 1.11ns 0.0574 0 0 361 B
#3064 BodyExtractorSimpleBody netcoreapp3.1 219ns 0.245ns 0.947ns 0.00371 0 0 272 B
#3064 BodyExtractorMoreComplexBody net472 14.9μs 6.82ns 24.6ns 1.21 0.0149 0 7.62 KB
#3064 BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 9.95ns 35.9ns 0.0911 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 181μs 164ns 615ns 0.181 0 0 20.33 KB
#3064 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3064 SendRequest netcoreapp3.1 180μs 164ns 634ns 0.27 0 0 20.32 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.54μs 0.423ns 1.58ns 0.126 0.000773 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.32μs 0.395ns 1.53ns 0.0113 0 0 824 B
#3064 ExecuteNonQuery net472 1.57μs 0.485ns 1.82ns 0.126 0.000785 0 794 B
#3064 ExecuteNonQuery netcoreapp3.1 1.3μs 0.371ns 1.34ns 0.0112 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.11μs 0.614ns 2.38ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.4μs 2.43ns 9.4ns 0.0132 0 0 984 B
master CallElasticsearchAsync net472 2.43μs 1.2ns 4.63ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.41μs 0.52ns 2.01ns 0.0147 0 0 1.1 KB
#3064 CallElasticsearch net472 2.23μs 0.668ns 2.59ns 0.158 0 0 1 KB
#3064 CallElasticsearch netcoreapp3.1 1.41μs 0.457ns 1.77ns 0.0127 0 0 984 B
#3064 CallElasticsearchAsync net472 2.32μs 0.984ns 3.81ns 0.18 0 0 1.14 KB
#3064 CallElasticsearchAsync netcoreapp3.1 1.45μs 0.431ns 1.67ns 0.0153 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.37μs 9.46ns 36.6ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.48μs 1.42ns 4.93ns 0.0166 0 0 1.22 KB
#3064 ExecuteAsync net472 2.41μs 5.49ns 21.3ns 0.199 0 0 1.26 KB
#3064 ExecuteAsync netcoreapp3.1 1.53μs 2.92ns 11.3ns 0.016 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.85μs 14.5ns 56.2ns 0.394 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.18μs 6.53ns 25.3ns 0.032 0 0 2.36 KB
#3064 SendAsync net472 4.97μs 6.86ns 26.6ns 0.393 0 0 2.48 KB
#3064 SendAsync netcoreapp3.1 3.2μs 3.97ns 14.9ns 0.032 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.79μs 1.05ns 3.94ns 0.264 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.3μs 0.877ns 3.4ns 0.023 0 0 1.73 KB
#3064 EnrichedLog net472 2.78μs 3.41ns 12.3ns 0.263 0 0 1.66 KB
#3064 EnrichedLog netcoreapp3.1 2.51μs 1.21ns 4.54ns 0.0226 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 150μs 168ns 652ns 0.674 0.225 0 4.5 KB
master EnrichedLog netcoreapp3.1 116μs 155ns 599ns 0 0 0 4.38 KB
#3064 EnrichedLog net472 151μs 138ns 534ns 0.683 0.228 0 4.5 KB
#3064 EnrichedLog netcoreapp3.1 116μs 220ns 823ns 0.0573 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.28μs 9.7ns 36.3ns 0.545 0.00266 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.18μs 8.2ns 29.6ns 0.0519 0 0 3.8 KB
#3064 EnrichedLog net472 5.59μs 13.9ns 53.8ns 0.544 0.00279 0 3.43 KB
#3064 EnrichedLog netcoreapp3.1 4.15μs 6.73ns 26.1ns 0.053 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.03μs 0.517ns 2ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.6μs 0.884ns 3.42ns 0.0161 0 0 1.21 KB
#3064 SendReceive net472 2.04μs 3.96ns 15.3ns 0.193 0 0 1.22 KB
#3064 SendReceive netcoreapp3.1 1.65μs 0.585ns 2.11ns 0.0158 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.73μs 1.48ns 5.73ns 0.328 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.08μs 2.71ns 10.5ns 0.0224 0 0 1.69 KB
#3064 EnrichedLog net472 4.61μs 0.931ns 3.36ns 0.329 0 0 2.08 KB
#3064 EnrichedLog netcoreapp3.1 4.06μs 2.39ns 8.93ns 0.0223 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 855ns 0.547ns 2.12ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 764ns 0.713ns 2.76ns 0.00874 0 0 648 B
master StartFinishScope net472 1.08μs 0.865ns 3.35ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 882ns 0.549ns 2.13ns 0.0101 0 0 768 B
#3064 StartFinishSpan net472 819ns 0.492ns 1.91ns 0.104 0 0 658 B
#3064 StartFinishSpan netcoreapp3.1 753ns 0.351ns 1.36ns 0.00897 0 0 648 B
#3064 StartFinishScope net472 1.09μs 1.1ns 4.25ns 0.117 0 0 738 B
#3064 StartFinishScope netcoreapp3.1 866ns 0.422ns 1.58ns 0.0105 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.29μs 0.811ns 3.14ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.02μs 0.626ns 2.42ns 0.0103 0 0 768 B
#3064 RunOnMethodBegin net472 1.24μs 0.665ns 2.49ns 0.117 0 0 738 B
#3064 RunOnMethodBegin netcoreapp3.1 1.08μs 0.395ns 1.53ns 0.0104 0 0 768 B

andrewlock avatar Aug 17 '22 13:08 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #3064 into master will not change line coverage :heavy_check_mark: Merging #3064 into master will not change branch coverage :no_entry: Merging #3064 into master will will increase complexity by 12

master #3064 Change
Lines 16882 / 23226 16899 / 23226
Lines % 73% 73% 0% :heavy_check_mark:
Branches 10067 / 14354 10088 / 14366
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15449 15461 12 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #3064 Change
Lines % 73% 73% 0% :heavy_check_mark:
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15449 15461 12 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Debugger.CapturedContext -12% :no_entry: -17% :no_entry: 2 :no_entry:
Datadog.Trace.Debugger.CapturedValue -6% :no_entry: 7% :heavy_check_mark: -4 :heavy_check_mark:
Datadog.Trace.Pdb.DatadogPdbReader 1% :heavy_check_mark: 12% :heavy_check_mark: 2 :no_entry:
Datadog.Trace.Ci.CIVisibility 5% :heavy_check_mark: 6% :heavy_check_mark: 0 :heavy_check_mark:

The following classes were added in #3064:

File Line coverage Branch coverage Complexity
Datadog.Trace.Debugger.SnapshotSerializationHelpers 88% 50% 4

View the full reports for further details:

andrewlock avatar Aug 17 '22 13:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3064 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 725μs 795ns 3.08μs 0.365 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 462μs 753ns 2.92μs 0 0 0 2.58 KB
#3064 WriteAndFlushEnrichedTraces net472 720μs 1.13μs 4.37μs 0.363 0 0 3.18 KB
#3064 WriteAndFlushEnrichedTraces netcoreapp3.1 462μs 679ns 2.63μs 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 183ns 0.181ns 0.676ns 0.0675 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 243ns 0.288ns 1.12ns 0.0058 0 0 424 B
master AllCycleMoreComplexBody net472 183ns 0.234ns 0.905ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 235ns 0.258ns 0.932ns 0.00553 0 0 400 B
master BodyExtractorSimpleBody net472 256ns 0.352ns 1.32ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 215ns 0.244ns 0.944ns 0.00357 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.7μs 14.2ns 54.8ns 1.21 0.0147 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.1μs 12ns 46.6ns 0.0908 0 0 6.75 KB
#3064 AllCycleSimpleBody net472 182ns 0.153ns 0.592ns 0.0675 9.16E-05 0 425 B
#3064 AllCycleSimpleBody netcoreapp3.1 236ns 0.412ns 1.6ns 0.00574 0 0 424 B
#3064 AllCycleMoreComplexBody net472 183ns 0.186ns 0.697ns 0.0638 0 0 401 B
#3064 AllCycleMoreComplexBody netcoreapp3.1 236ns 0.191ns 0.689ns 0.00549 0 0 400 B
#3064 BodyExtractorSimpleBody net472 276ns 0.256ns 0.991ns 0.0573 0 0 361 B
#3064 BodyExtractorSimpleBody netcoreapp3.1 221ns 0.273ns 1.06ns 0.00368 0 0 272 B
#3064 BodyExtractorMoreComplexBody net472 14.3μs 10.9ns 40.7ns 1.2 0.0214 0 7.62 KB
#3064 BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 16.4ns 61.5ns 0.0911 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0.00378ns 0.00206ns 0.00797ns 0 0 0 0 b
master SendRequest netcoreapp3.1 178μs 287ns 1.11μs 0.178 0 0 20.33 KB
#3064 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3064 SendRequest netcoreapp3.1 177μs 214ns 830ns 0.262 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.52μs 2.25ns 8.73ns 0.126 0.000755 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.19μs 1.15ns 4.46ns 0.0111 0 0 824 B
#3064 ExecuteNonQuery net472 1.56μs 1.81ns 6.76ns 0.126 0.000781 0 794 B
#3064 ExecuteNonQuery netcoreapp3.1 1.29μs 1.32ns 5.13ns 0.0109 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.19μs 0.668ns 2.59ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.31μs 0.862ns 3.22ns 0.0136 0 0 984 B
master CallElasticsearchAsync net472 2.35μs 0.877ns 3.39ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.44μs 0.815ns 3.16ns 0.015 0 0 1.1 KB
#3064 CallElasticsearch net472 2.24μs 0.888ns 3.32ns 0.159 0 0 1 KB
#3064 CallElasticsearch netcoreapp3.1 1.43μs 0.836ns 3.24ns 0.0129 0 0 984 B
#3064 CallElasticsearchAsync net472 2.28μs 0.657ns 2.54ns 0.181 0 0 1.14 KB
#3064 CallElasticsearchAsync netcoreapp3.1 1.48μs 0.528ns 2.05ns 0.0148 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.37μs 8.45ns 32.7ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.55μs 3.56ns 13.8ns 0.0166 0 0 1.22 KB
#3064 ExecuteAsync net472 2.42μs 3.93ns 15.2ns 0.2 0 0 1.26 KB
#3064 ExecuteAsync netcoreapp3.1 1.56μs 2.67ns 10.3ns 0.017 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.01μs 11.7ns 45.4ns 0.394 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.32μs 8.16ns 30.5ns 0.0324 0 0 2.36 KB
#3064 SendAsync net472 4.96μs 8.23ns 31.9ns 0.394 0 0 2.48 KB
#3064 SendAsync netcoreapp3.1 3.2μs 9.92ns 38.4ns 0.0325 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.94μs 4.39ns 17ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.45μs 0.898ns 3.36ns 0.0232 0 0 1.73 KB
#3064 EnrichedLog net472 3.01μs 3.49ns 13.5ns 0.263 0 0 1.66 KB
#3064 EnrichedLog netcoreapp3.1 2.41μs 0.647ns 2.33ns 0.0242 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 150μs 113ns 436ns 0.678 0.226 0 4.5 KB
master EnrichedLog netcoreapp3.1 113μs 137ns 531ns 0.0572 0 0 4.38 KB
#3064 EnrichedLog net472 151μs 230ns 890ns 0.681 0.227 0 4.5 KB
#3064 EnrichedLog netcoreapp3.1 114μs 159ns 615ns 0.057 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.46μs 17.6ns 68.1ns 0.543 0.00269 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.25μs 9.83ns 38.1ns 0.0527 0 0 3.8 KB
#3064 EnrichedLog net472 5.27μs 19.6ns 75.8ns 0.543 0.0026 0 3.43 KB
#3064 EnrichedLog netcoreapp3.1 4.18μs 4.32ns 16.7ns 0.0522 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2μs 0.971ns 3.76ns 0.194 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.66μs 0.624ns 2.33ns 0.0166 0 0 1.21 KB
#3064 SendReceive net472 2.06μs 1.43ns 5.55ns 0.193 0 0 1.22 KB
#3064 SendReceive netcoreapp3.1 1.65μs 6.21ns 24ns 0.0164 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.85μs 1.95ns 7.3ns 0.33 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.19μs 2ns 7.74ns 0.0229 0 0 1.69 KB
#3064 EnrichedLog net472 4.84μs 1.02ns 3.96ns 0.329 0 0 2.08 KB
#3064 EnrichedLog netcoreapp3.1 4.14μs 3.07ns 11.5ns 0.0226 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 859ns 0.611ns 2.36ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 779ns 0.29ns 1.08ns 0.00852 0 0 648 B
master StartFinishScope net472 1.07μs 0.815ns 3.16ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 850ns 0.687ns 2.66ns 0.0102 0 0 768 B
#3064 StartFinishSpan net472 867ns 0.572ns 2.22ns 0.104 0 0 658 B
#3064 StartFinishSpan netcoreapp3.1 760ns 0.253ns 0.979ns 0.00898 0 0 648 B
#3064 StartFinishScope net472 1.04μs 0.823ns 3.19ns 0.117 0 0 738 B
#3064 StartFinishScope netcoreapp3.1 907ns 0.44ns 1.65ns 0.0105 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.25μs 0.767ns 2.97ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.03μs 0.665ns 2.49ns 0.0103 0 0 768 B
#3064 RunOnMethodBegin net472 1.25μs 0.43ns 1.67ns 0.117 0 0 738 B
#3064 RunOnMethodBegin netcoreapp3.1 1.04μs 0.319ns 1.19ns 0.0105 0 0 768 B

andrewlock avatar Aug 18 '22 14:08 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #3064 into master will not change line coverage :heavy_check_mark: Merging #3064 into master will not change branch coverage :no_entry: Merging #3064 into master will will increase complexity by 12

master #3064 Change
Lines 16890 / 23241 16949 / 23241
Lines % 73% 73% 0% :heavy_check_mark:
Branches 10080 / 14356 10126 / 14368
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15452 15464 12 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #3064 Change
Lines % 73% 73% 0% :heavy_check_mark:
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15452 15464 12 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Debugger.CapturedContext -12% :no_entry: -17% :no_entry: 2 :no_entry:
Datadog.Trace.Debugger.CapturedValue -6% :no_entry: 7% :heavy_check_mark: -4 :heavy_check_mark:
Datadog.Trace.Pdb.DatadogPdbReader 1% :heavy_check_mark: 12% :heavy_check_mark: 2 :no_entry:
Datadog.Trace.Ci.CIVisibility 5% :heavy_check_mark: 6% :heavy_check_mark: 0 :heavy_check_mark:

The following classes were added in #3064:

File Line coverage Branch coverage Complexity
Datadog.Trace.Debugger.SnapshotSerializationHelpers 88% 50% 4

View the full reports for further details:

andrewlock avatar Aug 18 '22 15:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3064 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.271
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 739μs 265ns 991ns 0.37 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 456μs 144ns 540ns 0 0 0 2.58 KB
#3064 WriteAndFlushEnrichedTraces net472 726μs 308ns 1.19μs 0.361 0 0 3.18 KB
#3064 WriteAndFlushEnrichedTraces netcoreapp3.1 462μs 203ns 758ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #3064

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net472 1.271 180.28 229.09

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 182ns 0.145ns 0.524ns 0.0675 9.2E-05 0 425 B
master AllCycleSimpleBody netcoreapp3.1 236ns 0.212ns 0.792ns 0.00577 0 0 424 B
master AllCycleMoreComplexBody net472 182ns 0.791ns 3.06ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 237ns 0.264ns 1.02ns 0.00545 0 0 400 B
master BodyExtractorSimpleBody net472 261ns 0.238ns 0.92ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 218ns 0.294ns 1.14ns 0.00373 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.4μs 7.14ns 26.7ns 1.21 0.0215 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.4μs 14.3ns 53.6ns 0.0928 0 0 6.75 KB
#3064 AllCycleSimpleBody net472 187ns 0.575ns 2.23ns 0.0676 0 0 425 B
#3064 AllCycleSimpleBody netcoreapp3.1 234ns 0.441ns 1.71ns 0.00573 0 0 424 B
#3064 AllCycleMoreComplexBody net472 229ns 0.2ns 0.776ns 0.0637 0 0 401 B
#3064 AllCycleMoreComplexBody netcoreapp3.1 238ns 0.204ns 0.791ns 0.00538 0 0 400 B
#3064 BodyExtractorSimpleBody net472 254ns 0.245ns 0.947ns 0.0573 0 0 361 B
#3064 BodyExtractorSimpleBody netcoreapp3.1 232ns 0.34ns 1.32ns 0.00374 0 0 272 B
#3064 BodyExtractorMoreComplexBody net472 14.2μs 8.58ns 33.2ns 1.21 0.0216 0 7.62 KB
#3064 BodyExtractorMoreComplexBody netcoreapp3.1 12.1μs 11.4ns 44.2ns 0.0909 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 181μs 208ns 777ns 0.181 0 0 20.33 KB
#3064 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3064 SendRequest netcoreapp3.1 180μs 116ns 448ns 0.27 0 0 20.34 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.52μs 0.725ns 2.71ns 0.126 0.000742 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.32μs 0.33ns 1.24ns 0.0112 0 0 824 B
#3064 ExecuteNonQuery net472 1.51μs 0.517ns 2ns 0.126 0.000756 0 794 B
#3064 ExecuteNonQuery netcoreapp3.1 1.26μs 0.277ns 1.07ns 0.0108 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.23μs 0.532ns 1.92ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.38μs 0.794ns 2.86ns 0.0131 0 0 984 B
master CallElasticsearchAsync net472 2.4μs 1.08ns 4.19ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.48μs 0.549ns 2.13ns 0.0148 0 0 1.1 KB
#3064 CallElasticsearch net472 2.32μs 0.467ns 1.68ns 0.159 0 0 1 KB
#3064 CallElasticsearch netcoreapp3.1 1.4μs 0.465ns 1.8ns 0.0132 0 0 984 B
#3064 CallElasticsearchAsync net472 2.27μs 1.34ns 5.18ns 0.18 0 0 1.14 KB
#3064 CallElasticsearchAsync netcoreapp3.1 1.44μs 0.426ns 1.54ns 0.0153 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.3μs 7.29ns 28.2ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.52μs 4.8ns 18.6ns 0.0166 0 0 1.22 KB
#3064 ExecuteAsync net472 2.34μs 6.77ns 26.2ns 0.2 0 0 1.26 KB
#3064 ExecuteAsync netcoreapp3.1 1.58μs 5.45ns 21.1ns 0.0167 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.91μs 15.2ns 58.8ns 0.392 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.15μs 3.97ns 14.8ns 0.0319 0 0 2.36 KB
#3064 SendAsync net472 4.9μs 15.2ns 59ns 0.393 0 0 2.48 KB
#3064 SendAsync netcoreapp3.1 3.25μs 9.04ns 35ns 0.0321 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.75μs 0.677ns 2.62ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.37μs 0.565ns 2.11ns 0.0232 0 0 1.73 KB
#3064 EnrichedLog net472 2.87μs 1.41ns 5.48ns 0.264 0 0 1.66 KB
#3064 EnrichedLog netcoreapp3.1 2.34μs 0.712ns 2.66ns 0.0233 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 148μs 84.9ns 329ns 0.661 0.22 0 4.5 KB
master EnrichedLog netcoreapp3.1 115μs 270ns 1.05μs 0.0571 0 0 4.38 KB
#3064 EnrichedLog net472 148μs 90.1ns 349ns 0.668 0.223 0 4.5 KB
#3064 EnrichedLog netcoreapp3.1 115μs 284ns 1.1μs 0.0579 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.35μs 18ns 69.9ns 0.545 0.00265 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.09μs 9.07ns 33.9ns 0.0516 0 0 3.8 KB
#3064 EnrichedLog net472 5.43μs 14.2ns 54.8ns 0.545 0.0027 0 3.43 KB
#3064 EnrichedLog netcoreapp3.1 4.12μs 7.69ns 29.8ns 0.053 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.03μs 1.52ns 5.9ns 0.194 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.65μs 0.571ns 2.14ns 0.0165 0 0 1.21 KB
#3064 SendReceive net472 2.08μs 2.91ns 10.9ns 0.193 0 0 1.22 KB
#3064 SendReceive netcoreapp3.1 1.74μs 0.605ns 2.34ns 0.0165 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.7μs 1.29ns 5ns 0.329 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.1μs 0.798ns 2.99ns 0.0225 0 0 1.69 KB
#3064 EnrichedLog net472 4.73μs 1.63ns 6.11ns 0.329 0 0 2.08 KB
#3064 EnrichedLog netcoreapp3.1 4.13μs 1.57ns 6.08ns 0.0226 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 821ns 0.876ns 3.39ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 737ns 0.519ns 1.94ns 0.00882 0 0 648 B
master StartFinishScope net472 1.13μs 0.524ns 2.03ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 902ns 0.846ns 3.27ns 0.0104 0 0 768 B
#3064 StartFinishSpan net472 850ns 0.36ns 1.35ns 0.104 0 0 658 B
#3064 StartFinishSpan netcoreapp3.1 740ns 0.393ns 1.52ns 0.0085 0 0 648 B
#3064 StartFinishScope net472 1.07μs 0.344ns 1.33ns 0.117 0 0 738 B
#3064 StartFinishScope netcoreapp3.1 944ns 0.392ns 1.52ns 0.0104 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.22μs 1.28ns 4.78ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.04μs 0.376ns 1.41ns 0.0103 0 0 768 B
#3064 RunOnMethodBegin net472 1.28μs 0.474ns 1.84ns 0.117 0 0 738 B
#3064 RunOnMethodBegin netcoreapp3.1 1.01μs 0.978ns 3.79ns 0.0101 0 0 768 B

andrewlock avatar Aug 22 '22 12:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3064 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 732μs 393ns 1.52μs 0.361 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 466μs 152ns 569ns 0 0 0 2.58 KB
#3064 WriteAndFlushEnrichedTraces net472 721μs 278ns 1.08μs 0.361 0 0 3.18 KB
#3064 WriteAndFlushEnrichedTraces netcoreapp3.1 471μs 340ns 1.27μs 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 183ns 0.203ns 0.788ns 0.0675 9.17E-05 0 425 B
master AllCycleSimpleBody netcoreapp3.1 237ns 0.244ns 0.945ns 0.00583 0 0 424 B
master AllCycleMoreComplexBody net472 198ns 0.974ns 4.02ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 233ns 0.294ns 1.14ns 0.00551 0 0 400 B
master BodyExtractorSimpleBody net472 257ns 0.252ns 0.974ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 231ns 0.401ns 1.44ns 0.0037 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.6μs 16ns 59.9ns 1.21 0.0218 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 11.8μs 14.8ns 57.4ns 0.0883 0 0 6.75 KB
#3064 AllCycleSimpleBody net472 199ns 0.225ns 0.871ns 0.0675 0 0 425 B
#3064 AllCycleSimpleBody netcoreapp3.1 236ns 0.398ns 1.54ns 0.00577 0 0 424 B
#3064 AllCycleMoreComplexBody net472 200ns 0.676ns 2.62ns 0.0637 0 0 401 B
#3064 AllCycleMoreComplexBody netcoreapp3.1 234ns 0.246ns 0.919ns 0.00538 0 0 400 B
#3064 BodyExtractorSimpleBody net472 262ns 0.158ns 0.591ns 0.0573 0 0 361 B
#3064 BodyExtractorSimpleBody netcoreapp3.1 233ns 0.266ns 0.996ns 0.00374 0 0 272 B
#3064 BodyExtractorMoreComplexBody net472 14.3μs 10.4ns 38.9ns 1.21 0.0214 0 7.62 KB
#3064 BodyExtractorMoreComplexBody netcoreapp3.1 12μs 19.8ns 76.7ns 0.09 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 179μs 111ns 431ns 0.267 0 0 20.33 KB
#3064 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3064 SendRequest netcoreapp3.1 177μs 84ns 314ns 0.265 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.59μs 1.2ns 4.64ns 0.126 0.000794 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.24μs 0.678ns 2.54ns 0.0112 0 0 824 B
#3064 ExecuteNonQuery net472 1.6μs 0.586ns 2.19ns 0.126 0.000801 0 794 B
#3064 ExecuteNonQuery netcoreapp3.1 1.21μs 0.497ns 1.86ns 0.0109 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.23μs 0.764ns 2.96ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.41μs 0.971ns 3.76ns 0.0132 0 0 984 B
master CallElasticsearchAsync net472 2.3μs 0.515ns 2ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.46μs 1.29ns 4.84ns 0.0148 0 0 1.1 KB
#3064 CallElasticsearch net472 2.26μs 0.505ns 1.95ns 0.158 0 0 1 KB
#3064 CallElasticsearch netcoreapp3.1 1.39μs 0.771ns 2.89ns 0.0132 0 0 984 B
#3064 CallElasticsearchAsync net472 2.26μs 0.907ns 3.14ns 0.18 0 0 1.14 KB
#3064 CallElasticsearchAsync netcoreapp3.1 1.45μs 1.32ns 4.77ns 0.0148 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.3μs 6.78ns 26.3ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.55μs 3.47ns 13ns 0.0162 0 0 1.22 KB
#3064 ExecuteAsync net472 2.47μs 5.03ns 19.5ns 0.199 0 0 1.26 KB
#3064 ExecuteAsync netcoreapp3.1 1.56μs 3.42ns 12.3ns 0.0163 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.84μs 11.2ns 41.8ns 0.394 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.09μs 7.4ns 27.7ns 0.0325 0 0 2.36 KB
#3064 SendAsync net472 4.96μs 13.6ns 51ns 0.393 0 0 2.48 KB
#3064 SendAsync netcoreapp3.1 3.11μs 7.4ns 27.7ns 0.0322 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.86μs 0.665ns 2.49ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.38μs 1.09ns 4.07ns 0.0238 0 0 1.73 KB
#3064 EnrichedLog net472 2.73μs 0.352ns 1.27ns 0.264 0 0 1.66 KB
#3064 EnrichedLog netcoreapp3.1 2.4μs 1.38ns 5.17ns 0.024 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 148μs 258ns 999ns 0.668 0.223 0 4.5 KB
master EnrichedLog netcoreapp3.1 113μs 145ns 563ns 0 0 0 4.38 KB
#3064 EnrichedLog net472 146μs 91.4ns 354ns 0.662 0.221 0 4.5 KB
#3064 EnrichedLog netcoreapp3.1 113μs 231ns 895ns 0.0568 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.17μs 12.7ns 49.2ns 0.546 0.00256 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.27μs 7.04ns 26.3ns 0.0513 0 0 3.8 KB
#3064 EnrichedLog net472 5.24μs 19.3ns 74.8ns 0.544 0.00259 0 3.43 KB
#3064 EnrichedLog netcoreapp3.1 4.23μs 12.5ns 48.2ns 0.0519 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.04μs 0.789ns 3.06ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.64μs 0.878ns 3.17ns 0.0164 0 0 1.21 KB
#3064 SendReceive net472 1.99μs 0.662ns 2.56ns 0.194 0 0 1.22 KB
#3064 SendReceive netcoreapp3.1 1.72μs 1.17ns 4.52ns 0.0164 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.69μs 1.88ns 7.3ns 0.329 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.21μs 1.46ns 5.46ns 0.023 0 0 1.69 KB
#3064 EnrichedLog net472 4.75μs 1.1ns 3.97ns 0.329 0 0 2.08 KB
#3064 EnrichedLog netcoreapp3.1 4.16μs 1.39ns 5.18ns 0.0228 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 859ns 0.545ns 2.11ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 767ns 0.252ns 0.909ns 0.00892 0 0 648 B
master StartFinishScope net472 1.08μs 0.523ns 1.96ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 918ns 0.242ns 0.939ns 0.0105 0 0 768 B
#3064 StartFinishSpan net472 847ns 0.176ns 0.634ns 0.105 0 0 658 B
#3064 StartFinishSpan netcoreapp3.1 736ns 0.296ns 1.07ns 0.00883 0 0 648 B
#3064 StartFinishScope net472 1.13μs 0.397ns 1.54ns 0.117 0 0 738 B
#3064 StartFinishScope netcoreapp3.1 958ns 0.42ns 1.57ns 0.0105 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.21μs 0.459ns 1.78ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.07μs 0.22ns 0.762ns 0.0107 0 0 768 B
#3064 RunOnMethodBegin net472 1.26μs 1.03ns 3.84ns 0.117 0 0 738 B
#3064 RunOnMethodBegin netcoreapp3.1 1.02μs 0.473ns 1.77ns 0.0102 0 0 768 B

andrewlock avatar Aug 23 '22 09:08 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #3064 into master will not change line coverage :heavy_check_mark: Merging #3064 into master will not change branch coverage :no_entry: Merging #3064 into master will will increase complexity by 12

master #3064 Change
Lines 16940 / 23281 16918 / 23281
Lines % 73% 73% 0% :heavy_check_mark:
Branches 10106 / 14362 10108 / 14374
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15458 15470 12 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #3064 Change
Lines % 73% 73% 0% :heavy_check_mark:
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15458 15470 12 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Debugger.CapturedContext -12% :no_entry: -17% :no_entry: 2 :no_entry:
Datadog.Trace.Activity.DiagnosticSourceEventListener -6% :no_entry: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Debugger.CapturedValue -6% :no_entry: 7% :heavy_check_mark: -4 :heavy_check_mark:
Datadog.Trace.Pdb.DatadogPdbReader 1% :heavy_check_mark: 12% :heavy_check_mark: 2 :no_entry:
Datadog.Trace.Ci.CIVisibility 5% :heavy_check_mark: 6% :heavy_check_mark: 0 :heavy_check_mark:

The following classes were added in #3064:

File Line coverage Branch coverage Complexity
Datadog.Trace.Debugger.SnapshotSerializationHelpers 88% 50% 4

View the full reports for further details:

andrewlock avatar Aug 23 '22 10:08 andrewlock