dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard

[Tracer] Make timeout for trace submission configurable with `DD_TRACE_AGENT_TIMEOUT` and reduce default timeout from 100s to 10s

Open zacharycmontoya opened this issue 3 years ago • 15 comments

Summary of changes

  • Make the timeout for trace agent requests configurable via DD_TRACE_AGENT_TIMEOUT
  • Reduce timeout for trace agent requests from 100s (HttpClient default) to 10s
  • The timeout for named pipes is configured with the following order of precedence: DD_TRACE_PIPE_TIMEOUT_MS => DD_TRACE_AGENT_TIMEOUT => 500ms (same default as before)

Note: The default named pipe timeout is unchanged and remains at the previous value of 500ms

Reason for change

The previous behavior of a 100s timeout is quite large

Implementation details

Test coverage

Existing tests should all pass, and I've added several more to test the

Other details

zacharycmontoya avatar Jun 22 '22 23:06 zacharycmontoya

Benchmarks Report :snail:

Benchmarks for #2916 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 716μs 766ns 2.86μs 0.355 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 458μs 135ns 504ns 0 0 0 2.57 KB
#2916 WriteAndFlushEnrichedTraces net472 716μs 564ns 2.19μs 0.361 0 0 3.16 KB
#2916 WriteAndFlushEnrichedTraces netcoreapp3.1 467μs 458ns 1.77μs 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.83μs 2.18ns 8.15ns 0.236 0 0 1.48 KB
master AllCycleSimpleBody netcoreapp3.1 1.92μs 3.24ns 12.6ns 0.0182 0 0 1.36 KB
master AllCycleMoreComplexBody net472 18.2μs 14.2ns 53.2ns 1.38 0.0183 0 8.74 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.8μs 11.7ns 43.8ns 0.103 0 0 7.84 KB
master BodyExtractorSimpleBody net472 282ns 0.328ns 1.27ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 237ns 0.441ns 1.71ns 0.0037 0 0 272 B
master BodyExtractorMoreComplexBody net472 16.4μs 20.7ns 80.4ns 1.21 0.0164 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.7μs 8.03ns 29ns 0.0892 0 0 6.75 KB
#2916 AllCycleSimpleBody net472 1.91μs 2.85ns 11ns 0.235 0 0 1.48 KB
#2916 AllCycleSimpleBody netcoreapp3.1 2μs 3.79ns 13.7ns 0.0179 0 0 1.36 KB
#2916 AllCycleMoreComplexBody net472 18.3μs 19.5ns 75.6ns 1.38 0.0184 0 8.74 KB
#2916 AllCycleMoreComplexBody netcoreapp3.1 15.1μs 25.6ns 99.2ns 0.105 0 0 7.84 KB
#2916 BodyExtractorSimpleBody net472 286ns 0.562ns 2.18ns 0.0573 0 0 361 B
#2916 BodyExtractorSimpleBody netcoreapp3.1 241ns 0.473ns 1.83ns 0.0037 0 0 272 B
#2916 BodyExtractorMoreComplexBody net472 16.5μs 23.5ns 91.2ns 1.21 0.0163 0 7.62 KB
#2916 BodyExtractorMoreComplexBody netcoreapp3.1 12.5μs 20.4ns 78.9ns 0.0875 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 175μs 106ns 410ns 0.174 0 0 19.86 KB
#2916 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2916 SendRequest netcoreapp3.1 175μs 126ns 488ns 0.262 0 0 19.86 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.52μs 0.667ns 2.5ns 0.114 0.000757 0 722 B
master ExecuteNonQuery netcoreapp3.1 1.25μs 0.517ns 1.86ns 0.01 0 0 752 B
#2916 ExecuteNonQuery net472 1.47μs 0.54ns 2.09ns 0.115 0.00073 0 722 B
#2916 ExecuteNonQuery netcoreapp3.1 1.29μs 0.415ns 1.61ns 0.00967 0 0 752 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.16μs 0.6ns 2.32ns 0.148 0 0 931 B
master CallElasticsearch netcoreapp3.1 1.35μs 1.52ns 5.49ns 0.0127 0 0 912 B
master CallElasticsearchAsync net472 2.24μs 0.447ns 1.67ns 0.169 0 0 1.07 KB
master CallElasticsearchAsync netcoreapp3.1 1.42μs 1.64ns 6.36ns 0.0142 0 0 1.03 KB
#2916 CallElasticsearch net472 2.16μs 1.05ns 3.94ns 0.148 0 0 931 B
#2916 CallElasticsearch netcoreapp3.1 1.32μs 0.568ns 2.2ns 0.0127 0 0 912 B
#2916 CallElasticsearchAsync net472 2.37μs 0.77ns 2.98ns 0.17 0 0 1.07 KB
#2916 CallElasticsearchAsync netcoreapp3.1 1.49μs 0.569ns 2.05ns 0.0138 0 0 1.03 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.3μs 5.27ns 20.4ns 0.188 0 0 1.19 KB
master ExecuteAsync netcoreapp3.1 1.54μs 1.86ns 7.21ns 0.0154 0 0 1.15 KB
#2916 ExecuteAsync net472 2.34μs 1.45ns 5.62ns 0.189 0 0 1.19 KB
#2916 ExecuteAsync netcoreapp3.1 1.58μs 1.33ns 4.97ns 0.0151 0 0 1.15 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.88μs 6.2ns 24ns 0.382 0 0 2.41 KB
master SendAsync netcoreapp3.1 3.29μs 6.92ns 25.9ns 0.0311 0 0 2.33 KB
#2916 SendAsync net472 4.91μs 5.58ns 21.6ns 0.381 0 0 2.41 KB
#2916 SendAsync netcoreapp3.1 3.31μs 4.48ns 17.4ns 0.031 0 0 2.33 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.83μs 1.73ns 6.7ns 0.25 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 2.27μs 0.607ns 2.19ns 0.0227 0 0 1.65 KB
#2916 EnrichedLog net472 2.76μs 1.39ns 5.02ns 0.25 0 0 1.58 KB
#2916 EnrichedLog netcoreapp3.1 2.36μs 1.13ns 4.21ns 0.0224 0 0 1.65 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 145μs 129ns 500ns 0.653 0.218 0 4.42 KB
master EnrichedLog netcoreapp3.1 110μs 113ns 425ns 0.0543 0 0 4.3 KB
#2916 EnrichedLog net472 146μs 138ns 536ns 0.66 0.22 0 4.42 KB
#2916 EnrichedLog netcoreapp3.1 113μs 135ns 504ns 0 0 0 4.3 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.33μs 8.95ns 34.7ns 0.531 0.00263 0 3.35 KB
master EnrichedLog netcoreapp3.1 4.03μs 5.98ns 23.2ns 0.0498 0 0 3.72 KB
#2916 EnrichedLog net472 5.4μs 7.39ns 26.7ns 0.533 0.00269 0 3.35 KB
#2916 EnrichedLog netcoreapp3.1 4.07μs 12.3ns 47.8ns 0.0504 0 0 3.72 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.9μs 1.04ns 4.02ns 0.181 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.58μs 0.86ns 3.1ns 0.0151 0 0 1.13 KB
#2916 SendReceive net472 1.9μs 0.354ns 1.37ns 0.181 0 0 1.14 KB
#2916 SendReceive netcoreapp3.1 1.59μs 0.433ns 1.56ns 0.0151 0 0 1.13 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.72μs 1.67ns 6.46ns 0.317 0 0 2 KB
master EnrichedLog netcoreapp3.1 4.11μs 2.87ns 10.7ns 0.0206 0 0 1.61 KB
#2916 EnrichedLog net472 4.76μs 2.29ns 8.86ns 0.317 0 0 2 KB
#2916 EnrichedLog netcoreapp3.1 4.19μs 1.69ns 6.53ns 0.0207 0 0 1.61 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 800ns 0.292ns 1.13ns 0.0929 0 0 586 B
master StartFinishSpan netcoreapp3.1 722ns 0.256ns 0.99ns 0.0078 0 0 576 B
master StartFinishScope net472 1.18μs 0.516ns 2ns 0.106 0 0 666 B
master StartFinishScope netcoreapp3.1 883ns 0.412ns 1.48ns 0.00973 0 0 696 B
#2916 StartFinishSpan net472 862ns 0.464ns 1.8ns 0.093 0 0 586 B
#2916 StartFinishSpan netcoreapp3.1 738ns 0.375ns 1.45ns 0.00771 0 0 576 B
#2916 StartFinishScope net472 1.1μs 0.519ns 1.94ns 0.106 0 0 666 B
#2916 StartFinishScope netcoreapp3.1 833ns 0.192ns 0.693ns 0.00959 0 0 696 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.25μs 0.622ns 2.41ns 0.106 0 0 666 B
master RunOnMethodBegin netcoreapp3.1 931ns 0.248ns 0.961ns 0.0093 0 0 696 B
#2916 RunOnMethodBegin net472 1.24μs 0.449ns 1.74ns 0.105 0 0 666 B
#2916 RunOnMethodBegin netcoreapp3.1 986ns 0.325ns 1.21ns 0.00935 0 0 696 B

andrewlock avatar Jun 22 '22 23:06 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2916 into master will not change line coverage :heavy_check_mark: Merging #2916 into master will not change branch coverage :heavy_check_mark: Merging #2916 into master will not change complexity

master #2916 Change
Lines 14992 / 20437 15020 / 20438
Lines % 73% 73% 0% :heavy_check_mark:
Branches 9117 / 12924 9139 / 12924
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 13793 13793 0 :heavy_check_mark:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2916 Change
Lines % 73% 73% 0% :heavy_check_mark:
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 13793 13793 0 :heavy_check_mark:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Telemetry.Transports.JsonTelemetryTransport 0% :heavy_check_mark: 8% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:

View the full reports for further details:

andrewlock avatar Jun 22 '22 23:06 andrewlock

Hmm, I believed Andrew had changed it when he set it for telemetry and direct logs submission. We had discussed these timeouts over slack as well but don't recall at all the output. So I'm not sure he left this one aside for a particular reason or not.

Sat in my TODO list for about 3 months 😬

  • Fix the stupid timeouts in Trace httpclient

andrewlock avatar Jun 27 '22 08:06 andrewlock

Benchmarks Report :snail:

Benchmarks for #2916 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 718μs 376ns 1.46μs 0.355 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 464μs 113ns 423ns 0 0 0 2.58 KB
#2916 WriteAndFlushEnrichedTraces net472 717μs 271ns 977ns 0.357 0 0 3.18 KB
#2916 WriteAndFlushEnrichedTraces netcoreapp3.1 481μs 326ns 1.22μs 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.66μs 1.67ns 6.48ns 0.237 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.76μs 2.53ns 9.81ns 0.0184 0 0 1.37 KB
master AllCycleMoreComplexBody net472 17.3μs 13.1ns 49ns 1.39 0.0249 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 13.8μs 15.7ns 60.8ns 0.104 0 0 7.85 KB
master BodyExtractorSimpleBody net472 273ns 0.378ns 1.46ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 231ns 0.378ns 1.46ns 0.00371 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.1μs 9.4ns 36.4ns 1.2 0.0151 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 11.8μs 17.5ns 65.5ns 0.0885 0 0 6.75 KB
#2916 AllCycleSimpleBody net472 1.73μs 1.18ns 4.57ns 0.237 0 0 1.49 KB
#2916 AllCycleSimpleBody netcoreapp3.1 1.8μs 1.99ns 7.46ns 0.0189 0 0 1.37 KB
#2916 AllCycleMoreComplexBody net472 16.6μs 10.4ns 38.9ns 1.39 0.0171 0 8.75 KB
#2916 AllCycleMoreComplexBody netcoreapp3.1 13.7μs 13ns 48.7ns 0.103 0 0 7.85 KB
#2916 BodyExtractorSimpleBody net472 282ns 0.27ns 1.01ns 0.0574 0 0 361 B
#2916 BodyExtractorSimpleBody netcoreapp3.1 236ns 0.255ns 0.955ns 0.0037 0 0 272 B
#2916 BodyExtractorMoreComplexBody net472 14.6μs 13.4ns 50.2ns 1.2 0.0218 0 7.62 KB
#2916 BodyExtractorMoreComplexBody netcoreapp3.1 11.6μs 17.8ns 66.6ns 0.0929 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 175μs 150ns 580ns 0.262 0 0 19.94 KB
#2916 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2916 SendRequest netcoreapp3.1 177μs 155ns 602ns 0.264 0 0 19.94 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.57μs 0.696ns 2.7ns 0.126 0.000789 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.23μs 0.691ns 2.68ns 0.0111 0 0 824 B
#2916 ExecuteNonQuery net472 1.56μs 0.654ns 2.53ns 0.126 0.000782 0 794 B
#2916 ExecuteNonQuery netcoreapp3.1 1.34μs 0.59ns 2.21ns 0.0113 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.2μs 1.83ns 6.84ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.44μs 1.97ns 7.65ns 0.0137 0 0 984 B
master CallElasticsearchAsync net472 2.36μs 1.07ns 4.14ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.45μs 0.827ns 3.2ns 0.0152 0 0 1.1 KB
#2916 CallElasticsearch net472 2.24μs 0.384ns 1.49ns 0.159 0 0 1 KB
#2916 CallElasticsearch netcoreapp3.1 1.37μs 1.56ns 5.85ns 0.0131 0 0 984 B
#2916 CallElasticsearchAsync net472 2.37μs 0.693ns 2.68ns 0.18 0 0 1.14 KB
#2916 CallElasticsearchAsync netcoreapp3.1 1.43μs 0.507ns 1.83ns 0.015 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.31μs 6.12ns 23.7ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.59μs 2.24ns 8.69ns 0.0165 0 0 1.22 KB
#2916 ExecuteAsync net472 2.41μs 4.1ns 15.9ns 0.2 0 0 1.26 KB
#2916 ExecuteAsync netcoreapp3.1 1.55μs 1.55ns 5.8ns 0.0162 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.93μs 9.88ns 38.3ns 0.393 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.38μs 4.86ns 17.5ns 0.0333 0 0 2.4 KB
#2916 SendAsync net472 5.04μs 8.26ns 32ns 0.393 0 0 2.48 KB
#2916 SendAsync netcoreapp3.1 3.41μs 4.56ns 17.7ns 0.0323 0 0 2.4 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.85μs 1.44ns 5.58ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.42μs 1.19ns 4.44ns 0.0229 0 0 1.73 KB
#2916 EnrichedLog net472 2.92μs 0.696ns 2.61ns 0.263 0 0 1.66 KB
#2916 EnrichedLog netcoreapp3.1 2.22μs 0.658ns 2.46ns 0.0231 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 144μs 99.5ns 385ns 0.646 0.215 0 4.5 KB
master EnrichedLog netcoreapp3.1 113μs 105ns 408ns 0.0567 0 0 4.38 KB
#2916 EnrichedLog net472 145μs 86.4ns 335ns 0.653 0.218 0 4.5 KB
#2916 EnrichedLog netcoreapp3.1 110μs 86.6ns 324ns 0.0548 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.27μs 7.31ns 28.3ns 0.544 0.0026 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.22μs 8.97ns 34.7ns 0.0527 0 0 3.8 KB
#2916 EnrichedLog net472 5.43μs 13.6ns 50.8ns 0.544 0.00266 0 3.43 KB
#2916 EnrichedLog netcoreapp3.1 4.41μs 9.44ns 36.5ns 0.0527 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.05μs 1.66ns 6.43ns 0.192 0 0 1.21 KB
master SendReceive netcoreapp3.1 1.66μs 0.736ns 2.75ns 0.0158 0 0 1.2 KB
#2916 SendReceive net472 2.01μs 0.772ns 2.99ns 0.193 0 0 1.21 KB
#2916 SendReceive netcoreapp3.1 1.62μs 0.504ns 1.95ns 0.0163 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.66μs 1.85ns 7.15ns 0.329 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.13μs 1.98ns 7.41ns 0.0227 0 0 1.69 KB
#2916 EnrichedLog net472 4.71μs 1.56ns 5.86ns 0.33 0 0 2.08 KB
#2916 EnrichedLog netcoreapp3.1 4.05μs 0.953ns 3.3ns 0.0222 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 816ns 0.347ns 1.34ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 799ns 0.243ns 0.908ns 0.00868 0 0 648 B
master StartFinishScope net472 1.06μs 0.302ns 1.17ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 900ns 0.297ns 1.07ns 0.0104 0 0 768 B
#2916 StartFinishSpan net472 880ns 0.362ns 1.36ns 0.104 0 0 658 B
#2916 StartFinishSpan netcoreapp3.1 753ns 0.31ns 1.12ns 0.00871 0 0 648 B
#2916 StartFinishScope net472 1.08μs 0.563ns 2.18ns 0.117 0 0 738 B
#2916 StartFinishScope netcoreapp3.1 919ns 0.179ns 0.619ns 0.0102 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.19μs 0.396ns 1.53ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.08μs 0.402ns 1.45ns 0.0103 0 0 768 B
#2916 RunOnMethodBegin net472 1.2μs 0.545ns 2.11ns 0.117 0 0 738 B
#2916 RunOnMethodBegin netcoreapp3.1 1μs 2.6ns 10.1ns 0.0104 0 0 768 B

andrewlock avatar Jul 26 '22 00:07 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2916 into master will not change line coverage :heavy_check_mark: Merging #2916 into master will not change branch coverage :no_entry: Merging #2916 into master will will increase complexity by 14

master #2916 Change
Lines 16597 / 22658 16615 / 22671
Lines % 73% 73% 0% :heavy_check_mark:
Branches 9822 / 13976 9849 / 13990
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15031 15045 14 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2916 Change
Lines % 73% 73% 0% :heavy_check_mark:
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 15031 15045 14 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.GitInfo -17% :no_entry: -11% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.Ci.Processors.OriginTagTraceProcessor -9% :no_entry: -17% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:

View the full reports for further details:

andrewlock avatar Jul 26 '22 00:07 andrewlock

Benchmarks Report :snail:

Benchmarks for #2916 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 713μs 573ns 2.22μs 0.355 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 462μs 118ns 442ns 0 0 0 2.58 KB
#2916 WriteAndFlushEnrichedTraces net472 721μs 340ns 1.27μs 0.357 0 0 3.18 KB
#2916 WriteAndFlushEnrichedTraces netcoreapp3.1 462μs 181ns 679ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.72μs 1.7ns 6.6ns 0.237 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.81μs 3.74ns 14ns 0.019 0 0 1.37 KB
master AllCycleMoreComplexBody net472 16.6μs 8.67ns 33.6ns 1.39 0.0249 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.1μs 18ns 69.7ns 0.106 0 0 7.85 KB
master BodyExtractorSimpleBody net472 255ns 0.207ns 0.8ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 225ns 0.326ns 1.22ns 0.00377 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.5μs 13.7ns 53.1ns 1.21 0.0153 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.3μs 14.8ns 55.5ns 0.0865 0 0 6.75 KB
#2916 AllCycleSimpleBody net472 1.71μs 2.78ns 10.8ns 0.237 0 0 1.49 KB
#2916 AllCycleSimpleBody netcoreapp3.1 1.76μs 3.83ns 13.3ns 0.0184 0 0 1.37 KB
#2916 AllCycleMoreComplexBody net472 16.6μs 7.83ns 29.3ns 1.39 0.0248 0 8.75 KB
#2916 AllCycleMoreComplexBody netcoreapp3.1 14.2μs 19.7ns 73.7ns 0.0989 0 0 7.85 KB
#2916 BodyExtractorSimpleBody net472 249ns 0.241ns 0.87ns 0.0573 0 0 361 B
#2916 BodyExtractorSimpleBody netcoreapp3.1 221ns 0.27ns 1.05ns 0.00365 0 0 272 B
#2916 BodyExtractorMoreComplexBody net472 14.9μs 11.8ns 44.3ns 1.2 0.0149 0 7.62 KB
#2916 BodyExtractorMoreComplexBody netcoreapp3.1 11.7μs 15.4ns 57.8ns 0.0931 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 178μs 130ns 503ns 0.267 0 0 20.37 KB
#2916 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2916 SendRequest netcoreapp3.1 180μs 84.4ns 327ns 0.18 0 0 20.37 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.57μs 0.55ns 2.13ns 0.126 0.00078 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.31μs 0.471ns 1.83ns 0.0111 0 0 824 B
#2916 ExecuteNonQuery net472 1.52μs 0.593ns 2.22ns 0.126 0.000765 0 794 B
#2916 ExecuteNonQuery netcoreapp3.1 1.28μs 0.486ns 1.88ns 0.0109 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.22μs 0.497ns 1.79ns 0.158 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.34μs 0.688ns 2.67ns 0.0134 0 0 984 B
master CallElasticsearchAsync net472 2.4μs 0.862ns 3.34ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.45μs 0.307ns 1.15ns 0.0152 0 0 1.1 KB
#2916 CallElasticsearch net472 2.17μs 0.705ns 2.64ns 0.159 0 0 1 KB
#2916 CallElasticsearch netcoreapp3.1 1.4μs 0.632ns 2.37ns 0.0134 0 0 984 B
#2916 CallElasticsearchAsync net472 2.42μs 0.812ns 3.14ns 0.181 0 0 1.14 KB
#2916 CallElasticsearchAsync netcoreapp3.1 1.49μs 0.792ns 2.74ns 0.0149 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.4μs 0.826ns 3.09ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.57μs 0.918ns 3.44ns 0.0168 0 0 1.22 KB
#2916 ExecuteAsync net472 2.4μs 6.3ns 24.4ns 0.2 0 0 1.26 KB
#2916 ExecuteAsync netcoreapp3.1 1.56μs 0.764ns 2.96ns 0.0165 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.03μs 1.29ns 4.98ns 0.392 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.45μs 1.14ns 4.41ns 0.0326 0 0 2.4 KB
#2916 SendAsync net472 5.09μs 3.57ns 13.8ns 0.393 0 0 2.48 KB
#2916 SendAsync netcoreapp3.1 3.44μs 1.6ns 6.21ns 0.0327 0 0 2.4 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.87μs 1.3ns 4.88ns 0.264 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.43μs 3.24ns 12.6ns 0.0228 0 0 1.73 KB
#2916 EnrichedLog net472 2.95μs 2.28ns 8.84ns 0.263 0 0 1.66 KB
#2916 EnrichedLog netcoreapp3.1 2.33μs 0.899ns 3.36ns 0.0234 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 146μs 216ns 838ns 0.654 0.218 0 4.5 KB
master EnrichedLog netcoreapp3.1 112μs 289ns 1.12μs 0.0561 0 0 4.38 KB
#2916 EnrichedLog net472 147μs 74.9ns 290ns 0.664 0.221 0 4.5 KB
#2916 EnrichedLog netcoreapp3.1 113μs 73ns 273ns 0 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.5μs 1.36ns 5.28ns 0.545 0.00272 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.35μs 1.77ns 6.84ns 0.05 0 0 3.8 KB
#2916 EnrichedLog net472 5.42μs 5.12ns 19.8ns 0.545 0.0027 0 3.43 KB
#2916 EnrichedLog netcoreapp3.1 4.18μs 1.96ns 7.07ns 0.0523 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.97μs 0.694ns 2.4ns 0.192 0 0 1.21 KB
master SendReceive netcoreapp3.1 1.64μs 0.875ns 3.27ns 0.0164 0 0 1.2 KB
#2916 SendReceive net472 1.94μs 1.21ns 4.53ns 0.192 0 0 1.21 KB
#2916 SendReceive netcoreapp3.1 1.69μs 0.713ns 2.67ns 0.0161 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.62μs 1.72ns 6.64ns 0.329 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.16μs 1.27ns 4.74ns 0.0229 0 0 1.69 KB
#2916 EnrichedLog net472 4.6μs 2.28ns 8.83ns 0.328 0 0 2.08 KB
#2916 EnrichedLog netcoreapp3.1 4.05μs 1.64ns 6.34ns 0.0223 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 878ns 0.245ns 0.949ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 765ns 0.22ns 0.824ns 0.00881 0 0 648 B
master StartFinishScope net472 1.06μs 0.327ns 1.27ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 893ns 0.329ns 1.27ns 0.0103 0 0 768 B
#2916 StartFinishSpan net472 840ns 0.249ns 0.963ns 0.104 0 0 658 B
#2916 StartFinishSpan netcoreapp3.1 743ns 0.199ns 0.769ns 0.0089 0 0 648 B
#2916 StartFinishScope net472 1.05μs 0.363ns 1.31ns 0.117 0 0 738 B
#2916 StartFinishScope netcoreapp3.1 872ns 0.244ns 0.945ns 0.0104 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.25μs 0.832ns 3.22ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 964ns 0.381ns 1.43ns 0.0101 0 0 768 B
#2916 RunOnMethodBegin net472 1.23μs 0.308ns 1.15ns 0.117 0 0 738 B
#2916 RunOnMethodBegin netcoreapp3.1 1.04μs 0.276ns 0.995ns 0.0104 0 0 768 B

andrewlock avatar Jul 26 '22 17:07 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2916 into master will not change line coverage :heavy_check_mark: Merging #2916 into master will not change branch coverage :no_entry: Merging #2916 into master will will increase complexity by 16

master #2916 Change
Lines 16742 / 22777 16736 / 22790
Lines % 74% 73% 0% :heavy_check_mark:
Branches 9986 / 14146 10011 / 14162
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 15199 15215 16 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2916 Change
Lines % 74% 73% 0% :heavy_check_mark:
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 15199 15215 16 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.GitInfo -19% :no_entry: -12% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Ci.Processors.OriginTagTraceProcessor 9% :heavy_check_mark: 17% :heavy_check_mark: 0 :heavy_check_mark:

View the full reports for further details:

andrewlock avatar Jul 26 '22 17:07 andrewlock

Benchmarks Report :snail:

Benchmarks for #2916 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.122
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 722μs 493ns 1.85μs 0.365 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 465μs 258ns 964ns 0 0 0 2.58 KB
#2916 WriteAndFlushEnrichedTraces net472 720μs 374ns 1.45μs 0.357 0 0 3.18 KB
#2916 WriteAndFlushEnrichedTraces netcoreapp3.1 463μs 313ns 1.17μs 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.69μs 1.4ns 5.44ns 0.237 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.78μs 2.2ns 8.52ns 0.0187 0 0 1.37 KB
master AllCycleMoreComplexBody net472 17.1μs 10.6ns 38.1ns 1.39 0.0253 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.3μs 16.6ns 62.3ns 0.108 0 0 7.85 KB
master BodyExtractorSimpleBody net472 268ns 0.364ns 1.36ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 225ns 0.304ns 1.18ns 0.00374 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.7μs 8.88ns 34.4ns 1.2 0.0147 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 11.8μs 36.9ns 138ns 0.0935 0 0 6.75 KB
#2916 AllCycleSimpleBody net472 1.68μs 1.17ns 4.38ns 0.237 0 0 1.49 KB
#2916 AllCycleSimpleBody netcoreapp3.1 1.85μs 2.63ns 9.85ns 0.0183 0 0 1.37 KB
#2916 AllCycleMoreComplexBody net472 16.6μs 13.6ns 50.8ns 1.39 0.0249 0 8.75 KB
#2916 AllCycleMoreComplexBody netcoreapp3.1 14μs 19.3ns 72.4ns 0.104 0 0 7.85 KB
#2916 BodyExtractorSimpleBody net472 253ns 0.346ns 1.29ns 0.0573 0 0 361 B
#2916 BodyExtractorSimpleBody netcoreapp3.1 222ns 0.235ns 0.88ns 0.0037 0 0 272 B
#2916 BodyExtractorMoreComplexBody net472 15.2μs 11.1ns 42.8ns 1.21 0.0152 0 7.62 KB
#2916 BodyExtractorMoreComplexBody netcoreapp3.1 11.7μs 19.2ns 71.8ns 0.0883 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 180μs 141ns 546ns 0.27 0 0 20.37 KB
#2916 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2916 SendRequest netcoreapp3.1 179μs 78.3ns 293ns 0.268 0 0 20.37 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.52μs 0.432ns 1.62ns 0.126 0.000768 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.29μs 0.57ns 2.21ns 0.011 0 0 824 B
#2916 ExecuteNonQuery net472 1.54μs 0.397ns 1.54ns 0.126 0.000772 0 794 B
#2916 ExecuteNonQuery netcoreapp3.1 1.28μs 0.267ns 0.964ns 0.0109 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.26μs 0.596ns 2.31ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.4μs 2.45ns 9.49ns 0.0135 0 0 984 B
master CallElasticsearchAsync net472 2.29μs 1.34ns 5.19ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.5μs 0.64ns 2.48ns 0.015 0 0 1.1 KB
#2916 CallElasticsearch net472 2.23μs 2.47ns 9.57ns 0.159 0 0 1 KB
#2916 CallElasticsearch netcoreapp3.1 1.43μs 1.21ns 4.51ns 0.0132 0 0 984 B
#2916 CallElasticsearchAsync net472 2.29μs 2.7ns 10.4ns 0.181 0 0 1.14 KB
#2916 CallElasticsearchAsync netcoreapp3.1 1.45μs 0.974ns 3.77ns 0.0154 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.41μs 2.96ns 11.5ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.6μs 1.07ns 4.14ns 0.0167 0 0 1.22 KB
#2916 ExecuteAsync net472 2.55μs 2.19ns 8.47ns 0.199 0 0 1.26 KB
#2916 ExecuteAsync netcoreapp3.1 1.6μs 1.41ns 5.27ns 0.0169 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.93μs 2.35ns 8.48ns 0.393 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.43μs 2.03ns 7.87ns 0.0323 0 0 2.4 KB
#2916 SendAsync net472 4.98μs 3.03ns 11.3ns 0.393 0 0 2.48 KB
#2916 SendAsync netcoreapp3.1 3.46μs 2.47ns 9.23ns 0.0327 0 0 2.4 KB
Benchmarks.Trace.ILoggerBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #2916

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1 1.122 2,490.76 2,220.27

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.93μs 0.689ns 2.58ns 0.264 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.49μs 1.01ns 3.93ns 0.0237 0 0 1.73 KB
#2916 EnrichedLog net472 2.8μs 2.21ns 8.55ns 0.263 0 0 1.66 KB
#2916 EnrichedLog netcoreapp3.1 2.22μs 1.22ns 4.72ns 0.0233 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 146μs 60.2ns 233ns 0.659 0.22 0 4.5 KB
master EnrichedLog netcoreapp3.1 112μs 86.6ns 336ns 0 0 0 4.38 KB
#2916 EnrichedLog net472 147μs 114ns 442ns 0.665 0.222 0 4.5 KB
#2916 EnrichedLog netcoreapp3.1 113μs 109ns 409ns 0.0563 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.48μs 3.97ns 14.9ns 0.546 0.00274 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.22μs 2.79ns 10.8ns 0.0512 0 0 3.8 KB
#2916 EnrichedLog net472 5.41μs 3.58ns 13.9ns 0.543 0.0027 0 3.43 KB
#2916 EnrichedLog netcoreapp3.1 4.17μs 6.83ns 25.6ns 0.0506 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.95μs 0.549ns 2.13ns 0.192 0 0 1.21 KB
master SendReceive netcoreapp3.1 1.65μs 0.707ns 2.65ns 0.0157 0 0 1.2 KB
#2916 SendReceive net472 1.94μs 0.958ns 3.71ns 0.193 0 0 1.21 KB
#2916 SendReceive netcoreapp3.1 1.65μs 0.743ns 2.78ns 0.0165 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.68μs 1.14ns 4.42ns 0.33 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.04μs 0.982ns 3.81ns 0.0222 0 0 1.69 KB
#2916 EnrichedLog net472 4.73μs 4.57ns 17.1ns 0.33 0 0 2.08 KB
#2916 EnrichedLog netcoreapp3.1 4.15μs 5.1ns 19.8ns 0.0225 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 873ns 0.194ns 0.751ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 736ns 0.22ns 0.763ns 0.00864 0 0 648 B
master StartFinishScope net472 1.05μs 0.42ns 1.57ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 835ns 0.326ns 1.26ns 0.0104 0 0 768 B
#2916 StartFinishSpan net472 884ns 0.201ns 0.726ns 0.104 0 0 658 B
#2916 StartFinishSpan netcoreapp3.1 814ns 0.284ns 1.06ns 0.00892 0 0 648 B
#2916 StartFinishScope net472 1.09μs 0.475ns 1.84ns 0.117 0 0 738 B
#2916 StartFinishScope netcoreapp3.1 922ns 0.295ns 1.1ns 0.0106 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.19μs 0.367ns 1.42ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 951ns 0.84ns 2.91ns 0.0106 0 0 768 B
#2916 RunOnMethodBegin net472 1.21μs 0.225ns 0.87ns 0.117 0 0 738 B
#2916 RunOnMethodBegin netcoreapp3.1 1.03μs 0.35ns 1.35ns 0.0103 0 0 768 B

andrewlock avatar Jul 27 '22 21:07 andrewlock

Benchmarks Report :snail:

Benchmarks for #2916 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 715μs 211ns 732ns 0.355 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 456μs 158ns 593ns 0 0 0 2.58 KB
#2916 WriteAndFlushEnrichedTraces net472 716μs 236ns 882ns 0.355 0 0 3.18 KB
#2916 WriteAndFlushEnrichedTraces netcoreapp3.1 458μs 156ns 584ns 0 0 0 2.59 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.7μs 2.96ns 10.7ns 0.237 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.85μs 2.68ns 10ns 0.0183 0 0 1.37 KB
master AllCycleMoreComplexBody net472 16.7μs 14.4ns 55.8ns 1.39 0.0251 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 14μs 16.9ns 65.3ns 0.105 0 0 7.85 KB
master BodyExtractorSimpleBody net472 252ns 0.658ns 2.55ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 225ns 0.315ns 1.18ns 0.0037 0 0 272 B
master BodyExtractorMoreComplexBody net472 15μs 12.8ns 47.9ns 1.2 0.0151 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 11.9μs 19.6ns 73.2ns 0.0881 0 0 6.75 KB
#2916 AllCycleSimpleBody net472 1.74μs 1.33ns 5.14ns 0.236 0 0 1.49 KB
#2916 AllCycleSimpleBody netcoreapp3.1 1.82μs 2.73ns 10.6ns 0.0189 0 0 1.37 KB
#2916 AllCycleMoreComplexBody net472 16.7μs 8.09ns 30.3ns 1.39 0.0251 0 8.75 KB
#2916 AllCycleMoreComplexBody netcoreapp3.1 14.1μs 18.8ns 70.2ns 0.101 0 0 7.85 KB
#2916 BodyExtractorSimpleBody net472 272ns 0.264ns 1.02ns 0.0573 0 0 361 B
#2916 BodyExtractorSimpleBody netcoreapp3.1 234ns 0.355ns 1.37ns 0.00376 0 0 272 B
#2916 BodyExtractorMoreComplexBody net472 15.1μs 11.3ns 42.3ns 1.2 0.0151 0 7.62 KB
#2916 BodyExtractorMoreComplexBody netcoreapp3.1 12μs 14.7ns 56.9ns 0.0905 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 176μs 149ns 576ns 0.263 0 0 20.37 KB
#2916 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2916 SendRequest netcoreapp3.1 178μs 163ns 632ns 0.179 0 0 20.37 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.52μs 0.324ns 1.21ns 0.126 0.000765 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.27μs 0.525ns 1.96ns 0.0108 0 0 824 B
#2916 ExecuteNonQuery net472 1.56μs 0.497ns 1.86ns 0.126 0.000778 0 794 B
#2916 ExecuteNonQuery netcoreapp3.1 1.24μs 2.34ns 8.45ns 0.0112 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.15μs 2.02ns 7.83ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.37μs 1.2ns 4.64ns 0.0129 0 0 984 B
master CallElasticsearchAsync net472 2.36μs 4ns 15.5ns 0.18 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.53μs 1.85ns 7.18ns 0.0153 0 0 1.1 KB
#2916 CallElasticsearch net472 2.24μs 1.36ns 5.28ns 0.158 0 0 1 KB
#2916 CallElasticsearch netcoreapp3.1 1.5μs 1.53ns 5.91ns 0.0134 0 0 984 B
#2916 CallElasticsearchAsync net472 2.35μs 2ns 7.75ns 0.18 0 0 1.14 KB
#2916 CallElasticsearchAsync netcoreapp3.1 1.43μs 1.09ns 4.09ns 0.0149 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.38μs 5.21ns 20.2ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.53μs 2.06ns 7.71ns 0.0162 0 0 1.22 KB
#2916 ExecuteAsync net472 2.33μs 3.27ns 12.7ns 0.2 0 0 1.26 KB
#2916 ExecuteAsync netcoreapp3.1 1.56μs 2.08ns 8.06ns 0.0163 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.05μs 7.03ns 27.2ns 0.393 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.47μs 5.49ns 21.3ns 0.033 0 0 2.4 KB
#2916 SendAsync net472 5.05μs 8.77ns 34ns 0.392 0 0 2.48 KB
#2916 SendAsync netcoreapp3.1 3.57μs 3.14ns 12.2ns 0.0316 0 0 2.4 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.91μs 1.81ns 7.01ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.3μs 1.03ns 3.73ns 0.023 0 0 1.73 KB
#2916 EnrichedLog net472 2.82μs 1.38ns 5.33ns 0.263 0 0 1.66 KB
#2916 EnrichedLog netcoreapp3.1 2.34μs 1.47ns 5.69ns 0.0231 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 146μs 90.3ns 350ns 0.662 0.221 0 4.5 KB
master EnrichedLog netcoreapp3.1 109μs 204ns 791ns 0.054 0 0 4.38 KB
#2916 EnrichedLog net472 148μs 74ns 287ns 0.66 0.22 0 4.5 KB
#2916 EnrichedLog netcoreapp3.1 111μs 192ns 745ns 0.0557 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.31μs 6.79ns 25.4ns 0.544 0.00263 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.41μs 8.39ns 32.5ns 0.0528 0 0 3.8 KB
#2916 EnrichedLog net472 5.4μs 17.1ns 66.3ns 0.545 0.00267 0 3.43 KB
#2916 EnrichedLog netcoreapp3.1 4.26μs 6.86ns 25.7ns 0.0511 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.93μs 1.25ns 4.84ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.66μs 0.945ns 3.54ns 0.0166 0 0 1.21 KB
#2916 SendReceive net472 2μs 1.62ns 6.28ns 0.193 0 0 1.22 KB
#2916 SendReceive netcoreapp3.1 1.62μs 0.87ns 3.26ns 0.0161 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.62μs 5.36ns 20.8ns 0.33 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.01μs 4.23ns 16.4ns 0.0223 0 0 1.69 KB
#2916 EnrichedLog net472 4.72μs 3.9ns 15.1ns 0.329 0 0 2.08 KB
#2916 EnrichedLog netcoreapp3.1 4.07μs 4.32ns 16.2ns 0.0221 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 807ns 0.774ns 2.9ns 0.105 0 0 658 B
master StartFinishSpan netcoreapp3.1 734ns 0.234ns 0.811ns 0.00881 0 0 648 B
master StartFinishScope net472 1.01μs 0.336ns 1.3ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 862ns 0.279ns 1.04ns 0.0104 0 0 768 B
#2916 StartFinishSpan net472 830ns 0.234ns 0.906ns 0.105 0 0 658 B
#2916 StartFinishSpan netcoreapp3.1 737ns 0.268ns 1.04ns 0.00871 0 0 648 B
#2916 StartFinishScope net472 1.09μs 0.552ns 2.06ns 0.117 0 0 738 B
#2916 StartFinishScope netcoreapp3.1 893ns 0.396ns 1.53ns 0.0105 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.14μs 0.53ns 2.05ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 979ns 0.268ns 0.966ns 0.0103 0 0 768 B
#2916 RunOnMethodBegin net472 1.26μs 0.631ns 2.44ns 0.117 0 0 738 B
#2916 RunOnMethodBegin netcoreapp3.1 1μs 0.38ns 1.47ns 0.0105 0 0 768 B

andrewlock avatar Jul 28 '22 01:07 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2916 into master will not change line coverage :heavy_check_mark: Merging #2916 into master will not change branch coverage :no_entry: Merging #2916 into master will will increase complexity by 16

master #2916 Change
Lines 16748 / 22814 16775 / 22827
Lines % 73% 73% 0% :heavy_check_mark:
Branches 9984 / 14160 10010 / 14176
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 15217 15233 16 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2916 Change
Lines % 73% 73% 0% :heavy_check_mark:
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 15217 15233 16 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:

View the full reports for further details:

andrewlock avatar Jul 28 '22 02:07 andrewlock

Benchmarks Report :snail:

Benchmarks for #2916 compared to master:

  • 3 benchmarks are slower, with geometric mean 1.127
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 710μs 504ns 1.95μs 0.355 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 470μs 148ns 532ns 0 0 0 2.58 KB
#2916 WriteAndFlushEnrichedTraces net472 717μs 403ns 1.56μs 0.357 0 0 3.18 KB
#2916 WriteAndFlushEnrichedTraces netcoreapp3.1 455μs 352ns 1.32μs 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.71μs 2.43ns 9.43ns 0.237 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.88μs 2.98ns 11.6ns 0.0187 0 0 1.37 KB
master AllCycleMoreComplexBody net472 17μs 12.3ns 47.5ns 1.39 0.0249 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.3μs 12.7ns 45.9ns 0.107 0 0 7.85 KB
master BodyExtractorSimpleBody net472 273ns 0.283ns 1.06ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 224ns 0.214ns 0.831ns 0.00362 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.3μs 11.7ns 45.2ns 1.2 0.0153 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 11.6μs 17.6ns 65.7ns 0.0924 0 0 6.75 KB
#2916 AllCycleSimpleBody net472 1.69μs 2.78ns 10.8ns 0.237 0 0 1.49 KB
#2916 AllCycleSimpleBody netcoreapp3.1 1.84μs 1.27ns 4.57ns 0.0183 0 0 1.37 KB
#2916 AllCycleMoreComplexBody net472 16.8μs 10.3ns 38.6ns 1.39 0.0254 0 8.75 KB
#2916 AllCycleMoreComplexBody netcoreapp3.1 14.3μs 35.3ns 137ns 0.105 0 0 7.85 KB
#2916 BodyExtractorSimpleBody net472 266ns 0.228ns 0.853ns 0.0574 0 0 361 B
#2916 BodyExtractorSimpleBody netcoreapp3.1 233ns 0.346ns 1.34ns 0.00373 0 0 272 B
#2916 BodyExtractorMoreComplexBody net472 14.6μs 8.59ns 33.3ns 1.21 0.0218 0 7.62 KB
#2916 BodyExtractorMoreComplexBody netcoreapp3.1 11.7μs 15.1ns 58.6ns 0.0876 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 179μs 136ns 508ns 0.178 0 0 20.37 KB
#2916 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2916 SendRequest netcoreapp3.1 181μs 137ns 532ns 0.273 0 0 20.37 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.64μs 0.636ns 2.46ns 0.126 0.000823 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.31μs 0.493ns 1.91ns 0.0111 0 0 824 B
#2916 ExecuteNonQuery net472 1.55μs 0.145ns 0.524ns 0.126 0.000778 0 794 B
#2916 ExecuteNonQuery netcoreapp3.1 1.25μs 0.47ns 1.82ns 0.0113 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.25μs 2.39ns 8.93ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.41μs 1.65ns 6.18ns 0.0134 0 0 984 B
master CallElasticsearchAsync net472 2.27μs 3.63ns 14.1ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.53μs 1.17ns 4.53ns 0.0145 0 0 1.1 KB
#2916 CallElasticsearch net472 2.11μs 2.76ns 10.7ns 0.159 0 0 1 KB
#2916 CallElasticsearch netcoreapp3.1 1.41μs 1.77ns 6.87ns 0.0133 0 0 984 B
#2916 CallElasticsearchAsync net472 2.35μs 2.51ns 9.71ns 0.181 0 0 1.14 KB
#2916 CallElasticsearchAsync netcoreapp3.1 1.51μs 0.808ns 3.02ns 0.0151 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.42μs 1.88ns 7.02ns 0.199 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.68μs 2.47ns 9.58ns 0.0167 0 0 1.22 KB
#2916 ExecuteAsync net472 2.51μs 1.99ns 7.71ns 0.2 0 0 1.26 KB
#2916 ExecuteAsync netcoreapp3.1 1.67μs 0.427ns 1.65ns 0.0166 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.04μs 3.64ns 14.1ns 0.393 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.6μs 10.6ns 41.2ns 0.0331 0 0 2.4 KB
#2916 SendAsync net472 5.19μs 1.25ns 4.68ns 0.393 0 0 2.48 KB
#2916 SendAsync netcoreapp3.1 3.51μs 0.802ns 2.78ns 0.0316 0 0 2.4 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.84μs 1.91ns 7.4ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.25μs 6.49ns 23.4ns 0.0234 0 0 1.73 KB
#2916 EnrichedLog net472 2.97μs 2.39ns 9.24ns 0.264 0 0 1.66 KB
#2916 EnrichedLog netcoreapp3.1 2.29μs 1.2ns 4.51ns 0.0227 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 146μs 189ns 731ns 0.666 0.222 0 4.5 KB
master EnrichedLog netcoreapp3.1 109μs 228ns 885ns 0.0548 0 0 4.38 KB
#2916 EnrichedLog net472 147μs 83.9ns 325ns 0.664 0.221 0 4.5 KB
#2916 EnrichedLog netcoreapp3.1 112μs 168ns 650ns 0 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.41μs 1.93ns 7.49ns 0.543 0.00272 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.37μs 4.43ns 15.4ns 0.0517 0 0 3.8 KB
#2916 EnrichedLog net472 5.33μs 1.19ns 4.59ns 0.545 0.00267 0 3.43 KB
#2916 EnrichedLog netcoreapp3.1 4.35μs 1.69ns 6.31ns 0.0521 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.99μs 1.09ns 4.23ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.62μs 1.08ns 4.19ns 0.0162 0 0 1.21 KB
#2916 SendReceive net472 1.96μs 1.08ns 4.19ns 0.194 0 0 1.22 KB
#2916 SendReceive netcoreapp3.1 1.59μs 0.651ns 2.52ns 0.0168 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.67μs 5.47ns 21.2ns 0.33 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.08μs 3.18ns 11.9ns 0.0225 0 0 1.69 KB
#2916 EnrichedLog net472 4.68μs 5.58ns 21.6ns 0.329 0 0 2.08 KB
#2916 EnrichedLog netcoreapp3.1 4.13μs 5.56ns 21.5ns 0.0225 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #2916

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472 1.147 743.54 853.08

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 744ns 0.285ns 1.1ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 805ns 0.537ns 2.01ns 0.00853 0 0 648 B
master StartFinishScope net472 1.04μs 0.328ns 1.27ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 875ns 0.584ns 2.26ns 0.0105 0 0 768 B
#2916 StartFinishSpan net472 853ns 0.303ns 1.13ns 0.104 0 0 658 B
#2916 StartFinishSpan netcoreapp3.1 733ns 0.265ns 1.03ns 0.00872 0 0 648 B
#2916 StartFinishScope net472 1.1μs 0.269ns 1.04ns 0.117 0 0 738 B
#2916 StartFinishScope netcoreapp3.1 925ns 0.413ns 1.6ns 0.0102 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #2916

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1 1.120 972.40 1,089.41
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472 1.115 1,146.26 1,277.97

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.15μs 0.284ns 1.1ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 971ns 1.07ns 3.99ns 0.0106 0 0 768 B
#2916 RunOnMethodBegin net472 1.28μs 0.316ns 1.18ns 0.117 0 0 738 B
#2916 RunOnMethodBegin netcoreapp3.1 1.09μs 0.608ns 2.35ns 0.0103 0 0 768 B

andrewlock avatar Jul 28 '22 22:07 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2916 into master will not change line coverage :heavy_check_mark: Merging #2916 into master will not change branch coverage :no_entry: Merging #2916 into master will will increase complexity by 16

master #2916 Change
Lines 16804 / 22853 16826 / 22866
Lines % 74% 74% 0% :heavy_check_mark:
Branches 10015 / 14174 10035 / 14190
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 15236 15252 16 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2916 Change
Lines % 74% 74% 0% :heavy_check_mark:
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 15236 15252 16 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.GitInfo 4% :heavy_check_mark: 6% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:

View the full reports for further details:

andrewlock avatar Jul 28 '22 23:07 andrewlock

Benchmarks Report :snail:

Benchmarks for #2916 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 719μs 673ns 2.52μs 0.359 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 458μs 220ns 824ns 0 0 0 2.58 KB
#2916 WriteAndFlushEnrichedTraces net472 716μs 139ns 481ns 0.357 0 0 3.18 KB
#2916 WriteAndFlushEnrichedTraces netcoreapp3.1 454μs 440ns 1.7μs 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.71μs 4.24ns 16.4ns 0.236 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.87μs 1.95ns 7.3ns 0.0187 0 0 1.37 KB
master AllCycleMoreComplexBody net472 16.8μs 56.3ns 218ns 1.39 0.0249 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.2μs 18.3ns 70.9ns 0.107 0 0 7.85 KB
master BodyExtractorSimpleBody net472 251ns 0.19ns 0.737ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 226ns 0.427ns 1.6ns 0.0038 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.6μs 9.35ns 33.7ns 1.21 0.0147 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 12.1ns 46.8ns 0.0905 0 0 6.75 KB
#2916 AllCycleSimpleBody net472 1.68μs 1.13ns 4.38ns 0.237 0 0 1.49 KB
#2916 AllCycleSimpleBody netcoreapp3.1 1.77μs 1.7ns 6.37ns 0.0187 0 0 1.37 KB
#2916 AllCycleMoreComplexBody net472 16.8μs 7.16ns 27.7ns 1.39 0.0252 0 8.75 KB
#2916 AllCycleMoreComplexBody netcoreapp3.1 13.8μs 9.91ns 37.1ns 0.104 0 0 7.85 KB
#2916 BodyExtractorSimpleBody net472 256ns 0.329ns 1.27ns 0.0574 0 0 361 B
#2916 BodyExtractorSimpleBody netcoreapp3.1 218ns 0.234ns 0.906ns 0.00365 0 0 272 B
#2916 BodyExtractorMoreComplexBody net472 15μs 14.5ns 56.2ns 1.2 0.0149 0 7.62 KB
#2916 BodyExtractorMoreComplexBody netcoreapp3.1 11.7μs 9.31ns 34.8ns 0.0876 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 181μs 166ns 642ns 0.181 0 0 20.33 KB
#2916 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2916 SendRequest netcoreapp3.1 180μs 176ns 681ns 0.179 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.51μs 0.67ns 2.59ns 0.126 0.000761 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.27μs 0.378ns 1.47ns 0.0114 0 0 824 B
#2916 ExecuteNonQuery net472 1.58μs 1.02ns 3.95ns 0.126 0.000802 0 794 B
#2916 ExecuteNonQuery netcoreapp3.1 1.23μs 0.744ns 2.78ns 0.011 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.12μs 1.05ns 4.06ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.41μs 0.982ns 3.67ns 0.0134 0 0 984 B
master CallElasticsearchAsync net472 2.45μs 0.989ns 3.83ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.48μs 0.503ns 1.88ns 0.0148 0 0 1.1 KB
#2916 CallElasticsearch net472 2.25μs 0.934ns 3.62ns 0.158 0 0 1 KB
#2916 CallElasticsearch netcoreapp3.1 1.33μs 0.536ns 2.01ns 0.0134 0 0 984 B
#2916 CallElasticsearchAsync net472 2.4μs 0.677ns 2.62ns 0.18 0 0 1.14 KB
#2916 CallElasticsearchAsync netcoreapp3.1 1.51μs 1.79ns 6.93ns 0.0151 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.39μs 5.36ns 20ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.63μs 2.21ns 8.28ns 0.0164 0 0 1.22 KB
#2916 ExecuteAsync net472 2.48μs 2.04ns 7.91ns 0.2 0 0 1.26 KB
#2916 ExecuteAsync netcoreapp3.1 1.64μs 1.18ns 4.59ns 0.0165 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.92μs 5.04ns 19.5ns 0.392 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.22μs 2.52ns 9.43ns 0.032 0 0 2.36 KB
#2916 SendAsync net472 5.01μs 4.54ns 17.6ns 0.394 0 0 2.48 KB
#2916 SendAsync netcoreapp3.1 3.22μs 4.1ns 15.4ns 0.0323 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.73μs 0.783ns 2.82ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.36μs 0.97ns 3.76ns 0.0238 0 0 1.73 KB
#2916 EnrichedLog net472 2.83μs 0.901ns 3.37ns 0.263 0 0 1.66 KB
#2916 EnrichedLog netcoreapp3.1 2.27μs 0.861ns 3.33ns 0.023 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 146μs 114ns 440ns 0.659 0.22 0 4.5 KB
master EnrichedLog netcoreapp3.1 112μs 144ns 557ns 0.0562 0 0 4.38 KB
#2916 EnrichedLog net472 148μs 129ns 499ns 0.661 0.22 0 4.5 KB
#2916 EnrichedLog netcoreapp3.1 113μs 112ns 434ns 0.0562 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.42μs 5.29ns 19.8ns 0.543 0.0027 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.39μs 2.26ns 8.77ns 0.0525 0 0 3.8 KB
#2916 EnrichedLog net472 5.38μs 5.99ns 23.2ns 0.545 0.0027 0 3.43 KB
#2916 EnrichedLog netcoreapp3.1 4.22μs 3.1ns 12ns 0.0507 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.02μs 1.61ns 6.25ns 0.194 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.72μs 0.607ns 2.27ns 0.0163 0 0 1.21 KB
#2916 SendReceive net472 1.99μs 0.732ns 2.74ns 0.193 0 0 1.22 KB
#2916 SendReceive netcoreapp3.1 1.68μs 3.98ns 14.9ns 0.0158 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.63μs 1.88ns 7.05ns 0.328 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.22μs 1.37ns 5.32ns 0.0232 0 0 1.69 KB
#2916 EnrichedLog net472 4.64μs 1.51ns 5.83ns 0.33 0 0 2.08 KB
#2916 EnrichedLog netcoreapp3.1 4.08μs 1.11ns 4.14ns 0.0224 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 849ns 0.263ns 0.985ns 0.105 0 0 658 B
master StartFinishSpan netcoreapp3.1 762ns 0.226ns 0.875ns 0.00877 0 0 648 B
master StartFinishScope net472 1.09μs 0.281ns 1.05ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 888ns 0.66ns 2.47ns 0.0101 0 0 768 B
#2916 StartFinishSpan net472 870ns 0.342ns 1.23ns 0.104 0 0 658 B
#2916 StartFinishSpan netcoreapp3.1 736ns 4.03ns 23.2ns 0.0087 0 0 648 B
#2916 StartFinishScope net472 1.04μs 0.406ns 1.57ns 0.117 0 0 738 B
#2916 StartFinishScope netcoreapp3.1 866ns 0.175ns 0.607ns 0.0104 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.19μs 0.319ns 1.23ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.01μs 0.491ns 1.84ns 0.0101 0 0 768 B
#2916 RunOnMethodBegin net472 1.28μs 0.627ns 2.35ns 0.117 0 0 738 B
#2916 RunOnMethodBegin netcoreapp3.1 998ns 0.264ns 0.99ns 0.00997 0 0 768 B

andrewlock avatar Aug 03 '22 23:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #2916 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.127
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 727μs 312ns 1.21μs 0.361 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 470μs 171ns 661ns 0 0 0 2.58 KB
#2916 WriteAndFlushEnrichedTraces net472 713μs 260ns 1.01μs 0.357 0 0 3.18 KB
#2916 WriteAndFlushEnrichedTraces netcoreapp3.1 457μs 208ns 806ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #2916

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AppSecBodyBenchmark.BodyExtractorSimpleBody‑net472 1.127 252.75 284.88

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 184ns 0.274ns 1.03ns 0.0675 9.18E-05 0 425 B
master AllCycleSimpleBody netcoreapp3.1 235ns 0.31ns 1.2ns 0.00577 0 0 424 B
master AllCycleMoreComplexBody net472 186ns 0.336ns 1.3ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 242ns 0.264ns 1.02ns 0.00547 0 0 400 B
master BodyExtractorSimpleBody net472 253ns 0.285ns 1.1ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 236ns 0.279ns 1.04ns 0.00368 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.5μs 12.8ns 49.4ns 1.21 0.0218 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 11.6ns 45ns 0.091 0 0 6.75 KB
#2916 AllCycleSimpleBody net472 182ns 0.203ns 0.788ns 0.0675 0 0 425 B
#2916 AllCycleSimpleBody netcoreapp3.1 237ns 0.314ns 1.21ns 0.00593 0 0 424 B
#2916 AllCycleMoreComplexBody net472 181ns 0.2ns 0.748ns 0.0637 0 0 401 B
#2916 AllCycleMoreComplexBody netcoreapp3.1 236ns 0.16ns 0.6ns 0.00534 0 0 400 B
#2916 BodyExtractorSimpleBody net472 284ns 0.304ns 1.18ns 0.0573 0 0 361 B
#2916 BodyExtractorSimpleBody netcoreapp3.1 226ns 0.242ns 0.935ns 0.00374 0 0 272 B
#2916 BodyExtractorMoreComplexBody net472 14.7μs 29.1ns 113ns 1.21 0.0219 0 7.62 KB
#2916 BodyExtractorMoreComplexBody netcoreapp3.1 12.7μs 11.8ns 45.8ns 0.0887 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 179μs 106ns 412ns 0.179 0 0 20.57 KB
#2916 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2916 SendRequest netcoreapp3.1 179μs 140ns 542ns 0.27 0 0 20.57 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.91μs 0.595ns 2.15ns 0.15 0.00096 0 947 B
master ExecuteNonQuery netcoreapp3.1 1.41μs 1.04ns 3.89ns 0.0127 0 0 936 B
#2916 ExecuteNonQuery net472 1.84μs 0.464ns 1.73ns 0.15 0.000927 0 947 B
#2916 ExecuteNonQuery netcoreapp3.1 1.49μs 1.33ns 4.96ns 0.0126 0 0 936 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.53μs 0.884ns 3.31ns 0.183 0 0 1.16 KB
master CallElasticsearch netcoreapp3.1 1.53μs 0.572ns 2.22ns 0.0144 0 0 1.1 KB
master CallElasticsearchAsync net472 2.71μs 1.54ns 5.95ns 0.204 0 0 1.29 KB
master CallElasticsearchAsync netcoreapp3.1 1.54μs 0.621ns 2.4ns 0.0169 0 0 1.22 KB
#2916 CallElasticsearch net472 2.44μs 0.381ns 1.37ns 0.183 0 0 1.16 KB
#2916 CallElasticsearch netcoreapp3.1 1.51μs 1.63ns 6.33ns 0.015 0 0 1.1 KB
#2916 CallElasticsearchAsync net472 2.6μs 2.46ns 9.2ns 0.205 0 0 1.29 KB
#2916 CallElasticsearchAsync netcoreapp3.1 1.53μs 0.607ns 2.27ns 0.0167 0 0 1.22 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.58μs 12.6ns 54.9ns 0.223 0 0 1.41 KB
master ExecuteAsync netcoreapp3.1 1.65μs 2.18ns 8.14ns 0.0185 0 0 1.34 KB
#2916 ExecuteAsync net472 2.61μs 4.54ns 17.6ns 0.223 0 0 1.41 KB
#2916 ExecuteAsync netcoreapp3.1 1.73μs 4.39ns 17ns 0.0181 0 0 1.34 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.89μs 5.16ns 19.3ns 0.439 0 0 2.77 KB
master SendAsync netcoreapp3.1 3.56μs 4.37ns 15.7ns 0.0343 0 0 2.6 KB
#2916 SendAsync net472 5.82μs 9.35ns 36.2ns 0.438 0 0 2.77 KB
#2916 SendAsync netcoreapp3.1 3.58μs 8.72ns 32.6ns 0.0357 0 0 2.6 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.17μs 2.35ns 8.8ns 0.287 0 0 1.81 KB
master EnrichedLog netcoreapp3.1 2.52μs 1.1ns 4.1ns 0.0252 0 0 1.85 KB
#2916 EnrichedLog net472 3.05μs 4.06ns 15.7ns 0.287 0 0 1.81 KB
#2916 EnrichedLog netcoreapp3.1 2.44μs 2.05ns 7.95ns 0.0244 0 0 1.85 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 152μs 72.3ns 270ns 0.683 0.228 0 4.65 KB
master EnrichedLog netcoreapp3.1 115μs 138ns 536ns 0 0 0 4.49 KB
#2916 EnrichedLog net472 153μs 148ns 572ns 0.684 0.228 0 4.65 KB
#2916 EnrichedLog netcoreapp3.1 117μs 322ns 1.25μs 0.0589 0 0 4.49 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.44μs 11.9ns 46.2ns 0.57 0.00266 0 3.59 KB
master EnrichedLog netcoreapp3.1 4.36μs 8.23ns 31.9ns 0.0521 0 0 3.91 KB
#2916 EnrichedLog net472 5.5μs 8.13ns 30.4ns 0.569 0.00266 0 3.59 KB
#2916 EnrichedLog netcoreapp3.1 4.54μs 5.44ns 20.3ns 0.0514 0 0 3.91 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.48μs 2.03ns 7.85ns 0.217 0 0 1.37 KB
master SendReceive netcoreapp3.1 1.88μs 0.825ns 3.09ns 0.0179 0 0 1.32 KB
#2916 SendReceive net472 2.3μs 1.94ns 7.5ns 0.218 0 0 1.37 KB
#2916 SendReceive netcoreapp3.1 1.82μs 1.37ns 5.12ns 0.0183 0 0 1.32 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.12μs 9.54ns 35.7ns 0.353 0 0 2.23 KB
master EnrichedLog netcoreapp3.1 4.3μs 1.73ns 6.72ns 0.0235 0 0 1.8 KB
#2916 EnrichedLog net472 4.95μs 1.02ns 3.95ns 0.352 0 0 2.23 KB
#2916 EnrichedLog netcoreapp3.1 4.13μs 2.72ns 10.2ns 0.0227 0 0 1.8 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 1.16μs 0.426ns 1.65ns 0.128 0 0 810 B
master StartFinishSpan netcoreapp3.1 939ns 1.78ns 6.91ns 0.0103 0 0 760 B
master StartFinishScope net472 1.41μs 0.519ns 2.01ns 0.141 0 0 891 B
master StartFinishScope netcoreapp3.1 1.04μs 0.271ns 1.01ns 0.0119 0 0 880 B
#2916 StartFinishSpan net472 1.13μs 0.294ns 1.14ns 0.129 0 0 810 B
#2916 StartFinishSpan netcoreapp3.1 894ns 0.304ns 1.14ns 0.0103 0 0 760 B
#2916 StartFinishScope net472 1.42μs 2.14ns 7.99ns 0.141 0 0 891 B
#2916 StartFinishScope netcoreapp3.1 1.08μs 0.468ns 1.75ns 0.0115 0 0 880 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.44μs 0.334ns 1.2ns 0.141 0 0 891 B
master RunOnMethodBegin netcoreapp3.1 1.11μs 0.44ns 1.65ns 0.0116 0 0 880 B
#2916 RunOnMethodBegin net472 1.49μs 0.551ns 2.06ns 0.141 0 0 891 B
#2916 RunOnMethodBegin netcoreapp3.1 1.14μs 0.645ns 2.41ns 0.0119 0 0 880 B

andrewlock avatar Aug 23 '22 20:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #2916 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.119
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 802μs 703ns 2.72μs 0.398 0 0 3.22 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 540μs 245ns 947ns 0 0 0 2.63 KB
#2916 WriteAndFlushEnrichedTraces net472 799μs 886ns 3.43μs 0.398 0 0 3.22 KB
#2916 WriteAndFlushEnrichedTraces netcoreapp3.1 530μs 195ns 677ns 0 0 0 2.63 KB
Benchmarks.Trace.AppSecBodyBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #2916

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AppSecBodyBenchmark.BodyExtractorSimpleBody‑net472 1.119 269.92 301.93

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 197ns 0.109ns 0.408ns 0.0726 0 0 457 B
master AllCycleSimpleBody netcoreapp3.1 254ns 0.106ns 0.397ns 0.00624 0 0 456 B
master AllCycleMoreComplexBody net472 195ns 0.0922ns 0.333ns 0.0688 0 0 433 B
master AllCycleMoreComplexBody netcoreapp3.1 256ns 0.29ns 1.12ns 0.0059 0 0 432 B
master BodyExtractorSimpleBody net472 270ns 0.152ns 0.588ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 247ns 0.124ns 0.449ns 0.00372 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.9μs 8.78ns 32.9ns 1.2 0.0157 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.9μs 6.78ns 25.4ns 0.0902 0 0 6.75 KB
#2916 AllCycleSimpleBody net472 197ns 0.0743ns 0.278ns 0.0727 0 0 457 B
#2916 AllCycleSimpleBody netcoreapp3.1 259ns 0.285ns 1.03ns 0.00624 0 0 456 B
#2916 AllCycleMoreComplexBody net472 197ns 0.0878ns 0.317ns 0.0688 0 0 433 B
#2916 AllCycleMoreComplexBody netcoreapp3.1 249ns 0.195ns 0.729ns 0.00601 0 0 432 B
#2916 BodyExtractorSimpleBody net472 301ns 0.728ns 2.82ns 0.0574 0 0 361 B
#2916 BodyExtractorSimpleBody netcoreapp3.1 238ns 0.164ns 0.614ns 0.00369 0 0 272 B
#2916 BodyExtractorMoreComplexBody net472 15.6μs 13.9ns 50.1ns 1.21 0.0155 0 7.62 KB
#2916 BodyExtractorMoreComplexBody netcoreapp3.1 13μs 48.3ns 187ns 0.0894 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0.0266ns 0.0124ns 0.0568ns 0 0 0 0 b
master SendRequest netcoreapp3.1 182μs 222ns 859ns 0.272 0 0 20.68 KB
#2916 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2916 SendRequest netcoreapp3.1 181μs 183ns 710ns 0.182 0 0 20.68 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.84μs 1.11ns 4.32ns 0.155 0.000923 0 979 B
master ExecuteNonQuery netcoreapp3.1 1.43μs 0.535ns 2ns 0.0132 0 0 968 B
#2916 ExecuteNonQuery net472 1.8μs 0.514ns 1.85ns 0.155 0.000905 0 979 B
#2916 ExecuteNonQuery netcoreapp3.1 1.39μs 0.494ns 1.85ns 0.0132 0 0 968 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.55μs 0.61ns 2.28ns 0.188 0 0 1.19 KB
master CallElasticsearch netcoreapp3.1 1.52μs 0.551ns 1.99ns 0.0153 0 0 1.13 KB
master CallElasticsearchAsync net472 2.69μs 2.15ns 8.34ns 0.21 0 0 1.32 KB
master CallElasticsearchAsync netcoreapp3.1 1.54μs 0.689ns 2.58ns 0.0162 0 0 1.25 KB
#2916 CallElasticsearch net472 2.55μs 0.921ns 3.57ns 0.188 0 0 1.19 KB
#2916 CallElasticsearch netcoreapp3.1 1.55μs 0.606ns 2.27ns 0.0148 0 0 1.13 KB
#2916 CallElasticsearchAsync net472 2.7μs 1.17ns 4.51ns 0.21 0 0 1.32 KB
#2916 CallElasticsearchAsync netcoreapp3.1 1.65μs 0.755ns 2.92ns 0.0173 0 0 1.25 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.57μs 0.676ns 2.53ns 0.23 0 0 1.45 KB
master ExecuteAsync netcoreapp3.1 1.73μs 0.616ns 2.39ns 0.0191 0 0 1.38 KB
#2916 ExecuteAsync net472 2.83μs 1.09ns 4.06ns 0.23 0 0 1.45 KB
#2916 ExecuteAsync netcoreapp3.1 1.72μs 1.02ns 3.81ns 0.018 0 0 1.38 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.88μs 2.24ns 8.38ns 0.442 0 0 2.8 KB
master SendAsync netcoreapp3.1 3.65μs 1.71ns 6.41ns 0.0347 0 0 2.63 KB
#2916 SendAsync net472 5.99μs 3.17ns 11.8ns 0.444 0 0 2.8 KB
#2916 SendAsync netcoreapp3.1 3.76μs 2.59ns 9.7ns 0.0357 0 0 2.63 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.08μs 1.29ns 4.65ns 0.297 0 0 1.88 KB
master EnrichedLog netcoreapp3.1 2.61μs 2ns 7.47ns 0.0259 0 0 1.91 KB
#2916 EnrichedLog net472 3.26μs 3.62ns 14ns 0.298 0 0 1.88 KB
#2916 EnrichedLog netcoreapp3.1 2.54μs 1.15ns 4.15ns 0.0254 0 0 1.91 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 151μs 135ns 522ns 0.681 0.227 0 4.72 KB
master EnrichedLog netcoreapp3.1 123μs 187ns 675ns 0 0 0 4.55 KB
#2916 EnrichedLog net472 153μs 169ns 653ns 0.686 0.229 0 4.72 KB
#2916 EnrichedLog netcoreapp3.1 121μs 183ns 686ns 0 0 0 4.55 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.77μs 2.27ns 8.5ns 0.578 0.00288 0 3.65 KB
master EnrichedLog netcoreapp3.1 4.48μs 2.55ns 9.53ns 0.0539 0 0 3.98 KB
#2916 EnrichedLog net472 5.69μs 3.8ns 14.7ns 0.578 0.00285 0 3.65 KB
#2916 EnrichedLog netcoreapp3.1 4.46μs 1.19ns 4.29ns 0.0533 0 0 3.98 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.34μs 1.45ns 5.64ns 0.223 0 0 1.4 KB
master SendReceive netcoreapp3.1 1.83μs 0.791ns 3.06ns 0.0183 0 0 1.35 KB
#2916 SendReceive net472 2.33μs 3.61ns 14ns 0.222 0 0 1.4 KB
#2916 SendReceive netcoreapp3.1 1.75μs 0.543ns 1.96ns 0.0184 0 0 1.35 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.04μs 2.01ns 7.77ns 0.363 0 0 2.3 KB
master EnrichedLog netcoreapp3.1 4.29μs 3.86ns 15ns 0.024 0 0 1.86 KB
#2916 EnrichedLog net472 5.15μs 3.64ns 14.1ns 0.363 0 0 2.3 KB
#2916 EnrichedLog netcoreapp3.1 4.38μs 2.41ns 9.02ns 0.024 0 0 1.86 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 1.16μs 0.429ns 1.66ns 0.134 0 0 842 B
master StartFinishSpan netcoreapp3.1 959ns 0.345ns 1.29ns 0.0106 0 0 792 B
master StartFinishScope net472 1.39μs 0.723ns 2.61ns 0.146 0 0 923 B
master StartFinishScope netcoreapp3.1 1.07μs 0.287ns 1.04ns 0.0122 0 0 912 B
#2916 StartFinishSpan net472 1.15μs 0.543ns 2.1ns 0.134 0 0 842 B
#2916 StartFinishSpan netcoreapp3.1 957ns 1.35ns 4.88ns 0.0105 0 0 792 B
#2916 StartFinishScope net472 1.34μs 0.48ns 1.86ns 0.147 0 0 923 B
#2916 StartFinishScope netcoreapp3.1 1.09μs 0.89ns 3.45ns 0.0126 0 0 912 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.49μs 0.577ns 2.08ns 0.146 0 0 923 B
master RunOnMethodBegin netcoreapp3.1 1.18μs 0.423ns 1.53ns 0.0119 0 0 912 B
#2916 RunOnMethodBegin net472 1.58μs 0.821ns 3.18ns 0.147 0 0 923 B
#2916 RunOnMethodBegin netcoreapp3.1 1.15μs 0.743ns 2.88ns 0.0121 0 0 912 B

andrewlock avatar Oct 14 '22 16:10 andrewlock

Benchmarks Report :snail:

Benchmarks for #2916 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 816μs 577ns 2.16μs 0.406 0 0 3.22 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 559μs 456ns 1.71μs 0 0 0 2.63 KB
#2916 WriteAndFlushEnrichedTraces net472 813μs 428ns 1.6μs 0.403 0 0 3.22 KB
#2916 WriteAndFlushEnrichedTraces netcoreapp3.1 559μs 369ns 1.43μs 0 0 0 2.63 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 195ns 0.0577ns 0.208ns 0.0701 0 0 441 B
master AllCycleSimpleBody netcoreapp3.1 259ns 0.384ns 1.49ns 0.00608 0 0 440 B
master AllCycleMoreComplexBody net472 193ns 0.0261ns 0.0942ns 0.0663 0 0 417 B
master AllCycleMoreComplexBody netcoreapp3.1 251ns 0.0922ns 0.345ns 0.00564 0 0 416 B
master BodyExtractorSimpleBody net472 292ns 0.222ns 0.86ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 240ns 0.132ns 0.494ns 0.00374 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.8μs 12.3ns 47.6ns 1.21 0.0158 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.9μs 6.47ns 25.1ns 0.0904 0 0 6.75 KB
#2916 AllCycleSimpleBody net472 195ns 0.0551ns 0.199ns 0.0701 0 0 441 B
#2916 AllCycleSimpleBody netcoreapp3.1 255ns 0.27ns 1.04ns 0.00605 0 0 440 B
#2916 AllCycleMoreComplexBody net472 203ns 0.0697ns 0.251ns 0.0663 0 0 417 B
#2916 AllCycleMoreComplexBody netcoreapp3.1 251ns 0.167ns 0.646ns 0.00567 0 0 416 B
#2916 BodyExtractorSimpleBody net472 272ns 0.206ns 0.771ns 0.0574 0 0 361 B
#2916 BodyExtractorSimpleBody netcoreapp3.1 228ns 0.0932ns 0.349ns 0.00378 0 0 272 B
#2916 BodyExtractorMoreComplexBody net472 15.8μs 14.9ns 55.6ns 1.2 0.0158 0 7.62 KB
#2916 BodyExtractorMoreComplexBody netcoreapp3.1 13μs 5.96ns 22.3ns 0.0912 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 179μs 179ns 693ns 0.269 0 0 20.67 KB
#2916 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2916 SendRequest netcoreapp3.1 180μs 241ns 934ns 0.272 0 0 20.66 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.85μs 0.655ns 2.45ns 0.157 0.000926 0 995 B
master ExecuteNonQuery netcoreapp3.1 1.45μs 0.321ns 1.11ns 0.0132 0 0 984 B
#2916 ExecuteNonQuery net472 1.77μs 0.422ns 1.52ns 0.157 0.0009 0 995 B
#2916 ExecuteNonQuery netcoreapp3.1 1.41μs 0.436ns 1.69ns 0.0134 0 0 984 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.58μs 0.625ns 2.34ns 0.19 0 0 1.2 KB
master CallElasticsearch netcoreapp3.1 1.57μs 0.564ns 2.18ns 0.0151 0 0 1.14 KB
master CallElasticsearchAsync net472 2.66μs 0.802ns 3.11ns 0.212 0 0 1.34 KB
master CallElasticsearchAsync netcoreapp3.1 1.56μs 0.689ns 2.67ns 0.0172 0 0 1.26 KB
#2916 CallElasticsearch net472 2.57μs 0.895ns 3.35ns 0.19 0 0 1.2 KB
#2916 CallElasticsearch netcoreapp3.1 1.49μs 0.535ns 2ns 0.0155 0 0 1.14 KB
#2916 CallElasticsearchAsync net472 2.68μs 0.8ns 3.1ns 0.213 0 0 1.34 KB
#2916 CallElasticsearchAsync netcoreapp3.1 1.62μs 0.42ns 1.63ns 0.0171 0 0 1.26 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.73μs 1.18ns 4.41ns 0.232 0 0 1.47 KB
master ExecuteAsync netcoreapp3.1 1.75μs 2.73ns 10.6ns 0.019 0 0 1.39 KB
#2916 ExecuteAsync net472 2.69μs 1.16ns 4.34ns 0.233 0 0 1.47 KB
#2916 ExecuteAsync netcoreapp3.1 1.69μs 0.81ns 2.92ns 0.0185 0 0 1.39 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.87μs 3.01ns 11.7ns 0.446 0 0 2.82 KB
master SendAsync netcoreapp3.1 3.62μs 1.42ns 5.49ns 0.036 0 0 2.65 KB
#2916 SendAsync net472 5.85μs 2.46ns 9.53ns 0.447 0 0 2.82 KB
#2916 SendAsync netcoreapp3.1 3.61μs 1.42ns 5.33ns 0.0362 0 0 2.65 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.18μs 2.55ns 9.87ns 0.297 0 0 1.88 KB
master EnrichedLog netcoreapp3.1 2.49μs 1.51ns 5.84ns 0.026 0 0 1.91 KB
#2916 EnrichedLog net472 3.23μs 2.7ns 10.4ns 0.297 0 0 1.88 KB
#2916 EnrichedLog netcoreapp3.1 2.56μs 2.39ns 8.96ns 0.0254 0 0 1.91 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 151μs 150ns 562ns 0.68 0.227 0 4.72 KB
master EnrichedLog netcoreapp3.1 121μs 139ns 538ns 0.0607 0 0 4.55 KB
#2916 EnrichedLog net472 152μs 228ns 883ns 0.684 0.228 0 4.72 KB
#2916 EnrichedLog netcoreapp3.1 121μs 168ns 650ns 0.0609 0 0 4.55 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.63μs 2.02ns 7.83ns 0.58 0.0028 0 3.65 KB
master EnrichedLog netcoreapp3.1 4.41μs 1.81ns 6.78ns 0.053 0 0 3.98 KB
#2916 EnrichedLog net472 5.77μs 1.6ns 5.99ns 0.578 0.00288 0 3.65 KB
#2916 EnrichedLog netcoreapp3.1 4.42μs 8.99ns 31.1ns 0.0548 0 0 3.98 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.33μs 2.85ns 11ns 0.225 0 0 1.42 KB
master SendReceive netcoreapp3.1 1.79μs 0.589ns 2.2ns 0.018 0 0 1.37 KB
#2916 SendReceive net472 2.24μs 10.4ns 40.4ns 0.225 0 0 1.42 KB
#2916 SendReceive netcoreapp3.1 1.7μs 0.766ns 2.76ns 0.0187 0 0 1.37 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.99μs 2.19ns 8.48ns 0.362 0 0 2.3 KB
master EnrichedLog netcoreapp3.1 4.24μs 2.71ns 10.5ns 0.0253 0 0 1.86 KB
#2916 EnrichedLog net472 5.06μs 2.21ns 8.25ns 0.364 0 0 2.3 KB
#2916 EnrichedLog netcoreapp3.1 4.21μs 1.83ns 6.6ns 0.0236 0 0 1.86 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 1.16μs 0.486ns 1.82ns 0.136 0 0 859 B
master StartFinishSpan netcoreapp3.1 899ns 0.956ns 3.7ns 0.0107 0 0 808 B
master StartFinishScope net472 1.38μs 0.396ns 1.54ns 0.149 0 0 939 B
master StartFinishScope netcoreapp3.1 1.14μs 0.48ns 1.8ns 0.0125 0 0 928 B
#2916 StartFinishSpan net472 1.13μs 0.346ns 1.34ns 0.136 0 0 859 B
#2916 StartFinishSpan netcoreapp3.1 936ns 0.253ns 0.982ns 0.0106 0 0 808 B
#2916 StartFinishScope net472 1.37μs 0.504ns 1.95ns 0.149 0 0 939 B
#2916 StartFinishScope netcoreapp3.1 1.13μs 4.84ns 18.1ns 0.0124 0 0 928 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.53μs 0.397ns 1.43ns 0.148 0 0 939 B
master RunOnMethodBegin netcoreapp3.1 1.16μs 0.519ns 1.87ns 0.0125 0 0 928 B
#2916 RunOnMethodBegin net472 1.4μs 0.346ns 1.25ns 0.149 0 0 939 B
#2916 RunOnMethodBegin netcoreapp3.1 1.2μs 0.987ns 3.82ns 0.0126 0 0 928 B

andrewlock avatar Oct 17 '22 16:10 andrewlock

Datadog Report

Branch report: zach/http-timeout Commit report: b59add4

:x: dd-trace-dotnet 5 Failed (1 Known Flaky), 0 New Flaky, 233270 Passed, 1279 Skipped, 20m 53.01s Wall Time

:x: Failed Tests (5)

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.AgentMalfunctionTests - :snowflake: Known flaky - Details

    Expand for error
    esults do not match.
    ifferences:
    eceived: ProcessStartTests.SubmitsTraces.received.txt
    erified: ProcessStartTests.SubmitsTraces.verified.txt
    eceived Content:
    ]
    erified Content:
    
     {
       TraceId: Id_1,
    ..
    
  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.DefaultTransportLargePayloadTests - Details

    Expand for error
    uplicate trace ID 4764802941010121733 with 4 entries 
    uplicate trace ID 2279362277298759019 with 4 entries 
    uplicate trace ID 7043085773998794757 with 4 entries 
    uplicate trace ID 5659049009775410608 with 4 entries 
    uplicate trace ID 3980674800421557361 with 4 entries 
    uplicate trace ID 5366404713445531998 with 4 entries 
    uplicate trace ID 5829376432211298549 with 4 entries 
    uplicate trace ID 8779389218292778175 with 4 entries 
    uplicate trace ID 688222679232992592 with 4 entries 
    uplicate trace ID 7100640274203950296 with 4 entries 
    ..
    
  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.DefaultTransportLargePayloadTests - Details

    Expand for error
    uplicate trace ID 2553611272312032104 with 2 entries 
    uplicate trace ID 604925034792306176 with 2 entries 
    uplicate trace ID 7283800010447513082 with 2 entries 
    uplicate trace ID 9203275452945917366 with 2 entries 
    uplicate trace ID 1609437720427163747 with 2 entries 
    uplicate trace ID 2611050111989778799 with 2 entries 
    uplicate trace ID 349553304865396604 with 2 entries 
    uplicate trace ID 6931136627353170698 with 2 entries 
    uplicate trace ID 8244702453546256343 with 2 entries 
    uplicate trace ID 327357501733496538 with 2 entries 
    ..
    
  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.ServiceStackRedisTests - Details

    Expand for error
    xpected spans.Count to be 36, but found 60.
    
  • SubmitsTraces - Datadog.Trace.Security.IntegrationTests.Iast.DeduplicationTests - Details

    Expand for error
    esults do not match.
    ifferences:
    eceived: iast.deduplication.duplicated.received.txt
    erified: iast.deduplication.duplicated.verified.txt
    eceived Content:
    
     {
       TraceId: Id_1,
       SpanId: Id_2,
       Name: weak_hashing,
    ..
    

datadog-ddstaging[bot] avatar Nov 10 '22 17:11 datadog-ddstaging[bot]

@zacharycmontoya any chance we can rebase and merge this soon? 😅

andrewlock avatar Mar 20 '23 11:03 andrewlock

Closing because this is extremely stale. This was supposed to be a small update but flaky tests led to some doubt whether this would be stable, so I'll close this until we see a new need for it.

zacharycmontoya avatar Jan 29 '24 15:01 zacharycmontoya