dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard

Enable agent-based telemetry by default

Open andrewlock opened this issue 3 years ago • 22 comments

Summary of changes

  • Enables telemetry by default
  • Preferentially uses agent-based telemetry
  • If agent-based telemetry fails, falls back to agentless
  • ~Add version-check for agent telemetry in case of error to try and avoid retries~ Doesn't really work, as the agent doesn't send the Agent Version when sending a 404, so ripped it out again.

Reason for change

Other tracers preferentially send to the agent over the intake, which makes sense. We were waiting for UDS support before we do that (#2617).

Implementation details

  • Renamed TelemetryCircuitBreaker to TelemetryTransportManager and refactored to handle the transport failover as well as transient failures
  • For the agent transport, if the request is not successful ~attempt to read the agent version, and if it's not high enough, then don't bother with any retries~ it doesn't return this if it gives a 404, so makes the check a bit pointless
  • ~Currently, we only check the agent on an error - we could capture it on success too, but I can't see a scenario where that would be useful~
  • ~We currently extract and store the version header separately from the trace API responses. I looked at sharing the value across both, but it adds a lot of complexity (if we keep it testable), so I'd like to avoid that if possible. This might be required though if the agent doesn't send the agent version header for unsuccessful requests (to be confirmed)~

Test coverage

Unit tests for the TelemetryTransportManager behaviour and ~the agent version extraction.~ Integration tests are largely unchanged as nothing has changed in the success paths.

Other details

  • [x] ~TODO: confirm that the agent sends the agent version header when the telemetry proxy endpoint isn't supported. If not, we can rip out the agent version logic entirely~ It doesn't. Ripping out the agent version logic
  • [ ] In AAS, this will switch to using named pipes to send the telemetry, but we don't currently have testing of named-pipes in our mock agent. I'm thinking we should potentially add that first before merging this

andrewlock avatar May 13 '22 15:05 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 728μs 842ns 3.26μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 496μs 596ns 2.31μs 0 0 0 2.57 KB
#2800 WriteAndFlushEnrichedTraces net472 736μs 416ns 1.5μs 0 0 0 3.16 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 464μs 308ns 1.19μs 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 194ns 0.0514ns 0.192ns 0.0661 0 0 417 B
master AllCycleSimpleBody netcoreapp3.1 245ns 0.0758ns 0.273ns 0.00578 0 0 416 B
master AllCycleMoreComplexBody net472 193ns 0.0475ns 0.184ns 0.0623 0 0 393 B
master AllCycleMoreComplexBody netcoreapp3.1 244ns 0.0961ns 0.36ns 0.00539 0 0 392 B
master BodyExtractorSimpleBody net472 261ns 0.177ns 0.684ns 0.057 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 239ns 0.161ns 0.602ns 0.0036 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.6μs 7.05ns 26.4ns 1.19 0.0157 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 13.2μs 4.62ns 16.7ns 0.0923 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 195ns 0.109ns 0.423ns 0.0661 0 0 417 B
#2800 AllCycleSimpleBody netcoreapp3.1 245ns 0.192ns 0.742ns 0.00579 0 0 416 B
#2800 AllCycleMoreComplexBody net472 194ns 0.0604ns 0.218ns 0.0623 0 0 393 B
#2800 AllCycleMoreComplexBody netcoreapp3.1 244ns 0.138ns 0.498ns 0.00541 0 0 392 B
#2800 BodyExtractorSimpleBody net472 262ns 0.214ns 0.83ns 0.0571 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 229ns 0.103ns 0.386ns 0.0037 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 16.1μs 10.7ns 41.5ns 1.19 0.0161 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 12.8μs 5.14ns 19.2ns 0.0892 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 171μs 195ns 754ns 0.256 0 0 19.74 KB
#2800 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 173μs 105ns 407ns 0.258 0 0 19.74 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.39μs 0.536ns 2.07ns 0.0943 0.000689 0 594 B
master ExecuteNonQuery netcoreapp3.1 1.16μs 1.98ns 7.15ns 0.00861 0 0 632 B
#2800 ExecuteNonQuery net472 1.4μs 0.528ns 1.98ns 0.0943 0.000699 0 594 B
#2800 ExecuteNonQuery netcoreapp3.1 1.08μs 0.733ns 2.64ns 0.00864 0 0 632 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.03μs 0.528ns 1.83ns 0.125 0 0 802 B
master CallElasticsearch netcoreapp3.1 1.25μs 0.597ns 2.31ns 0.0106 0 0 792 B
master CallElasticsearchAsync net472 2.2μs 0.956ns 3.7ns 0.147 0 0 939 B
master CallElasticsearchAsync netcoreapp3.1 1.33μs 0.344ns 1.24ns 0.0126 0 0 912 B
#2800 CallElasticsearch net472 2.12μs 0.593ns 2.3ns 0.125 0 0 802 B
#2800 CallElasticsearch netcoreapp3.1 1.26μs 0.971ns 3.76ns 0.0106 0 0 792 B
#2800 CallElasticsearchAsync net472 2.2μs 0.648ns 2.51ns 0.147 0 0 939 B
#2800 CallElasticsearchAsync netcoreapp3.1 1.33μs 0.507ns 1.83ns 0.0122 0 0 912 B
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.22μs 1.22ns 4.4ns 0.165 0 0 1.06 KB
master ExecuteAsync netcoreapp3.1 1.46μs 0.539ns 1.87ns 0.0139 0 0 1.03 KB
#2800 ExecuteAsync net472 2.29μs 1.11ns 4.28ns 0.166 0 0 1.06 KB
#2800 ExecuteAsync netcoreapp3.1 1.46μs 1.76ns 6.59ns 0.0138 0 0 1.03 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.92μs 1.28ns 4.96ns 0.357 0 0 2.28 KB
master SendAsync netcoreapp3.1 3.28μs 2.66ns 9.97ns 0.0294 0 0 2.21 KB
#2800 SendAsync net472 4.94μs 1.69ns 6.31ns 0.356 0 0 2.28 KB
#2800 SendAsync netcoreapp3.1 3.28μs 2.09ns 7.81ns 0.0296 0 0 2.21 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.71μs 0.82ns 3.07ns 0.228 0 0 1.45 KB
master EnrichedLog netcoreapp3.1 2.16μs 1.09ns 4.09ns 0.0209 0 0 1.53 KB
#2800 EnrichedLog net472 2.63μs 1.1ns 4.25ns 0.228 0 0 1.45 KB
#2800 EnrichedLog netcoreapp3.1 2.18μs 1.31ns 4.89ns 0.0209 0 0 1.53 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 148μs 189ns 732ns 0.515 0.147 0 4.29 KB
master EnrichedLog netcoreapp3.1 112μs 162ns 627ns 0 0 0 4.18 KB
#2800 EnrichedLog net472 148μs 109ns 421ns 0.516 0.147 0 4.29 KB
#2800 EnrichedLog netcoreapp3.1 113μs 71.9ns 279ns 0.0562 0 0 4.18 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.14μs 2.08ns 7.78ns 0.506 0.00254 0 3.23 KB
master EnrichedLog netcoreapp3.1 4.17μs 1.92ns 7.19ns 0.0479 0 0 3.6 KB
#2800 EnrichedLog net472 5.28μs 2.21ns 8.56ns 0.507 0.00263 0 3.23 KB
#2800 EnrichedLog netcoreapp3.1 4.2μs 1.32ns 4.77ns 0.0497 0 0 3.6 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.88μs 0.964ns 3.73ns 0.159 0 0 1.01 KB
master SendReceive netcoreapp3.1 1.49μs 0.716ns 2.68ns 0.0139 0 0 1.01 KB
#2800 SendReceive net472 1.81μs 0.58ns 2.17ns 0.158 0 0 1.01 KB
#2800 SendReceive netcoreapp3.1 1.54μs 0.602ns 2.33ns 0.0139 0 0 1.01 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.66μs 2.27ns 8.78ns 0.292 0 0 1.87 KB
master EnrichedLog netcoreapp3.1 3.87μs 1.31ns 5.06ns 0.0193 0 0 1.49 KB
#2800 EnrichedLog net472 4.49μs 1.92ns 7.44ns 0.291 0 0 1.87 KB
#2800 EnrichedLog netcoreapp3.1 3.9μs 1.07ns 4.14ns 0.0195 0 0 1.49 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 795ns 0.981ns 3.8ns 0.0719 0 0 457 B
master StartFinishSpan netcoreapp3.1 628ns 0.227ns 0.851ns 0.00631 0 0 456 B
master StartFinishScope net472 1.07μs 0.206ns 0.772ns 0.0842 0 0 538 B
master StartFinishScope netcoreapp3.1 747ns 0.272ns 1.02ns 0.00778 0 0 576 B
#2800 StartFinishSpan net472 755ns 0.632ns 2.45ns 0.0718 0 0 457 B
#2800 StartFinishSpan netcoreapp3.1 662ns 0.304ns 1.18ns 0.00596 0 0 456 B
#2800 StartFinishScope net472 1.09μs 0.52ns 1.88ns 0.0844 0 0 538 B
#2800 StartFinishScope netcoreapp3.1 735ns 0.14ns 0.543ns 0.00772 0 0 576 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.15μs 0.562ns 1.95ns 0.0842 0 0 538 B
master RunOnMethodBegin netcoreapp3.1 925ns 0.237ns 0.856ns 0.00791 0 0 576 B
#2800 RunOnMethodBegin net472 1.14μs 0.707ns 2.74ns 0.0842 0 0 538 B
#2800 RunOnMethodBegin netcoreapp3.1 899ns 0.863ns 3.34ns 0.00761 0 0 576 B

andrewlock avatar May 13 '22 16:05 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 726μs 900ns 3.37μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 454μs 124ns 446ns 0 0 0 2.57 KB
#2800 WriteAndFlushEnrichedTraces net472 731μs 736ns 2.85μs 0 0 0 3.16 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 470μs 387ns 1.5μs 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 194ns 0.0597ns 0.231ns 0.0661 0 0 417 B
master AllCycleSimpleBody netcoreapp3.1 249ns 0.661ns 2.47ns 0.00581 0 0 416 B
master AllCycleMoreComplexBody net472 193ns 0.0872ns 0.338ns 0.0623 0 0 393 B
master AllCycleMoreComplexBody netcoreapp3.1 245ns 0.191ns 0.741ns 0.0054 0 0 392 B
master BodyExtractorSimpleBody net472 261ns 0.136ns 0.507ns 0.0571 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 233ns 0.511ns 1.98ns 0.00378 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.9μs 5.62ns 21ns 1.19 0.0159 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 13.3μs 7.29ns 28.2ns 0.0926 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 195ns 0.0809ns 0.313ns 0.0661 0 0 417 B
#2800 AllCycleSimpleBody netcoreapp3.1 247ns 0.112ns 0.402ns 0.00567 0 0 416 B
#2800 AllCycleMoreComplexBody net472 192ns 0.0262ns 0.098ns 0.0623 0 0 393 B
#2800 AllCycleMoreComplexBody netcoreapp3.1 244ns 0.0758ns 0.294ns 0.0054 0 0 392 B
#2800 BodyExtractorSimpleBody net472 262ns 0.137ns 0.53ns 0.0571 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 228ns 0.0825ns 0.309ns 0.00369 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 16μs 4.84ns 17.5ns 1.2 0.0163 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 12.9μs 4.2ns 16.3ns 0.0904 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 172μs 127ns 491ns 0.258 0 0 19.74 KB
#2800 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 172μs 119ns 461ns 0.257 0 0 19.74 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.4μs 0.386ns 1.49ns 0.0939 0.000701 0 594 B
master ExecuteNonQuery netcoreapp3.1 1.16μs 0.357ns 1.38ns 0.0087 0 0 632 B
#2800 ExecuteNonQuery net472 1.34μs 0.762ns 2.85ns 0.0941 0.000672 0 594 B
#2800 ExecuteNonQuery netcoreapp3.1 1.15μs 0.29ns 1.05ns 0.00859 0 0 632 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.03μs 0.605ns 2.26ns 0.125 0 0 802 B
master CallElasticsearch netcoreapp3.1 1.3μs 0.819ns 3.17ns 0.0104 0 0 792 B
master CallElasticsearchAsync net472 2.2μs 0.653ns 2.53ns 0.147 0 0 939 B
master CallElasticsearchAsync netcoreapp3.1 1.39μs 0.201ns 0.753ns 0.0125 0 0 912 B
#2800 CallElasticsearch net472 2.03μs 1.47ns 5.7ns 0.125 0 0 802 B
#2800 CallElasticsearch netcoreapp3.1 1.26μs 0.504ns 1.88ns 0.0108 0 0 792 B
#2800 CallElasticsearchAsync net472 2.14μs 1.08ns 4.16ns 0.147 0 0 939 B
#2800 CallElasticsearchAsync netcoreapp3.1 1.39μs 0.476ns 1.72ns 0.0122 0 0 912 B
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.24μs 0.801ns 3ns 0.165 0 0 1.06 KB
master ExecuteAsync netcoreapp3.1 1.5μs 3.85ns 14.9ns 0.0141 0 0 1.03 KB
#2800 ExecuteAsync net472 2.26μs 0.936ns 3.62ns 0.166 0 0 1.06 KB
#2800 ExecuteAsync netcoreapp3.1 1.44μs 0.563ns 2.11ns 0.0137 0 0 1.03 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.78μs 0.989ns 3.83ns 0.358 0 0 2.28 KB
master SendAsync netcoreapp3.1 3.23μs 0.858ns 3.21ns 0.0293 0 0 2.21 KB
#2800 SendAsync net472 4.76μs 2.03ns 7.86ns 0.356 0 0 2.28 KB
#2800 SendAsync netcoreapp3.1 3.35μs 0.854ns 3.19ns 0.0305 0 0 2.21 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.67μs 1.6ns 6.19ns 0.228 0 0 1.45 KB
master EnrichedLog netcoreapp3.1 2.28μs 0.881ns 3.3ns 0.0205 0 0 1.53 KB
#2800 EnrichedLog net472 2.72μs 1.01ns 3.9ns 0.228 0 0 1.45 KB
#2800 EnrichedLog netcoreapp3.1 2.15μs 1.13ns 4.22ns 0.0204 0 0 1.53 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 149μs 219ns 847ns 0.518 0.148 0 4.29 KB
master EnrichedLog netcoreapp3.1 111μs 189ns 732ns 0.0558 0 0 4.18 KB
#2800 EnrichedLog net472 149μs 141ns 545ns 0.52 0.149 0 4.29 KB
#2800 EnrichedLog netcoreapp3.1 113μs 139ns 537ns 0.0561 0 0 4.18 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.22μs 1.81ns 7.02ns 0.508 0.00262 0 3.23 KB
master EnrichedLog netcoreapp3.1 4.07μs 1.76ns 6.8ns 0.0489 0 0 3.6 KB
#2800 EnrichedLog net472 5.3μs 1.85ns 7.18ns 0.506 0.00265 0 3.23 KB
#2800 EnrichedLog netcoreapp3.1 4.07μs 2.8ns 10.5ns 0.0484 0 0 3.6 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.89μs 0.816ns 3.16ns 0.158 0 0 1.01 KB
master SendReceive netcoreapp3.1 1.56μs 0.625ns 2.42ns 0.0133 0 0 1.01 KB
#2800 SendReceive net472 1.81μs 1.17ns 4.53ns 0.159 0 0 1.01 KB
#2800 SendReceive netcoreapp3.1 1.59μs 0.621ns 2.24ns 0.0136 0 0 1.01 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.65μs 2.22ns 8.61ns 0.292 0 0 1.87 KB
master EnrichedLog netcoreapp3.1 3.87μs 1.19ns 4.46ns 0.0193 0 0 1.49 KB
#2800 EnrichedLog net472 4.56μs 2.7ns 10.5ns 0.292 0 0 1.87 KB
#2800 EnrichedLog netcoreapp3.1 3.8μs 1.79ns 6.68ns 0.0189 0 0 1.49 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 775ns 0.424ns 1.59ns 0.0718 0 0 457 B
master StartFinishSpan netcoreapp3.1 670ns 0.133ns 0.48ns 0.00604 0 0 456 B
master StartFinishScope net472 998ns 0.269ns 1.01ns 0.0846 0 0 538 B
master StartFinishScope netcoreapp3.1 737ns 0.258ns 0.893ns 0.00773 0 0 576 B
#2800 StartFinishSpan net472 738ns 0.297ns 1.15ns 0.0719 0 0 457 B
#2800 StartFinishSpan netcoreapp3.1 635ns 0.191ns 0.741ns 0.00634 0 0 456 B
#2800 StartFinishScope net472 991ns 0.872ns 3.38ns 0.0846 0 0 538 B
#2800 StartFinishScope netcoreapp3.1 803ns 0.273ns 1.02ns 0.008 0 0 576 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.16μs 0.56ns 2.17ns 0.084 0 0 538 B
master RunOnMethodBegin netcoreapp3.1 920ns 0.203ns 0.759ns 0.00783 0 0 576 B
#2800 RunOnMethodBegin net472 1.14μs 0.33ns 1.28ns 0.0843 0 0 538 B
#2800 RunOnMethodBegin netcoreapp3.1 874ns 0.25ns 0.902ns 0.00788 0 0 576 B

andrewlock avatar May 16 '22 10:05 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 726μs 900ns 3.37μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 454μs 124ns 446ns 0 0 0 2.57 KB
#2800 WriteAndFlushEnrichedTraces net472 727μs 900ns 3.37μs 0 0 0 3.16 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 481μs 236ns 915ns 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 194ns 0.0597ns 0.231ns 0.0661 0 0 417 B
master AllCycleSimpleBody netcoreapp3.1 249ns 0.661ns 2.47ns 0.00581 0 0 416 B
master AllCycleMoreComplexBody net472 193ns 0.0872ns 0.338ns 0.0623 0 0 393 B
master AllCycleMoreComplexBody netcoreapp3.1 245ns 0.191ns 0.741ns 0.0054 0 0 392 B
master BodyExtractorSimpleBody net472 261ns 0.136ns 0.507ns 0.0571 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 233ns 0.511ns 1.98ns 0.00378 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.9μs 5.62ns 21ns 1.19 0.0159 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 13.3μs 7.29ns 28.2ns 0.0926 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 196ns 0.404ns 1.56ns 0.0661 0 0 417 B
#2800 AllCycleSimpleBody netcoreapp3.1 246ns 0.213ns 0.823ns 0.00568 0 0 416 B
#2800 AllCycleMoreComplexBody net472 207ns 0.0589ns 0.228ns 0.0623 0 0 393 B
#2800 AllCycleMoreComplexBody netcoreapp3.1 248ns 0.752ns 2.81ns 0.00532 0 0 392 B
#2800 BodyExtractorSimpleBody net472 265ns 0.117ns 0.439ns 0.0571 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 242ns 0.196ns 0.761ns 0.00375 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 15.8μs 7.6ns 29.4ns 1.19 0.0158 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 13.1μs 4.34ns 16.2ns 0.092 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 172μs 127ns 491ns 0.258 0 0 19.74 KB
#2800 SendRequest net472 0.000438ns 0.000438ns 0.00164ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 171μs 79.5ns 308ns 0.258 0 0 19.74 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.4μs 0.386ns 1.49ns 0.0939 0.000701 0 594 B
master ExecuteNonQuery netcoreapp3.1 1.16μs 0.357ns 1.38ns 0.0087 0 0 632 B
#2800 ExecuteNonQuery net472 1.5μs 1.13ns 4.22ns 0.0942 0.00076 0 594 B
#2800 ExecuteNonQuery netcoreapp3.1 1.15μs 0.37ns 1.43ns 0.00856 0 0 632 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.03μs 0.605ns 2.26ns 0.125 0 0 802 B
master CallElasticsearch netcoreapp3.1 1.3μs 0.819ns 3.17ns 0.0104 0 0 792 B
master CallElasticsearchAsync net472 2.2μs 0.653ns 2.53ns 0.147 0 0 939 B
master CallElasticsearchAsync netcoreapp3.1 1.39μs 0.201ns 0.753ns 0.0125 0 0 912 B
#2800 CallElasticsearch net472 2.03μs 0.949ns 3.55ns 0.125 0 0 802 B
#2800 CallElasticsearch netcoreapp3.1 1.29μs 0.768ns 2.97ns 0.0103 0 0 792 B
#2800 CallElasticsearchAsync net472 2.19μs 0.484ns 1.87ns 0.147 0 0 939 B
#2800 CallElasticsearchAsync netcoreapp3.1 1.29μs 0.612ns 2.29ns 0.0124 0 0 912 B
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.24μs 0.801ns 3ns 0.165 0 0 1.06 KB
master ExecuteAsync netcoreapp3.1 1.5μs 3.85ns 14.9ns 0.0141 0 0 1.03 KB
#2800 ExecuteAsync net472 2.22μs 1.13ns 4.39ns 0.166 0 0 1.06 KB
#2800 ExecuteAsync netcoreapp3.1 1.47μs 0.512ns 1.92ns 0.014 0 0 1.03 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.78μs 0.989ns 3.83ns 0.358 0 0 2.28 KB
master SendAsync netcoreapp3.1 3.23μs 0.858ns 3.21ns 0.0293 0 0 2.21 KB
#2800 SendAsync net472 4.75μs 1.33ns 5.15ns 0.358 0 0 2.28 KB
#2800 SendAsync netcoreapp3.1 3.17μs 3.35ns 12.5ns 0.03 0 0 2.21 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.67μs 1.6ns 6.19ns 0.228 0 0 1.45 KB
master EnrichedLog netcoreapp3.1 2.28μs 0.881ns 3.3ns 0.0205 0 0 1.53 KB
#2800 EnrichedLog net472 2.77μs 1.32ns 5.12ns 0.227 0 0 1.45 KB
#2800 EnrichedLog netcoreapp3.1 2.2μs 0.974ns 3.64ns 0.0209 0 0 1.53 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 149μs 219ns 847ns 0.518 0.148 0 4.29 KB
master EnrichedLog netcoreapp3.1 111μs 189ns 732ns 0.0558 0 0 4.18 KB
#2800 EnrichedLog net472 149μs 110ns 426ns 0.52 0.148 0 4.29 KB
#2800 EnrichedLog netcoreapp3.1 113μs 147ns 548ns 0.0568 0 0 4.18 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.22μs 1.81ns 7.02ns 0.508 0.00262 0 3.23 KB
master EnrichedLog netcoreapp3.1 4.07μs 1.76ns 6.8ns 0.0489 0 0 3.6 KB
#2800 EnrichedLog net472 5.25μs 1.61ns 6.03ns 0.507 0.00261 0 3.23 KB
#2800 EnrichedLog netcoreapp3.1 4.13μs 1.27ns 4.76ns 0.0492 0 0 3.6 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.89μs 0.816ns 3.16ns 0.158 0 0 1.01 KB
master SendReceive netcoreapp3.1 1.56μs 0.625ns 2.42ns 0.0133 0 0 1.01 KB
#2800 SendReceive net472 1.79μs 0.626ns 2.42ns 0.159 0 0 1.01 KB
#2800 SendReceive netcoreapp3.1 1.53μs 3.24ns 12.1ns 0.0137 0 0 1.01 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.65μs 2.22ns 8.61ns 0.292 0 0 1.87 KB
master EnrichedLog netcoreapp3.1 3.87μs 1.19ns 4.46ns 0.0193 0 0 1.49 KB
#2800 EnrichedLog net472 4.56μs 2.26ns 8.74ns 0.292 0 0 1.87 KB
#2800 EnrichedLog netcoreapp3.1 3.96μs 1.33ns 5.15ns 0.0198 0 0 1.49 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 775ns 0.424ns 1.59ns 0.0718 0 0 457 B
master StartFinishSpan netcoreapp3.1 670ns 0.133ns 0.48ns 0.00604 0 0 456 B
master StartFinishScope net472 998ns 0.269ns 1.01ns 0.0846 0 0 538 B
master StartFinishScope netcoreapp3.1 737ns 0.258ns 0.893ns 0.00773 0 0 576 B
#2800 StartFinishSpan net472 737ns 0.291ns 1.13ns 0.0718 0 0 457 B
#2800 StartFinishSpan netcoreapp3.1 629ns 0.216ns 0.837ns 0.00609 0 0 456 B
#2800 StartFinishScope net472 1.05μs 0.497ns 1.92ns 0.0843 0 0 538 B
#2800 StartFinishScope netcoreapp3.1 736ns 0.309ns 1.15ns 0.00779 0 0 576 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.16μs 0.56ns 2.17ns 0.084 0 0 538 B
master RunOnMethodBegin netcoreapp3.1 920ns 0.203ns 0.759ns 0.00783 0 0 576 B
#2800 RunOnMethodBegin net472 1.12μs 0.245ns 0.849ns 0.0841 0 0 538 B
#2800 RunOnMethodBegin netcoreapp3.1 888ns 0.373ns 1.4ns 0.00802 0 0 576 B

andrewlock avatar May 16 '22 12:05 andrewlock

Can we force agentless telemetry when using ciapp agentless mode?

tonyredondo avatar May 16 '22 13:05 tonyredondo

Can we force agentless telemetry when using ciapp agentless mode?

@tonyredondo Yep, that's what we're doing in https://github.com/DataDog/dd-trace-dotnet/pull/2800/commits/e6da2af1360f593e5ceb2601e3f36bef94dec3e5 🙂

andrewlock avatar May 16 '22 13:05 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 726μs 900ns 3.37μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 454μs 124ns 446ns 0 0 0 2.57 KB
#2800 WriteAndFlushEnrichedTraces net472 728μs 928ns 3.47μs 0 0 0 3.16 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 450μs 224ns 808ns 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 194ns 0.0597ns 0.231ns 0.0661 0 0 417 B
master AllCycleSimpleBody netcoreapp3.1 249ns 0.661ns 2.47ns 0.00581 0 0 416 B
master AllCycleMoreComplexBody net472 193ns 0.0872ns 0.338ns 0.0623 0 0 393 B
master AllCycleMoreComplexBody netcoreapp3.1 245ns 0.191ns 0.741ns 0.0054 0 0 392 B
master BodyExtractorSimpleBody net472 261ns 0.136ns 0.507ns 0.0571 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 233ns 0.511ns 1.98ns 0.00378 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.9μs 5.62ns 21ns 1.19 0.0159 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 13.3μs 7.29ns 28.2ns 0.0926 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 200ns 0.0331ns 0.128ns 0.0661 0 0 417 B
#2800 AllCycleSimpleBody netcoreapp3.1 245ns 0.0423ns 0.158ns 0.00566 0 0 416 B
#2800 AllCycleMoreComplexBody net472 196ns 0.0927ns 0.359ns 0.0623 0 0 393 B
#2800 AllCycleMoreComplexBody netcoreapp3.1 243ns 0.0589ns 0.22ns 0.00541 0 0 392 B
#2800 BodyExtractorSimpleBody net472 276ns 0.136ns 0.526ns 0.0572 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 237ns 0.13ns 0.504ns 0.00369 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 15.7μs 5.04ns 18.8ns 1.19 0.0157 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 13.1μs 5.12ns 19.2ns 0.0917 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 172μs 127ns 491ns 0.258 0 0 19.74 KB
#2800 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 171μs 88.7ns 343ns 0.256 0 0 19.74 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.4μs 0.386ns 1.49ns 0.0939 0.000701 0 594 B
master ExecuteNonQuery netcoreapp3.1 1.16μs 0.357ns 1.38ns 0.0087 0 0 632 B
#2800 ExecuteNonQuery net472 1.45μs 0.602ns 2.25ns 0.0943 0.000725 0 594 B
#2800 ExecuteNonQuery netcoreapp3.1 1.14μs 0.47ns 1.76ns 0.00862 0 0 632 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.03μs 0.605ns 2.26ns 0.125 0 0 802 B
master CallElasticsearch netcoreapp3.1 1.3μs 0.819ns 3.17ns 0.0104 0 0 792 B
master CallElasticsearchAsync net472 2.2μs 0.653ns 2.53ns 0.147 0 0 939 B
master CallElasticsearchAsync netcoreapp3.1 1.39μs 0.201ns 0.753ns 0.0125 0 0 912 B
#2800 CallElasticsearch net472 1.97μs 0.407ns 1.58ns 0.125 0 0 802 B
#2800 CallElasticsearch netcoreapp3.1 1.21μs 1.01ns 3.66ns 0.0109 0 0 792 B
#2800 CallElasticsearchAsync net472 2.18μs 1.01ns 3.93ns 0.146 0 0 939 B
#2800 CallElasticsearchAsync netcoreapp3.1 1.34μs 0.527ns 1.97ns 0.0122 0 0 912 B
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.24μs 0.801ns 3ns 0.165 0 0 1.06 KB
master ExecuteAsync netcoreapp3.1 1.5μs 3.85ns 14.9ns 0.0141 0 0 1.03 KB
#2800 ExecuteAsync net472 2.31μs 0.664ns 2.3ns 0.166 0 0 1.06 KB
#2800 ExecuteAsync netcoreapp3.1 1.42μs 0.866ns 3.24ns 0.0142 0 0 1.03 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.78μs 0.989ns 3.83ns 0.358 0 0 2.28 KB
master SendAsync netcoreapp3.1 3.23μs 0.858ns 3.21ns 0.0293 0 0 2.21 KB
#2800 SendAsync net472 4.86μs 2.46ns 9.22ns 0.357 0 0 2.28 KB
#2800 SendAsync netcoreapp3.1 3.21μs 1.07ns 4.14ns 0.0305 0 0 2.21 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.67μs 1.6ns 6.19ns 0.228 0 0 1.45 KB
master EnrichedLog netcoreapp3.1 2.28μs 0.881ns 3.3ns 0.0205 0 0 1.53 KB
#2800 EnrichedLog net472 2.79μs 2.03ns 7.86ns 0.227 0 0 1.45 KB
#2800 EnrichedLog netcoreapp3.1 2.16μs 0.707ns 2.55ns 0.0202 0 0 1.53 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 149μs 219ns 847ns 0.518 0.148 0 4.29 KB
master EnrichedLog netcoreapp3.1 111μs 189ns 732ns 0.0558 0 0 4.18 KB
#2800 EnrichedLog net472 147μs 103ns 400ns 0.515 0.147 0 4.29 KB
#2800 EnrichedLog netcoreapp3.1 111μs 224ns 868ns 0.0556 0 0 4.18 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.22μs 1.81ns 7.02ns 0.508 0.00262 0 3.23 KB
master EnrichedLog netcoreapp3.1 4.07μs 1.76ns 6.8ns 0.0489 0 0 3.6 KB
#2800 EnrichedLog net472 5.32μs 2.33ns 8.73ns 0.507 0.00265 0 3.23 KB
#2800 EnrichedLog netcoreapp3.1 4.11μs 1.59ns 6.17ns 0.0472 0 0 3.6 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.89μs 0.816ns 3.16ns 0.158 0 0 1.01 KB
master SendReceive netcoreapp3.1 1.56μs 0.625ns 2.42ns 0.0133 0 0 1.01 KB
#2800 SendReceive net472 1.84μs 0.731ns 2.83ns 0.159 0 0 1.01 KB
#2800 SendReceive netcoreapp3.1 1.52μs 0.808ns 3.02ns 0.0137 0 0 1.01 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.65μs 2.22ns 8.61ns 0.292 0 0 1.87 KB
master EnrichedLog netcoreapp3.1 3.87μs 1.19ns 4.46ns 0.0193 0 0 1.49 KB
#2800 EnrichedLog net472 4.55μs 1.79ns 6.95ns 0.292 0 0 1.87 KB
#2800 EnrichedLog netcoreapp3.1 3.9μs 0.978ns 3.66ns 0.0195 0 0 1.49 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 775ns 0.424ns 1.59ns 0.0718 0 0 457 B
master StartFinishSpan netcoreapp3.1 670ns 0.133ns 0.48ns 0.00604 0 0 456 B
master StartFinishScope net472 998ns 0.269ns 1.01ns 0.0846 0 0 538 B
master StartFinishScope netcoreapp3.1 737ns 0.258ns 0.893ns 0.00773 0 0 576 B
#2800 StartFinishSpan net472 716ns 0.378ns 1.42ns 0.0719 0 0 457 B
#2800 StartFinishSpan netcoreapp3.1 629ns 0.148ns 0.535ns 0.006 0 0 456 B
#2800 StartFinishScope net472 1.05μs 0.653ns 2.44ns 0.0845 0 0 538 B
#2800 StartFinishScope netcoreapp3.1 788ns 0.294ns 1.06ns 0.00792 0 0 576 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.16μs 0.56ns 2.17ns 0.084 0 0 538 B
master RunOnMethodBegin netcoreapp3.1 920ns 0.203ns 0.759ns 0.00783 0 0 576 B
#2800 RunOnMethodBegin net472 1.18μs 1.3ns 5.02ns 0.0841 0 0 538 B
#2800 RunOnMethodBegin netcoreapp3.1 896ns 0.245ns 0.883ns 0.00758 0 0 576 B

andrewlock avatar May 16 '22 13:05 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2800 into master will not change line coverage :heavy_check_mark: Merging #2800 into master will not change branch coverage :no_entry: Merging #2800 into master will will increase complexity by 67

master #2800 Change
Lines 14125 / 19513 14190 / 19570
Lines % 72% 73% 0% :heavy_check_mark:
Branches 8261 / 11859 8312 / 11916
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 13134 13201 67 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2800 Change
Lines % 72% 73% 0% :heavy_check_mark:
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 13134 13201 67 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.Processors.OriginTagTraceProcessor -9% :no_entry: -17% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.Ci.GitInfo 3% :heavy_check_mark: 6% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Agent.Transports.SocketHandlerRequestFactory 7% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Agent.Transports.HttpStreamRequestFactory 12% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:

The following classes were added in #2800:

File Line coverage Branch coverage Complexity
Datadog.Trace.HttpOverStreams.TelemetryAgentHttpHeaderHelper 80% 100% 3
Datadog.Trace.Telemetry.TelemetryTransportManager 100% 96% 26
Datadog.Trace.Telemetry.Transports.AgentlessTelemetryTransport 75% 100% 3
Datadog.Trace.Telemetry.Transports.AgentTelemetryTransport 100% 100% 15
Datadog.Trace.Telemetry.Transports.JsonTelemetryTransport 63% 25% 22
...And 2 more

5 classes were removed from Datadog.Trace in #2800

View the full reports for further details:

andrewlock avatar May 16 '22 13:05 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 728μs 671ns 2.6μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 468μs 133ns 516ns 0 0 0 2.57 KB
#2800 WriteAndFlushEnrichedTraces net472 736μs 707ns 2.74μs 0 0 0 3.16 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 462μs 116ns 417ns 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 196ns 0.0533ns 0.199ns 0.0661 0 0 417 B
master AllCycleSimpleBody netcoreapp3.1 246ns 0.0624ns 0.234ns 0.0057 0 0 416 B
master AllCycleMoreComplexBody net472 194ns 0.0226ns 0.0784ns 0.0622 0 0 393 B
master AllCycleMoreComplexBody netcoreapp3.1 243ns 0.165ns 0.594ns 0.00539 0 0 392 B
master BodyExtractorSimpleBody net472 268ns 0.193ns 0.722ns 0.0571 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 238ns 0.0666ns 0.249ns 0.00371 0 0 272 B
master BodyExtractorMoreComplexBody net472 16μs 7.49ns 29ns 1.19 0.0159 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 13.2μs 5.82ns 22.5ns 0.0922 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 194ns 0.0738ns 0.286ns 0.0661 0 0 417 B
#2800 AllCycleSimpleBody netcoreapp3.1 246ns 0.0935ns 0.35ns 0.00567 0 0 416 B
#2800 AllCycleMoreComplexBody net472 195ns 0.0266ns 0.096ns 0.0622 0 0 393 B
#2800 AllCycleMoreComplexBody netcoreapp3.1 243ns 0.264ns 1.02ns 0.00535 0 0 392 B
#2800 BodyExtractorSimpleBody net472 269ns 0.0879ns 0.34ns 0.0571 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 234ns 0.075ns 0.27ns 0.00382 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 15.7μs 6.53ns 25.3ns 1.19 0.0157 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 13μs 3.71ns 13.9ns 0.0905 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 171μs 85.9ns 333ns 0.257 0 0 19.74 KB
#2800 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 173μs 146ns 544ns 0.173 0 0 19.74 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.43μs 2.08ns 7.79ns 0.0938 0.00071 0 594 B
master ExecuteNonQuery netcoreapp3.1 1.17μs 0.315ns 1.22ns 0.00871 0 0 632 B
#2800 ExecuteNonQuery net472 1.31μs 0.388ns 1.5ns 0.0939 0.000652 0 594 B
#2800 ExecuteNonQuery netcoreapp3.1 1.18μs 0.713ns 2.67ns 0.00827 0 0 632 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2μs 0.942ns 3.65ns 0.125 0 0 802 B
master CallElasticsearch netcoreapp3.1 1.26μs 0.434ns 1.56ns 0.0107 0 0 792 B
master CallElasticsearchAsync net472 2.16μs 0.887ns 3.2ns 0.147 0 0 939 B
master CallElasticsearchAsync netcoreapp3.1 1.32μs 0.344ns 1.24ns 0.0126 0 0 912 B
#2800 CallElasticsearch net472 2.03μs 0.478ns 1.72ns 0.125 0 0 802 B
#2800 CallElasticsearch netcoreapp3.1 1.3μs 0.685ns 2.65ns 0.0106 0 0 792 B
#2800 CallElasticsearchAsync net472 2.15μs 0.875ns 3.28ns 0.147 0 0 939 B
#2800 CallElasticsearchAsync netcoreapp3.1 1.3μs 0.505ns 1.89ns 0.0118 0 0 912 B
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.34μs 1.1ns 4.27ns 0.166 0 0 1.06 KB
master ExecuteAsync netcoreapp3.1 1.47μs 0.585ns 2.26ns 0.0138 0 0 1.03 KB
#2800 ExecuteAsync net472 2.34μs 1.26ns 4.87ns 0.166 0 0 1.06 KB
#2800 ExecuteAsync netcoreapp3.1 1.44μs 0.228ns 0.855ns 0.0136 0 0 1.03 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.94μs 1.99ns 6.9ns 0.357 0 0 2.28 KB
master SendAsync netcoreapp3.1 3.19μs 1.33ns 4.99ns 0.0303 0 0 2.21 KB
#2800 SendAsync net472 4.81μs 2.03ns 7.87ns 0.358 0 0 2.28 KB
#2800 SendAsync netcoreapp3.1 3.15μs 0.975ns 3.65ns 0.0301 0 0 2.21 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.79μs 1.49ns 5.58ns 0.228 0 0 1.45 KB
master EnrichedLog netcoreapp3.1 2.1μs 1.75ns 6.8ns 0.0209 0 0 1.53 KB
#2800 EnrichedLog net472 2.7μs 1.37ns 5.29ns 0.228 0 0 1.45 KB
#2800 EnrichedLog netcoreapp3.1 2.2μs 0.839ns 3.25ns 0.0208 0 0 1.53 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 148μs 190ns 737ns 0.513 0.147 0 4.29 KB
master EnrichedLog netcoreapp3.1 114μs 194ns 753ns 0.0564 0 0 4.18 KB
#2800 EnrichedLog net472 149μs 131ns 508ns 0.52 0.149 0 4.29 KB
#2800 EnrichedLog netcoreapp3.1 112μs 127ns 494ns 0 0 0 4.18 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.15μs 1.67ns 6.46ns 0.508 0.00258 0 3.23 KB
master EnrichedLog netcoreapp3.1 4.19μs 2.03ns 7.85ns 0.0481 0 0 3.6 KB
#2800 EnrichedLog net472 5.26μs 1.54ns 5.76ns 0.506 0.00262 0 3.23 KB
#2800 EnrichedLog netcoreapp3.1 4μs 1.23ns 4.76ns 0.0499 0 0 3.6 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.92μs 3.48ns 13.5ns 0.158 0 0 1.01 KB
master SendReceive netcoreapp3.1 1.52μs 0.501ns 1.94ns 0.0136 0 0 1.01 KB
#2800 SendReceive net472 1.85μs 0.688ns 2.66ns 0.159 0 0 1.01 KB
#2800 SendReceive netcoreapp3.1 1.5μs 0.455ns 1.76ns 0.0135 0 0 1.01 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.62μs 2.42ns 9.35ns 0.293 0 0 1.87 KB
master EnrichedLog netcoreapp3.1 4.02μs 1.32ns 5.1ns 0.0201 0 0 1.49 KB
#2800 EnrichedLog net472 4.55μs 2.52ns 9.75ns 0.292 0 0 1.87 KB
#2800 EnrichedLog netcoreapp3.1 3.97μs 1.15ns 4.29ns 0.0199 0 0 1.49 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 842ns 0.632ns 2.45ns 0.0719 0 0 457 B
master StartFinishSpan netcoreapp3.1 659ns 0.341ns 1.23ns 0.00629 0 0 456 B
master StartFinishScope net472 1.02μs 2.12ns 8.21ns 0.0844 0 0 538 B
master StartFinishScope netcoreapp3.1 802ns 0.837ns 3.24ns 0.00765 0 0 576 B
#2800 StartFinishSpan net472 774ns 0.324ns 1.21ns 0.072 0 0 457 B
#2800 StartFinishSpan netcoreapp3.1 624ns 0.122ns 0.471ns 0.00624 0 0 456 B
#2800 StartFinishScope net472 1.01μs 0.296ns 1.15ns 0.0843 0 0 538 B
#2800 StartFinishScope netcoreapp3.1 815ns 0.454ns 1.7ns 0.00812 0 0 576 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.18μs 0.608ns 2.27ns 0.0844 0 0 538 B
master RunOnMethodBegin netcoreapp3.1 875ns 0.437ns 1.57ns 0.00787 0 0 576 B
#2800 RunOnMethodBegin net472 1.17μs 0.636ns 2.38ns 0.0843 0 0 538 B
#2800 RunOnMethodBegin netcoreapp3.1 868ns 0.336ns 1.21ns 0.00779 0 0 576 B

andrewlock avatar May 16 '22 14:05 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2800 into master will not change line coverage :heavy_check_mark: Merging #2800 into master will not change branch coverage :no_entry: Merging #2800 into master will will increase complexity by 49

master #2800 Change
Lines 14102 / 19513 14149 / 19555
Lines % 72% 72% 0% :heavy_check_mark:
Branches 8248 / 11859 8306 / 11900
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 13134 13183 49 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2800 Change
Lines % 72% 72% 0% :heavy_check_mark:
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 13134 13183 49 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.GitInfo -17% :no_entry: -11% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.Ci.Processors.OriginTagTraceProcessor -9% :no_entry: -17% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.Agent.Transports.SocketHandlerRequestFactory 7% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Agent.Transports.HttpStreamRequestFactory 12% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:

The following classes were added in #2800:

File Line coverage Branch coverage Complexity
Datadog.Trace.HttpOverStreams.TelemetryAgentHttpHeaderHelper 80% 100% 3
Datadog.Trace.Telemetry.TelemetryTransportManager 100% 96% 24
Datadog.Trace.Telemetry.Transports.AgentlessTelemetryTransport 100% 100% 2
Datadog.Trace.Telemetry.Transports.AgentTelemetryTransport 100% 100% 2
Datadog.Trace.Telemetry.Transports.JsonTelemetryTransport 92% 62% 20
...And 2 more

5 classes were removed from Datadog.Trace in #2800

View the full reports for further details:

andrewlock avatar May 16 '22 15:05 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 725μs 391ns 1.52μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 484μs 547ns 2.12μs 0 0 0 2.57 KB
#2800 WriteAndFlushEnrichedTraces net472 728μs 674ns 2.61μs 0 0 0 3.16 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 461μs 338ns 1.26μs 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 197ns 0.0613ns 0.237ns 0.0661 0 0 417 B
master AllCycleSimpleBody netcoreapp3.1 245ns 0.125ns 0.467ns 0.0058 0 0 416 B
master AllCycleMoreComplexBody net472 193ns 0.0523ns 0.196ns 0.0622 0 0 393 B
master AllCycleMoreComplexBody netcoreapp3.1 244ns 0.109ns 0.422ns 0.00545 0 0 392 B
master BodyExtractorSimpleBody net472 262ns 0.143ns 0.555ns 0.0571 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 236ns 0.158ns 0.593ns 0.00369 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.6μs 6.23ns 24.1ns 1.19 0.0156 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 13.2μs 3.82ns 14.8ns 0.0921 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 194ns 0.054ns 0.209ns 0.0661 0 0 417 B
#2800 AllCycleSimpleBody netcoreapp3.1 246ns 0.246ns 0.919ns 0.00582 0 0 416 B
#2800 AllCycleMoreComplexBody net472 193ns 0.0278ns 0.1ns 0.0622 0 0 393 B
#2800 AllCycleMoreComplexBody netcoreapp3.1 243ns 0.114ns 0.426ns 0.00536 0 0 392 B
#2800 BodyExtractorSimpleBody net472 266ns 0.126ns 0.472ns 0.0571 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 235ns 0.0585ns 0.219ns 0.00371 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 16.6μs 4.57ns 16.5ns 1.19 0.0166 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 13.6μs 3.74ns 13.5ns 0.0885 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 172μs 72.9ns 282ns 0.257 0 0 19.74 KB
#2800 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 173μs 134ns 517ns 0.172 0 0 19.74 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.4μs 0.443ns 1.72ns 0.094 0.000701 0 594 B
master ExecuteNonQuery netcoreapp3.1 1.11μs 0.575ns 2.15ns 0.00834 0 0 632 B
#2800 ExecuteNonQuery net472 1.49μs 0.617ns 2.39ns 0.0938 0.000744 0 594 B
#2800 ExecuteNonQuery netcoreapp3.1 1.16μs 0.8ns 2.99ns 0.00809 0 0 632 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 1.97μs 0.337ns 1.26ns 0.126 0 0 802 B
master CallElasticsearch netcoreapp3.1 1.22μs 0.456ns 1.71ns 0.0104 0 0 792 B
master CallElasticsearchAsync net472 2.23μs 0.914ns 3.54ns 0.146 0 0 939 B
master CallElasticsearchAsync netcoreapp3.1 1.29μs 0.566ns 2.19ns 0.0123 0 0 912 B
#2800 CallElasticsearch net472 2.07μs 0.376ns 1.41ns 0.125 0 0 802 B
#2800 CallElasticsearch netcoreapp3.1 1.27μs 0.423ns 1.64ns 0.0107 0 0 792 B
#2800 CallElasticsearchAsync net472 2.31μs 0.692ns 2.59ns 0.147 0 0 939 B
#2800 CallElasticsearchAsync netcoreapp3.1 1.29μs 0.444ns 1.66ns 0.0123 0 0 912 B
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.29μs 1.29ns 5.01ns 0.166 0 0 1.06 KB
master ExecuteAsync netcoreapp3.1 1.41μs 0.226ns 0.814ns 0.0141 0 0 1.03 KB
#2800 ExecuteAsync net472 2.26μs 3.54ns 13.3ns 0.166 0 0 1.06 KB
#2800 ExecuteAsync netcoreapp3.1 1.46μs 1.94ns 7.25ns 0.0137 0 0 1.03 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.84μs 1.79ns 6.94ns 0.358 0 0 2.28 KB
master SendAsync netcoreapp3.1 3.21μs 0.791ns 2.85ns 0.0305 0 0 2.21 KB
#2800 SendAsync net472 4.74μs 0.999ns 3.87ns 0.357 0 0 2.28 KB
#2800 SendAsync netcoreapp3.1 3.25μs 1.42ns 5.48ns 0.0292 0 0 2.21 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.84μs 6.44ns 24.1ns 0.227 0 0 1.45 KB
master EnrichedLog netcoreapp3.1 2.15μs 0.633ns 2.37ns 0.0213 0 0 1.53 KB
#2800 EnrichedLog net472 2.75μs 2.56ns 9.56ns 0.228 0 0 1.45 KB
#2800 EnrichedLog netcoreapp3.1 2.15μs 0.83ns 3.11ns 0.0204 0 0 1.53 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 146μs 248ns 962ns 0.515 0.147 0 4.29 KB
master EnrichedLog netcoreapp3.1 112μs 147ns 568ns 0.0556 0 0 4.18 KB
#2800 EnrichedLog net472 148μs 73.1ns 264ns 0.52 0.148 0 4.29 KB
#2800 EnrichedLog netcoreapp3.1 112μs 105ns 380ns 0.0547 0 0 4.18 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.4μs 2.9ns 11.2ns 0.507 0.0027 0 3.23 KB
master EnrichedLog netcoreapp3.1 4.13μs 1.3ns 5.03ns 0.0496 0 0 3.6 KB
#2800 EnrichedLog net472 5.26μs 2.31ns 8.62ns 0.508 0.00262 0 3.23 KB
#2800 EnrichedLog netcoreapp3.1 4.27μs 1.48ns 5.73ns 0.0491 0 0 3.6 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.83μs 1.04ns 4.02ns 0.159 0 0 1.01 KB
master SendReceive netcoreapp3.1 1.57μs 0.308ns 1.07ns 0.0142 0 0 1.01 KB
#2800 SendReceive net472 1.84μs 0.389ns 1.4ns 0.159 0 0 1.01 KB
#2800 SendReceive netcoreapp3.1 1.51μs 0.51ns 1.91ns 0.0137 0 0 1.01 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.63μs 3.77ns 14.6ns 0.292 0 0 1.87 KB
master EnrichedLog netcoreapp3.1 3.83μs 2.02ns 7.55ns 0.0191 0 0 1.49 KB
#2800 EnrichedLog net472 4.65μs 2.01ns 7.8ns 0.291 0 0 1.87 KB
#2800 EnrichedLog netcoreapp3.1 3.92μs 1.98ns 7.68ns 0.0195 0 0 1.49 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 816ns 0.928ns 3.59ns 0.0719 0 0 457 B
master StartFinishSpan netcoreapp3.1 656ns 0.228ns 0.855ns 0.00629 0 0 456 B
master StartFinishScope net472 1.04μs 0.921ns 3.19ns 0.0845 0 0 538 B
master StartFinishScope netcoreapp3.1 744ns 0.305ns 1.14ns 0.00748 0 0 576 B
#2800 StartFinishSpan net472 779ns 0.304ns 1.14ns 0.0718 0 0 457 B
#2800 StartFinishSpan netcoreapp3.1 651ns 0.159ns 0.593ns 0.00619 0 0 456 B
#2800 StartFinishScope net472 979ns 0.774ns 3ns 0.0842 0 0 538 B
#2800 StartFinishScope netcoreapp3.1 783ns 0.341ns 1.28ns 0.00751 0 0 576 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.15μs 0.839ns 3.25ns 0.0839 0 0 538 B
master RunOnMethodBegin netcoreapp3.1 917ns 1.38ns 5.36ns 0.00771 0 0 576 B
#2800 RunOnMethodBegin net472 1.13μs 0.226ns 0.877ns 0.0841 0 0 538 B
#2800 RunOnMethodBegin netcoreapp3.1 889ns 0.711ns 2.76ns 0.0075 0 0 576 B

andrewlock avatar May 17 '22 10:05 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 725μs 391ns 1.52μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 484μs 547ns 2.12μs 0 0 0 2.57 KB
#2800 WriteAndFlushEnrichedTraces net472 737μs 447ns 1.55μs 0 0 0 3.16 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 469μs 201ns 753ns 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 197ns 0.0613ns 0.237ns 0.0661 0 0 417 B
master AllCycleSimpleBody netcoreapp3.1 245ns 0.125ns 0.467ns 0.0058 0 0 416 B
master AllCycleMoreComplexBody net472 193ns 0.0523ns 0.196ns 0.0622 0 0 393 B
master AllCycleMoreComplexBody netcoreapp3.1 244ns 0.109ns 0.422ns 0.00545 0 0 392 B
master BodyExtractorSimpleBody net472 262ns 0.143ns 0.555ns 0.0571 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 236ns 0.158ns 0.593ns 0.00369 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.6μs 6.23ns 24.1ns 1.19 0.0156 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 13.2μs 3.82ns 14.8ns 0.0921 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 195ns 0.0662ns 0.229ns 0.0661 0 0 417 B
#2800 AllCycleSimpleBody netcoreapp3.1 246ns 0.113ns 0.423ns 0.00568 0 0 416 B
#2800 AllCycleMoreComplexBody net472 192ns 0.0366ns 0.142ns 0.0622 0 0 393 B
#2800 AllCycleMoreComplexBody netcoreapp3.1 244ns 0.117ns 0.455ns 0.00541 0 0 392 B
#2800 BodyExtractorSimpleBody net472 270ns 0.221ns 0.857ns 0.0571 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 235ns 0.0816ns 0.305ns 0.00367 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 15.8μs 12.9ns 50ns 1.19 0.0158 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 13.7μs 3.61ns 13.5ns 0.0887 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 172μs 72.9ns 282ns 0.257 0 0 19.74 KB
#2800 SendRequest net472 0.00141ns 0.000962ns 0.0036ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 171μs 166ns 644ns 0.255 0 0 19.74 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.4μs 0.443ns 1.72ns 0.094 0.000701 0 594 B
master ExecuteNonQuery netcoreapp3.1 1.11μs 0.575ns 2.15ns 0.00834 0 0 632 B
#2800 ExecuteNonQuery net472 1.3μs 0.872ns 3.38ns 0.0939 0.000652 0 594 B
#2800 ExecuteNonQuery netcoreapp3.1 1.16μs 0.587ns 2.2ns 0.00872 0 0 632 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 1.97μs 0.337ns 1.26ns 0.126 0 0 802 B
master CallElasticsearch netcoreapp3.1 1.22μs 0.456ns 1.71ns 0.0104 0 0 792 B
master CallElasticsearchAsync net472 2.23μs 0.914ns 3.54ns 0.146 0 0 939 B
master CallElasticsearchAsync netcoreapp3.1 1.29μs 0.566ns 2.19ns 0.0123 0 0 912 B
#2800 CallElasticsearch net472 2.05μs 1.34ns 5.18ns 0.125 0 0 802 B
#2800 CallElasticsearch netcoreapp3.1 1.28μs 0.292ns 1.09ns 0.0109 0 0 792 B
#2800 CallElasticsearchAsync net472 2.13μs 0.463ns 1.79ns 0.147 0 0 939 B
#2800 CallElasticsearchAsync netcoreapp3.1 1.32μs 0.646ns 2.33ns 0.0119 0 0 912 B
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.29μs 1.29ns 5.01ns 0.166 0 0 1.06 KB
master ExecuteAsync netcoreapp3.1 1.41μs 0.226ns 0.814ns 0.0141 0 0 1.03 KB
#2800 ExecuteAsync net472 2.21μs 0.674ns 2.61ns 0.166 0 0 1.06 KB
#2800 ExecuteAsync netcoreapp3.1 1.44μs 0.557ns 2.16ns 0.0138 0 0 1.03 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.84μs 1.79ns 6.94ns 0.358 0 0 2.28 KB
master SendAsync netcoreapp3.1 3.21μs 0.791ns 2.85ns 0.0305 0 0 2.21 KB
#2800 SendAsync net472 4.93μs 1.42ns 4.91ns 0.357 0 0 2.28 KB
#2800 SendAsync netcoreapp3.1 3.2μs 1.73ns 6.24ns 0.0303 0 0 2.21 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.84μs 6.44ns 24.1ns 0.227 0 0 1.45 KB
master EnrichedLog netcoreapp3.1 2.15μs 0.633ns 2.37ns 0.0213 0 0 1.53 KB
#2800 EnrichedLog net472 2.75μs 1.26ns 4.87ns 0.228 0 0 1.45 KB
#2800 EnrichedLog netcoreapp3.1 2.22μs 1.07ns 4.13ns 0.021 0 0 1.53 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 146μs 248ns 962ns 0.515 0.147 0 4.29 KB
master EnrichedLog netcoreapp3.1 112μs 147ns 568ns 0.0556 0 0 4.18 KB
#2800 EnrichedLog net472 149μs 76.7ns 297ns 0.52 0.148 0 4.29 KB
#2800 EnrichedLog netcoreapp3.1 112μs 86.2ns 311ns 0.0559 0 0 4.18 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.4μs 2.9ns 11.2ns 0.507 0.0027 0 3.23 KB
master EnrichedLog netcoreapp3.1 4.13μs 1.3ns 5.03ns 0.0496 0 0 3.6 KB
#2800 EnrichedLog net472 5.29μs 2.03ns 7.88ns 0.507 0.00264 0 3.23 KB
#2800 EnrichedLog netcoreapp3.1 3.98μs 0.581ns 2.09ns 0.0496 0 0 3.6 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.83μs 1.04ns 4.02ns 0.159 0 0 1.01 KB
master SendReceive netcoreapp3.1 1.57μs 0.308ns 1.07ns 0.0142 0 0 1.01 KB
#2800 SendReceive net472 1.78μs 1.87ns 6.75ns 0.159 0 0 1.01 KB
#2800 SendReceive netcoreapp3.1 1.58μs 0.665ns 2.58ns 0.0141 0 0 1.01 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.63μs 3.77ns 14.6ns 0.292 0 0 1.87 KB
master EnrichedLog netcoreapp3.1 3.83μs 2.02ns 7.55ns 0.0191 0 0 1.49 KB
#2800 EnrichedLog net472 4.66μs 2.42ns 9.39ns 0.292 0 0 1.87 KB
#2800 EnrichedLog netcoreapp3.1 3.86μs 1.55ns 6.01ns 0.0194 0 0 1.49 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 816ns 0.928ns 3.59ns 0.0719 0 0 457 B
master StartFinishSpan netcoreapp3.1 656ns 0.228ns 0.855ns 0.00629 0 0 456 B
master StartFinishScope net472 1.04μs 0.921ns 3.19ns 0.0845 0 0 538 B
master StartFinishScope netcoreapp3.1 744ns 0.305ns 1.14ns 0.00748 0 0 576 B
#2800 StartFinishSpan net472 754ns 0.268ns 0.929ns 0.0717 0 0 457 B
#2800 StartFinishSpan netcoreapp3.1 644ns 0.265ns 1.03ns 0.0061 0 0 456 B
#2800 StartFinishScope net472 1.01μs 0.625ns 2.42ns 0.0845 0 0 538 B
#2800 StartFinishScope netcoreapp3.1 781ns 0.272ns 1.06ns 0.00786 0 0 576 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.15μs 0.839ns 3.25ns 0.0839 0 0 538 B
master RunOnMethodBegin netcoreapp3.1 917ns 1.38ns 5.36ns 0.00771 0 0 576 B
#2800 RunOnMethodBegin net472 1.17μs 0.255ns 0.953ns 0.0843 0 0 538 B
#2800 RunOnMethodBegin netcoreapp3.1 863ns 0.793ns 2.86ns 0.00774 0 0 576 B

andrewlock avatar May 17 '22 11:05 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2800 into master will not change line coverage :heavy_check_mark: Merging #2800 into master will not change branch coverage :no_entry: Merging #2800 into master will will increase complexity by 49

master #2800 Change
Lines 14102 / 19513 14208 / 19555
Lines % 72% 73% 0% :heavy_check_mark:
Branches 8248 / 11859 8316 / 11900
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 13134 13183 49 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2800 Change
Lines % 72% 73% 0% :heavy_check_mark:
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 13134 13183 49 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.Processors.OriginTagTraceProcessor -9% :no_entry: -17% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.Agent.Transports.SocketHandlerRequestFactory 7% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Agent.Transports.HttpStreamRequestFactory 12% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:

The following classes were added in #2800:

File Line coverage Branch coverage Complexity
Datadog.Trace.HttpOverStreams.TelemetryAgentHttpHeaderHelper 80% 100% 3
Datadog.Trace.Telemetry.TelemetryTransportManager 100% 96% 24
Datadog.Trace.Telemetry.Transports.AgentlessTelemetryTransport 100% 100% 2
Datadog.Trace.Telemetry.Transports.AgentTelemetryTransport 100% 100% 2
Datadog.Trace.Telemetry.Transports.JsonTelemetryTransport 92% 54% 20
...And 2 more

5 classes were removed from Datadog.Trace in #2800

View the full reports for further details:

andrewlock avatar May 17 '22 13:05 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 725μs 391ns 1.52μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 484μs 547ns 2.12μs 0 0 0 2.57 KB
#2800 WriteAndFlushEnrichedTraces net472 730μs 881ns 3.41μs 0 0 0 3.16 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 454μs 271ns 1.05μs 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 197ns 0.0613ns 0.237ns 0.0661 0 0 417 B
master AllCycleSimpleBody netcoreapp3.1 245ns 0.125ns 0.467ns 0.0058 0 0 416 B
master AllCycleMoreComplexBody net472 193ns 0.0523ns 0.196ns 0.0622 0 0 393 B
master AllCycleMoreComplexBody netcoreapp3.1 244ns 0.109ns 0.422ns 0.00545 0 0 392 B
master BodyExtractorSimpleBody net472 262ns 0.143ns 0.555ns 0.0571 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 236ns 0.158ns 0.593ns 0.00369 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.6μs 6.23ns 24.1ns 1.19 0.0156 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 13.2μs 3.82ns 14.8ns 0.0921 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 195ns 0.0394ns 0.153ns 0.0661 0 0 417 B
#2800 AllCycleSimpleBody netcoreapp3.1 246ns 0.0607ns 0.21ns 0.00583 0 0 416 B
#2800 AllCycleMoreComplexBody net472 195ns 0.0381ns 0.147ns 0.0623 0 0 393 B
#2800 AllCycleMoreComplexBody netcoreapp3.1 243ns 0.171ns 0.663ns 0.00537 0 0 392 B
#2800 BodyExtractorSimpleBody net472 275ns 0.182ns 0.705ns 0.0572 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 235ns 0.0989ns 0.37ns 0.00377 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 15.6μs 6.77ns 26.2ns 1.19 0.0156 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 12.9μs 5.12ns 19.8ns 0.0902 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 172μs 72.9ns 282ns 0.257 0 0 19.74 KB
#2800 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 173μs 109ns 424ns 0.172 0 0 19.74 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.4μs 0.443ns 1.72ns 0.094 0.000701 0 594 B
master ExecuteNonQuery netcoreapp3.1 1.11μs 0.575ns 2.15ns 0.00834 0 0 632 B
#2800 ExecuteNonQuery net472 1.39μs 0.595ns 2.3ns 0.0942 0.000698 0 594 B
#2800 ExecuteNonQuery netcoreapp3.1 1.18μs 0.313ns 1.21ns 0.0082 0 0 632 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 1.97μs 0.337ns 1.26ns 0.126 0 0 802 B
master CallElasticsearch netcoreapp3.1 1.22μs 0.456ns 1.71ns 0.0104 0 0 792 B
master CallElasticsearchAsync net472 2.23μs 0.914ns 3.54ns 0.146 0 0 939 B
master CallElasticsearchAsync netcoreapp3.1 1.29μs 0.566ns 2.19ns 0.0123 0 0 912 B
#2800 CallElasticsearch net472 2.02μs 0.76ns 2.94ns 0.125 0 0 802 B
#2800 CallElasticsearch netcoreapp3.1 1.28μs 0.682ns 2.64ns 0.0109 0 0 792 B
#2800 CallElasticsearchAsync net472 2.22μs 1.74ns 6.04ns 0.147 0 0 939 B
#2800 CallElasticsearchAsync netcoreapp3.1 1.37μs 0.459ns 1.65ns 0.0124 0 0 912 B
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.29μs 1.29ns 5.01ns 0.166 0 0 1.06 KB
master ExecuteAsync netcoreapp3.1 1.41μs 0.226ns 0.814ns 0.0141 0 0 1.03 KB
#2800 ExecuteAsync net472 2.24μs 0.497ns 1.79ns 0.166 0 0 1.06 KB
#2800 ExecuteAsync netcoreapp3.1 1.43μs 0.416ns 1.56ns 0.0143 0 0 1.03 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.84μs 1.79ns 6.94ns 0.358 0 0 2.28 KB
master SendAsync netcoreapp3.1 3.21μs 0.791ns 2.85ns 0.0305 0 0 2.21 KB
#2800 SendAsync net472 4.97μs 1.05ns 3.77ns 0.357 0 0 2.28 KB
#2800 SendAsync netcoreapp3.1 3.25μs 1.08ns 3.75ns 0.0308 0 0 2.21 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.84μs 6.44ns 24.1ns 0.227 0 0 1.45 KB
master EnrichedLog netcoreapp3.1 2.15μs 0.633ns 2.37ns 0.0213 0 0 1.53 KB
#2800 EnrichedLog net472 2.65μs 1.29ns 5.01ns 0.228 0 0 1.45 KB
#2800 EnrichedLog netcoreapp3.1 2.12μs 1.08ns 4.18ns 0.0201 0 0 1.53 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 146μs 248ns 962ns 0.515 0.147 0 4.29 KB
master EnrichedLog netcoreapp3.1 112μs 147ns 568ns 0.0556 0 0 4.18 KB
#2800 EnrichedLog net472 148μs 215ns 774ns 0.515 0.147 0 4.29 KB
#2800 EnrichedLog netcoreapp3.1 110μs 176ns 660ns 0.0552 0 0 4.18 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.4μs 2.9ns 11.2ns 0.507 0.0027 0 3.23 KB
master EnrichedLog netcoreapp3.1 4.13μs 1.3ns 5.03ns 0.0496 0 0 3.6 KB
#2800 EnrichedLog net472 5.28μs 2.95ns 11.4ns 0.507 0.00264 0 3.23 KB
#2800 EnrichedLog netcoreapp3.1 4.25μs 4.99ns 18ns 0.0485 0 0 3.6 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.83μs 1.04ns 4.02ns 0.159 0 0 1.01 KB
master SendReceive netcoreapp3.1 1.57μs 0.308ns 1.07ns 0.0142 0 0 1.01 KB
#2800 SendReceive net472 1.85μs 0.848ns 3.17ns 0.158 0 0 1.01 KB
#2800 SendReceive netcoreapp3.1 1.53μs 0.652ns 2.52ns 0.0136 0 0 1.01 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.63μs 3.77ns 14.6ns 0.292 0 0 1.87 KB
master EnrichedLog netcoreapp3.1 3.83μs 2.02ns 7.55ns 0.0191 0 0 1.49 KB
#2800 EnrichedLog net472 4.6μs 1.65ns 6.18ns 0.292 0 0 1.87 KB
#2800 EnrichedLog netcoreapp3.1 3.77μs 0.9ns 3.37ns 0.0189 0 0 1.49 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 816ns 0.928ns 3.59ns 0.0719 0 0 457 B
master StartFinishSpan netcoreapp3.1 656ns 0.228ns 0.855ns 0.00629 0 0 456 B
master StartFinishScope net472 1.04μs 0.921ns 3.19ns 0.0845 0 0 538 B
master StartFinishScope netcoreapp3.1 744ns 0.305ns 1.14ns 0.00748 0 0 576 B
#2800 StartFinishSpan net472 758ns 0.845ns 3.27ns 0.072 0 0 457 B
#2800 StartFinishSpan netcoreapp3.1 632ns 0.131ns 0.452ns 0.006 0 0 456 B
#2800 StartFinishScope net472 1.09μs 0.89ns 3.45ns 0.0843 0 0 538 B
#2800 StartFinishScope netcoreapp3.1 780ns 0.252ns 0.908ns 0.00769 0 0 576 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.15μs 0.839ns 3.25ns 0.0839 0 0 538 B
master RunOnMethodBegin netcoreapp3.1 917ns 1.38ns 5.36ns 0.00771 0 0 576 B
#2800 RunOnMethodBegin net472 1.17μs 1.03ns 3.98ns 0.0843 0 0 538 B
#2800 RunOnMethodBegin netcoreapp3.1 889ns 0.873ns 3.15ns 0.0075 0 0 576 B

andrewlock avatar May 17 '22 17:05 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2800 into master will not change line coverage :heavy_check_mark: Merging #2800 into master will not change branch coverage :no_entry: Merging #2800 into master will will increase complexity by 49

master #2800 Change
Lines 14102 / 19513 14172 / 19555
Lines % 72% 72% 0% :heavy_check_mark:
Branches 8248 / 11859 8293 / 11900
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 13134 13183 49 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2800 Change
Lines % 72% 72% 0% :heavy_check_mark:
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 13134 13183 49 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.Processors.OriginTagTraceProcessor -9% :no_entry: -17% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.Agent.Transports.SocketHandlerRequestFactory 7% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Agent.Transports.HttpStreamRequestFactory 12% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:

The following classes were added in #2800:

File Line coverage Branch coverage Complexity
Datadog.Trace.HttpOverStreams.TelemetryAgentHttpHeaderHelper 80% 100% 3
Datadog.Trace.Telemetry.TelemetryTransportManager 100% 96% 24
Datadog.Trace.Telemetry.Transports.AgentlessTelemetryTransport 100% 100% 2
Datadog.Trace.Telemetry.Transports.AgentTelemetryTransport 100% 100% 2
Datadog.Trace.Telemetry.Transports.JsonTelemetryTransport 54% 8% 20
...And 2 more

5 classes were removed from Datadog.Trace in #2800

View the full reports for further details:

andrewlock avatar May 17 '22 17:05 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 737μs 308ns 1.15μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 484μs 524ns 1.96μs 0 0 0 2.57 KB
#2800 WriteAndFlushEnrichedTraces net472 755μs 331ns 1.24μs 0 0 0 3.16 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 470μs 110ns 413ns 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.88μs 1.61ns 6.25ns 0.234 0 0 1.48 KB
master AllCycleSimpleBody netcoreapp3.1 1.91μs 2.56ns 8.88ns 0.0181 0 0 1.36 KB
master AllCycleMoreComplexBody net472 17.8μs 8.54ns 33.1ns 1.37 0.0177 0 8.74 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.7μs 8.67ns 32.4ns 0.103 0 0 7.84 KB
master BodyExtractorSimpleBody net472 266ns 0.214ns 0.827ns 0.057 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 240ns 0.104ns 0.388ns 0.00374 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.8μs 6.84ns 26.5ns 1.19 0.0157 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 13μs 4.81ns 18.6ns 0.0914 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 1.83μs 1.87ns 7.24ns 0.234 0 0 1.48 KB
#2800 AllCycleSimpleBody netcoreapp3.1 1.88μs 1.24ns 4.65ns 0.0179 0 0 1.36 KB
#2800 AllCycleMoreComplexBody net472 18μs 7.55ns 28.2ns 1.37 0.0179 0 8.74 KB
#2800 AllCycleMoreComplexBody netcoreapp3.1 14.7μs 3.83ns 14.3ns 0.103 0 0 7.84 KB
#2800 BodyExtractorSimpleBody net472 275ns 0.256ns 0.923ns 0.0571 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 233ns 0.25ns 0.968ns 0.00372 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 16.1μs 5.43ns 21ns 1.2 0.0161 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 13μs 4.53ns 17.5ns 0.0906 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0.00335ns 0.00233ns 0.00901ns 0 0 0 0 b
master SendRequest netcoreapp3.1 172μs 92.8ns 359ns 0.256 0 0 19.74 KB
#2800 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 172μs 154ns 598ns 0.172 0 0 19.74 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.49μs 0.838ns 3.14ns 0.0937 0.00075 0 594 B
master ExecuteNonQuery netcoreapp3.1 1.13μs 0.253ns 0.947ns 0.00842 0 0 632 B
#2800 ExecuteNonQuery net472 1.41μs 0.597ns 2.15ns 0.0942 0.000703 0 594 B
#2800 ExecuteNonQuery netcoreapp3.1 1.14μs 0.255ns 0.954ns 0.00804 0 0 632 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.06μs 0.888ns 3.44ns 0.126 0 0 802 B
master CallElasticsearch netcoreapp3.1 1.24μs 0.446ns 1.55ns 0.0105 0 0 792 B
master CallElasticsearchAsync net472 2.23μs 0.629ns 2.43ns 0.147 0 0 939 B
master CallElasticsearchAsync netcoreapp3.1 1.33μs 0.462ns 1.66ns 0.0126 0 0 912 B
#2800 CallElasticsearch net472 2μs 0.541ns 2.09ns 0.125 0 0 802 B
#2800 CallElasticsearch netcoreapp3.1 1.24μs 5.78ns 22.4ns 0.0104 0 0 792 B
#2800 CallElasticsearchAsync net472 2.16μs 0.876ns 3.39ns 0.147 0 0 939 B
#2800 CallElasticsearchAsync netcoreapp3.1 1.29μs 0.477ns 1.72ns 0.0122 0 0 912 B
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.21μs 0.797ns 3.09ns 0.166 0 0 1.06 KB
master ExecuteAsync netcoreapp3.1 1.44μs 0.504ns 1.95ns 0.0143 0 0 1.03 KB
#2800 ExecuteAsync net472 2.21μs 1.11ns 4.31ns 0.166 0 0 1.06 KB
#2800 ExecuteAsync netcoreapp3.1 1.44μs 0.579ns 2.24ns 0.0136 0 0 1.03 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.86μs 1.74ns 6.52ns 0.356 0 0 2.28 KB
master SendAsync netcoreapp3.1 3.17μs 0.991ns 3.43ns 0.0286 0 0 2.21 KB
#2800 SendAsync net472 4.87μs 2.76ns 9.97ns 0.358 0 0 2.28 KB
#2800 SendAsync netcoreapp3.1 3.24μs 1.54ns 5.96ns 0.0292 0 0 2.21 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.82μs 1.45ns 5.42ns 0.228 0 0 1.45 KB
master EnrichedLog netcoreapp3.1 2.11μs 1.05ns 4.08ns 0.0211 0 0 1.53 KB
#2800 EnrichedLog net472 2.78μs 1.24ns 4.81ns 0.227 0 0 1.45 KB
#2800 EnrichedLog netcoreapp3.1 2.2μs 0.574ns 2.15ns 0.0207 0 0 1.53 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 148μs 168ns 651ns 0.514 0.147 0 4.29 KB
master EnrichedLog netcoreapp3.1 113μs 226ns 875ns 0 0 0 4.18 KB
#2800 EnrichedLog net472 145μs 112ns 403ns 0.509 0.146 0 4.29 KB
#2800 EnrichedLog netcoreapp3.1 111μs 223ns 863ns 0.0549 0 0 4.18 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.24μs 5.49ns 21.3ns 0.507 0.00261 0 3.23 KB
master EnrichedLog netcoreapp3.1 4.02μs 1.05ns 3.8ns 0.0483 0 0 3.6 KB
#2800 EnrichedLog net472 5.21μs 4.28ns 16.6ns 0.506 0.00259 0 3.23 KB
#2800 EnrichedLog netcoreapp3.1 4.22μs 4.77ns 18.5ns 0.0479 0 0 3.6 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.83μs 1.12ns 4.18ns 0.159 0 0 1.01 KB
master SendReceive netcoreapp3.1 1.51μs 2.16ns 7.79ns 0.0135 0 0 1.01 KB
#2800 SendReceive net472 1.9μs 0.435ns 1.63ns 0.159 0 0 1.01 KB
#2800 SendReceive netcoreapp3.1 1.55μs 0.724ns 2.71ns 0.0131 0 0 1.01 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.47μs 4.1ns 15.9ns 0.292 0 0 1.87 KB
master EnrichedLog netcoreapp3.1 3.88μs 1.55ns 5.81ns 0.0194 0 0 1.49 KB
#2800 EnrichedLog net472 4.43μs 2.02ns 7.82ns 0.291 0 0 1.87 KB
#2800 EnrichedLog netcoreapp3.1 3.89μs 2.02ns 7.54ns 0.0195 0 0 1.49 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 837ns 0.63ns 2.44ns 0.0717 0 0 457 B
master StartFinishSpan netcoreapp3.1 655ns 0.735ns 2.65ns 0.00608 0 0 456 B
master StartFinishScope net472 997ns 0.255ns 0.988ns 0.0846 0 0 538 B
master StartFinishScope netcoreapp3.1 795ns 0.191ns 0.741ns 0.00756 0 0 576 B
#2800 StartFinishSpan net472 757ns 0.243ns 0.943ns 0.0718 0 0 457 B
#2800 StartFinishSpan netcoreapp3.1 645ns 0.0976ns 0.338ns 0.00614 0 0 456 B
#2800 StartFinishScope net472 1.03μs 0.145ns 0.541ns 0.0844 0 0 538 B
#2800 StartFinishScope netcoreapp3.1 774ns 0.3ns 1.12ns 0.00775 0 0 576 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.16μs 0.273ns 1.02ns 0.0842 0 0 538 B
master RunOnMethodBegin netcoreapp3.1 891ns 0.232ns 0.835ns 0.0076 0 0 576 B
#2800 RunOnMethodBegin net472 1.19μs 0.813ns 3.15ns 0.0842 0 0 538 B
#2800 RunOnMethodBegin netcoreapp3.1 860ns 2.54ns 9.86ns 0.0075 0 0 576 B

andrewlock avatar May 23 '22 13:05 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2800 into master will not change line coverage :heavy_check_mark: Merging #2800 into master will not change branch coverage :no_entry: Merging #2800 into master will will increase complexity by 37

master #2800 Change
Lines 14147 / 19610 14182 / 19654
Lines % 72% 72% 0% :heavy_check_mark:
Branches 8284 / 11916 8319 / 11946
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 13209 13246 37 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2800 Change
Lines % 72% 72% 0% :heavy_check_mark:
Branches % 70% 70% 0% :heavy_check_mark:
Complexity 13209 13246 37 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.GitInfo -17% :no_entry: -11% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.Ci.Processors.OriginTagTraceProcessor -9% :no_entry: -17% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.Telemetry.TelemetryFactory -2% :warning: -25% :no_entry: 2 :no_entry:
Datadog.Trace.Agent.Transports.SocketHandlerRequestFactory 7% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Agent.Transports.HttpStreamRequestFactory 12% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:

The following classes were added in #2800:

File Line coverage Branch coverage Complexity
Datadog.Trace.Telemetry.TelemetryTransportManager 100% 96% 24
Datadog.Trace.Telemetry.Transports.AgentlessTelemetryTransport 100% 100% 2
Datadog.Trace.Telemetry.Transports.AgentTelemetryTransport 100% 100% 2

1 classes were removed from Datadog.Trace in #2800

View the full reports for further details:

andrewlock avatar May 23 '22 14:05 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 717μs 415ns 1.61μs 0.359 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 455μs 351ns 1.36μs 0 0 0 2.57 KB
#2800 WriteAndFlushEnrichedTraces net472 713μs 538ns 1.94μs 0.357 0 0 3.16 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 453μs 154ns 554ns 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.91μs 1.99ns 7.7ns 0.235 0 0 1.48 KB
master AllCycleSimpleBody netcoreapp3.1 1.86μs 1.54ns 5.56ns 0.0187 0 0 1.36 KB
master AllCycleMoreComplexBody net472 17.7μs 29.3ns 113ns 1.38 0.0177 0 8.74 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.5μs 16.3ns 63.2ns 0.101 0 0 7.84 KB
master BodyExtractorSimpleBody net472 270ns 0.357ns 1.38ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 246ns 0.443ns 1.66ns 0.0037 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.6μs 31ns 120ns 1.2 0.0155 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.7μs 17.2ns 66.6ns 0.0887 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 1.81μs 2.91ns 11.3ns 0.235 0 0 1.48 KB
#2800 AllCycleSimpleBody netcoreapp3.1 1.86μs 2.21ns 7.96ns 0.0187 0 0 1.36 KB
#2800 AllCycleMoreComplexBody net472 18μs 19.5ns 75.5ns 1.39 0.0182 0 8.74 KB
#2800 AllCycleMoreComplexBody netcoreapp3.1 15.2μs 12.8ns 49.6ns 0.107 0 0 7.84 KB
#2800 BodyExtractorSimpleBody net472 277ns 0.477ns 1.85ns 0.0574 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 236ns 0.591ns 2.13ns 0.00365 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 16.1μs 22.2ns 85.9ns 1.21 0.0161 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 13.8μs 20.5ns 79.3ns 0.0892 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 176μs 138ns 534ns 0.175 0 0 19.86 KB
#2800 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 175μs 169ns 656ns 0.173 0 0 19.86 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.5μs 0.698ns 2.7ns 0.115 0.000759 0 722 B
master ExecuteNonQuery netcoreapp3.1 1.23μs 0.415ns 1.55ns 0.00981 0 0 752 B
#2800 ExecuteNonQuery net472 1.47μs 0.767ns 2.87ns 0.115 0.00073 0 722 B
#2800 ExecuteNonQuery netcoreapp3.1 1.26μs 0.442ns 1.6ns 0.0101 0 0 752 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.25μs 1.31ns 5.07ns 0.148 0 0 931 B
master CallElasticsearch netcoreapp3.1 1.31μs 0.504ns 1.88ns 0.0125 0 0 912 B
master CallElasticsearchAsync net472 2.25μs 0.664ns 2.57ns 0.169 0 0 1.07 KB
master CallElasticsearchAsync netcoreapp3.1 1.46μs 0.689ns 2.58ns 0.0138 0 0 1.03 KB
#2800 CallElasticsearch net472 2.04μs 0.576ns 2.23ns 0.147 0 0 931 B
#2800 CallElasticsearch netcoreapp3.1 1.4μs 0.573ns 2.22ns 0.0119 0 0 912 B
#2800 CallElasticsearchAsync net472 2.25μs 0.448ns 1.68ns 0.169 0 0 1.07 KB
#2800 CallElasticsearchAsync netcoreapp3.1 1.43μs 0.738ns 2.76ns 0.0136 0 0 1.03 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.43μs 7.55ns 29.2ns 0.188 0 0 1.19 KB
master ExecuteAsync netcoreapp3.1 1.52μs 3.14ns 12.2ns 0.015 0 0 1.15 KB
#2800 ExecuteAsync net472 2.33μs 2.26ns 8.14ns 0.189 0 0 1.19 KB
#2800 ExecuteAsync netcoreapp3.1 1.53μs 0.526ns 1.97ns 0.0159 0 0 1.15 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.79μs 17.9ns 67.1ns 0.382 0 0 2.41 KB
master SendAsync netcoreapp3.1 3.27μs 5.77ns 22.4ns 0.0308 0 0 2.33 KB
#2800 SendAsync net472 4.84μs 8.96ns 34.7ns 0.382 0 0 2.41 KB
#2800 SendAsync netcoreapp3.1 3.27μs 9.15ns 35.5ns 0.0323 0 0 2.33 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.7μs 1.04ns 4.04ns 0.251 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 2.26μs 11.5ns 54.1ns 0.0224 0 0 1.65 KB
#2800 EnrichedLog net472 2.85μs 0.884ns 3.42ns 0.251 0 0 1.58 KB
#2800 EnrichedLog netcoreapp3.1 2.32μs 1.06ns 3.96ns 0.0224 0 0 1.65 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 144μs 59.4ns 230ns 0.653 0.218 0 4.42 KB
master EnrichedLog netcoreapp3.1 109μs 80.3ns 311ns 0.0544 0 0 4.3 KB
#2800 EnrichedLog net472 146μs 261ns 1.01μs 0.651 0.217 0 4.42 KB
#2800 EnrichedLog netcoreapp3.1 112μs 188ns 727ns 0.0563 0 0 4.3 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.3μs 11.7ns 45.4ns 0.531 0.0026 0 3.35 KB
master EnrichedLog netcoreapp3.1 3.99μs 10.3ns 40ns 0.0515 0 0 3.72 KB
#2800 EnrichedLog net472 5.31μs 9.01ns 34.9ns 0.532 0.00263 0 3.35 KB
#2800 EnrichedLog netcoreapp3.1 4.02μs 5.45ns 21.1ns 0.0503 0 0 3.72 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.91μs 0.825ns 3.2ns 0.181 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.64μs 1.02ns 3.95ns 0.0148 0 0 1.13 KB
#2800 SendReceive net472 1.91μs 0.743ns 2.88ns 0.181 0 0 1.14 KB
#2800 SendReceive netcoreapp3.1 1.63μs 0.45ns 1.68ns 0.0147 0 0 1.13 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.63μs 1.46ns 5.64ns 0.316 0 0 2 KB
master EnrichedLog netcoreapp3.1 4.06μs 1.85ns 7.15ns 0.0221 0 0 1.61 KB
#2800 EnrichedLog net472 4.63μs 1.61ns 6.22ns 0.316 0 0 2 KB
#2800 EnrichedLog netcoreapp3.1 4.17μs 4.31ns 16.7ns 0.0208 0 0 1.61 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 824ns 0.193ns 0.747ns 0.0928 0 0 586 B
master StartFinishSpan netcoreapp3.1 703ns 0.239ns 0.895ns 0.00772 0 0 576 B
master StartFinishScope net472 1.07μs 0.516ns 1.93ns 0.106 0 0 666 B
master StartFinishScope netcoreapp3.1 835ns 0.323ns 1.25ns 0.00959 0 0 696 B
#2800 StartFinishSpan net472 789ns 0.244ns 0.943ns 0.0928 0 0 586 B
#2800 StartFinishSpan netcoreapp3.1 694ns 0.2ns 0.748ns 0.008 0 0 576 B
#2800 StartFinishScope net472 1.16μs 0.295ns 1.06ns 0.105 0 0 666 B
#2800 StartFinishScope netcoreapp3.1 817ns 0.258ns 0.966ns 0.00942 0 0 696 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.2μs 0.651ns 2.52ns 0.105 0 0 666 B
master RunOnMethodBegin netcoreapp3.1 970ns 0.265ns 0.956ns 0.00916 0 0 696 B
#2800 RunOnMethodBegin net472 1.14μs 0.171ns 0.616ns 0.105 0 0 666 B
#2800 RunOnMethodBegin netcoreapp3.1 982ns 0.64ns 2.48ns 0.00935 0 0 696 B

andrewlock avatar Jul 01 '22 16:07 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2800 into master will not change line coverage :heavy_check_mark: Merging #2800 into master will not change branch coverage :no_entry: Merging #2800 into master will will increase complexity by 37

master #2800 Change
Lines 15049 / 20461 15122 / 20505
Lines % 74% 74% 0% :heavy_check_mark:
Branches 9138 / 12948 9181 / 12978
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 13819 13856 37 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2800 Change
Lines % 74% 74% 0% :heavy_check_mark:
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 13819 13856 37 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Telemetry.TelemetryFactory -2% :warning: -25% :no_entry: 2 :no_entry:
Datadog.Trace.Agent.Transports.SocketHandlerRequestFactory 7% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Agent.Transports.HttpStreamRequestFactory 12% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:

The following classes were added in #2800:

File Line coverage Branch coverage Complexity
Datadog.Trace.Telemetry.TelemetryTransportManager 100% 96% 24
Datadog.Trace.Telemetry.Transports.AgentlessTelemetryTransport 100% 100% 2
Datadog.Trace.Telemetry.Transports.AgentTelemetryTransport 100% 100% 2

1 classes were removed from Datadog.Trace in #2800

View the full reports for further details:

andrewlock avatar Jul 01 '22 16:07 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 726μs 758ns 2.94μs 0.363 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 465μs 258ns 998ns 0 0 0 2.57 KB
#2800 WriteAndFlushEnrichedTraces net472 723μs 1.01μs 3.92μs 0.359 0 0 3.16 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 462μs 471ns 1.82μs 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.87μs 5.35ns 20.7ns 0.236 0 0 1.48 KB
master AllCycleSimpleBody netcoreapp3.1 1.84μs 3.28ns 12.7ns 0.0184 0 0 1.36 KB
master AllCycleMoreComplexBody net472 17.6μs 30.1ns 113ns 1.38 0.0176 0 8.74 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.5μs 22.5ns 81.1ns 0.106 0 0 7.84 KB
master BodyExtractorSimpleBody net472 270ns 1.3ns 5.51ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 247ns 0.398ns 1.54ns 0.0037 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.7μs 21.2ns 79.4ns 1.2 0.0157 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.4μs 37.6ns 146ns 0.0926 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 1.74μs 2.13ns 8.26ns 0.236 0 0 1.48 KB
#2800 AllCycleSimpleBody netcoreapp3.1 1.83μs 3.71ns 13.9ns 0.0183 0 0 1.36 KB
#2800 AllCycleMoreComplexBody net472 17.6μs 22.5ns 81ns 1.38 0.0176 0 8.74 KB
#2800 AllCycleMoreComplexBody netcoreapp3.1 14.5μs 61.2ns 237ns 0.107 0 0 7.84 KB
#2800 BodyExtractorSimpleBody net472 269ns 0.913ns 3.54ns 0.0573 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 235ns 0.341ns 1.28ns 0.0037 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 15.8μs 27.8ns 100ns 1.21 0.0158 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 12.1μs 25.3ns 94.8ns 0.0919 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 174μs 169ns 654ns 0.262 0 0 19.86 KB
#2800 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 175μs 155ns 578ns 0.175 0 0 19.87 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.54μs 1.19ns 4.6ns 0.115 0.00077 0 722 B
master ExecuteNonQuery netcoreapp3.1 1.25μs 0.554ns 2.14ns 0.00998 0 0 752 B
#2800 ExecuteNonQuery net472 1.52μs 0.867ns 3.36ns 0.114 0.000751 0 722 B
#2800 ExecuteNonQuery netcoreapp3.1 1.27μs 1.02ns 3.95ns 0.0102 0 0 752 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.12μs 0.649ns 2.51ns 0.147 0 0 931 B
master CallElasticsearch netcoreapp3.1 1.39μs 0.583ns 2.26ns 0.0125 0 0 912 B
master CallElasticsearchAsync net472 2.35μs 1.06ns 4.09ns 0.17 0 0 1.07 KB
master CallElasticsearchAsync netcoreapp3.1 1.42μs 0.374ns 1.4ns 0.0138 0 0 1.03 KB
#2800 CallElasticsearch net472 2.13μs 0.256ns 0.924ns 0.147 0 0 931 B
#2800 CallElasticsearch netcoreapp3.1 1.34μs 0.887ns 3.2ns 0.0121 0 0 912 B
#2800 CallElasticsearchAsync net472 2.29μs 0.752ns 2.91ns 0.169 0 0 1.07 KB
#2800 CallElasticsearchAsync netcoreapp3.1 1.4μs 1.02ns 3.69ns 0.014 0 0 1.03 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.24μs 3.15ns 10.9ns 0.188 0 0 1.19 KB
master ExecuteAsync netcoreapp3.1 1.5μs 4.1ns 15.3ns 0.0156 0 0 1.15 KB
#2800 ExecuteAsync net472 2.35μs 4.33ns 16.8ns 0.188 0 0 1.19 KB
#2800 ExecuteAsync netcoreapp3.1 1.48μs 2.04ns 7.62ns 0.0154 0 0 1.15 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.66μs 9.32ns 33.6ns 0.382 0 0 2.41 KB
master SendAsync netcoreapp3.1 3.16μs 5.23ns 19.6ns 0.0314 0 0 2.33 KB
#2800 SendAsync net472 4.67μs 11.2ns 43.5ns 0.381 0 0 2.41 KB
#2800 SendAsync netcoreapp3.1 3.19μs 4.3ns 16.7ns 0.0318 0 0 2.33 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.75μs 2.27ns 8.78ns 0.251 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 2.27μs 1.31ns 4.89ns 0.0227 0 0 1.65 KB
#2800 EnrichedLog net472 2.83μs 1.31ns 5.08ns 0.251 0 0 1.58 KB
#2800 EnrichedLog netcoreapp3.1 2.37μs 1.01ns 3.79ns 0.0224 0 0 1.65 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 146μs 119ns 461ns 0.657 0.219 0 4.42 KB
master EnrichedLog netcoreapp3.1 111μs 217ns 842ns 0.056 0 0 4.3 KB
#2800 EnrichedLog net472 144μs 60.3ns 233ns 0.653 0.218 0 4.42 KB
#2800 EnrichedLog netcoreapp3.1 110μs 258ns 1E+03ns 0.0546 0 0 4.3 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.11μs 8.83ns 34.2ns 0.531 0.00254 0 3.35 KB
master EnrichedLog netcoreapp3.1 4.04μs 7.35ns 28.5ns 0.051 0 0 3.72 KB
#2800 EnrichedLog net472 5.13μs 10.1ns 38.9ns 0.532 0.00255 0 3.35 KB
#2800 EnrichedLog netcoreapp3.1 3.97μs 8.04ns 30.1ns 0.0512 0 0 3.72 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.01μs 1.82ns 6.82ns 0.181 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.63μs 0.647ns 2.42ns 0.0153 0 0 1.13 KB
#2800 SendReceive net472 1.9μs 0.486ns 1.75ns 0.181 0 0 1.14 KB
#2800 SendReceive netcoreapp3.1 1.54μs 0.708ns 2.55ns 0.0154 0 0 1.13 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.62μs 1.66ns 6.44ns 0.316 0 0 2 KB
master EnrichedLog netcoreapp3.1 4.16μs 1.59ns 6.16ns 0.0208 0 0 1.61 KB
#2800 EnrichedLog net472 4.58μs 1.93ns 7.49ns 0.316 0 0 2 KB
#2800 EnrichedLog netcoreapp3.1 3.94μs 1.73ns 6.47ns 0.0215 0 0 1.61 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 817ns 0.69ns 2.58ns 0.093 0 0 586 B
master StartFinishSpan netcoreapp3.1 712ns 0.393ns 1.36ns 0.0079 0 0 576 B
master StartFinishScope net472 1.09μs 0.479ns 1.86ns 0.105 0 0 666 B
master StartFinishScope netcoreapp3.1 859ns 0.352ns 1.32ns 0.00959 0 0 696 B
#2800 StartFinishSpan net472 774ns 0.308ns 1.11ns 0.0928 0 0 586 B
#2800 StartFinishSpan netcoreapp3.1 748ns 0.215ns 0.805ns 0.00787 0 0 576 B
#2800 StartFinishScope net472 1.09μs 0.479ns 1.85ns 0.105 0 0 666 B
#2800 StartFinishScope netcoreapp3.1 845ns 0.287ns 1.07ns 0.00938 0 0 696 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.25μs 0.38ns 1.47ns 0.105 0 0 666 B
master RunOnMethodBegin netcoreapp3.1 969ns 1.95ns 7.28ns 0.00955 0 0 696 B
#2800 RunOnMethodBegin net472 1.22μs 0.361ns 1.35ns 0.105 0 0 666 B
#2800 RunOnMethodBegin netcoreapp3.1 1E+03ns 0.256ns 0.959ns 0.00951 0 0 696 B

andrewlock avatar Jul 06 '22 11:07 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2800 into master will not change line coverage :heavy_check_mark: Merging #2800 into master will not change branch coverage :no_entry: Merging #2800 into master will will increase complexity by 37

master #2800 Change
Lines 15076 / 20464 15121 / 20508
Lines % 74% 74% 0% :heavy_check_mark:
Branches 9153 / 12950 9184 / 12980
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 13821 13858 37 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2800 Change
Lines % 74% 74% 0% :heavy_check_mark:
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 13821 13858 37 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Telemetry.TelemetryFactory -2% :warning: -25% :no_entry: 2 :no_entry:
Datadog.Trace.Agent.Transports.SocketHandlerRequestFactory 7% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Agent.Transports.HttpStreamRequestFactory 12% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:

The following classes were added in #2800:

File Line coverage Branch coverage Complexity
Datadog.Trace.Telemetry.TelemetryTransportManager 100% 96% 24
Datadog.Trace.Telemetry.Transports.AgentlessTelemetryTransport 100% 100% 2
Datadog.Trace.Telemetry.Transports.AgentTelemetryTransport 100% 100% 2

1 classes were removed from Datadog.Trace in #2800

View the full reports for further details:

andrewlock avatar Jul 06 '22 12:07 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 711μs 356ns 1.33μs 0.353 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 464μs 280ns 1.01μs 0 0 0 2.58 KB
#2800 WriteAndFlushEnrichedTraces net472 726μs 504ns 1.95μs 0.353 0 0 3.18 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 451μs 206ns 769ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 1.68μs 1.06ns 4.09ns 0.237 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.8μs 5.02ns 18.1ns 0.018 0 0 1.37 KB
master AllCycleMoreComplexBody net472 16.5μs 10.3ns 39.8ns 1.38 0.025 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.2μs 17.3ns 64.7ns 0.106 0 0 7.85 KB
master BodyExtractorSimpleBody net472 264ns 0.831ns 3.11ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 222ns 0.304ns 1.14ns 0.00368 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.9μs 11ns 38.2ns 1.21 0.0221 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 11.7μs 10.8ns 38.9ns 0.0936 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 1.73μs 2.14ns 8.27ns 0.237 0 0 1.49 KB
#2800 AllCycleSimpleBody netcoreapp3.1 1.81μs 2.63ns 10.2ns 0.0181 0 0 1.37 KB
#2800 AllCycleMoreComplexBody net472 16.6μs 13.7ns 51.2ns 1.38 0.025 0 8.75 KB
#2800 AllCycleMoreComplexBody netcoreapp3.1 14μs 13.4ns 50ns 0.105 0 0 7.85 KB
#2800 BodyExtractorSimpleBody net472 252ns 0.377ns 1.46ns 0.0573 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 223ns 0.219ns 0.82ns 0.00367 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 15.1μs 15.5ns 60.1ns 1.2 0.0151 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 12.1μs 17.6ns 65.9ns 0.091 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 179μs 114ns 428ns 0.268 0 0 20.37 KB
#2800 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 178μs 137ns 495ns 0.267 0 0 20.38 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.55μs 0.529ns 1.98ns 0.126 0 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.26μs 0.291ns 1.01ns 0.0112 0 0 824 B
#2800 ExecuteNonQuery net472 1.6μs 0.44ns 1.65ns 0.126 0.000785 0 794 B
#2800 ExecuteNonQuery netcoreapp3.1 1.21μs 0.643ns 2.41ns 0.0115 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.21μs 2.42ns 9.39ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.39μs 1.56ns 5.85ns 0.0132 0 0 984 B
master CallElasticsearchAsync net472 2.26μs 5.74ns 22.2ns 0.18 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.45μs 1.66ns 6.45ns 0.0146 0 0 1.1 KB
#2800 CallElasticsearch net472 2.07μs 1.91ns 7.41ns 0.159 0 0 1 KB
#2800 CallElasticsearch netcoreapp3.1 1.45μs 1.66ns 6.41ns 0.0131 0 0 984 B
#2800 CallElasticsearchAsync net472 2.35μs 3.3ns 12.8ns 0.181 0 0 1.14 KB
#2800 CallElasticsearchAsync netcoreapp3.1 1.42μs 5.14ns 19.9ns 0.0146 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.55μs 2.58ns 10ns 0.199 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.64μs 1.05ns 3.63ns 0.0164 0 0 1.22 KB
#2800 ExecuteAsync net472 2.43μs 1.42ns 5.5ns 0.2 0 0 1.26 KB
#2800 ExecuteAsync netcoreapp3.1 1.58μs 0.555ns 2ns 0.0166 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5μs 1.79ns 6.95ns 0.394 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.41μs 1.64ns 5.91ns 0.0323 0 0 2.4 KB
#2800 SendAsync net472 5.05μs 0.963ns 3.73ns 0.393 0 0 2.48 KB
#2800 SendAsync netcoreapp3.1 3.45μs 1.29ns 4.84ns 0.0327 0 0 2.4 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.89μs 1.28ns 4.79ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.44μs 0.652ns 2.44ns 0.0232 0 0 1.73 KB
#2800 EnrichedLog net472 2.79μs 2.01ns 7.77ns 0.263 0 0 1.66 KB
#2800 EnrichedLog netcoreapp3.1 2.41μs 1.13ns 4.4ns 0.024 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 147μs 87.6ns 328ns 0.667 0.222 0 4.5 KB
master EnrichedLog netcoreapp3.1 112μs 178ns 688ns 0 0 0 4.38 KB
#2800 EnrichedLog net472 146μs 62.9ns 244ns 0.658 0.219 0 4.5 KB
#2800 EnrichedLog netcoreapp3.1 112μs 73.5ns 265ns 0.0563 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.39μs 1.16ns 4.32ns 0.544 0.0027 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.37μs 1.26ns 4.53ns 0.0522 0 0 3.8 KB
#2800 EnrichedLog net472 5.25μs 1.18ns 4.56ns 0.545 0.00262 0 3.43 KB
#2800 EnrichedLog netcoreapp3.1 4.23μs 2.22ns 8.6ns 0.0529 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.96μs 1.57ns 5.88ns 0.194 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.69μs 0.656ns 2.37ns 0.0169 0 0 1.21 KB
#2800 SendReceive net472 1.98μs 1.41ns 5.27ns 0.194 0 0 1.22 KB
#2800 SendReceive netcoreapp3.1 1.74μs 0.533ns 1.99ns 0.0157 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.74μs 4.64ns 18ns 0.328 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.07μs 3.36ns 12.6ns 0.0224 0 0 1.69 KB
#2800 EnrichedLog net472 4.76μs 3.02ns 11.7ns 0.329 0 0 2.08 KB
#2800 EnrichedLog netcoreapp3.1 4.03μs 5.15ns 19.3ns 0.0224 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 808ns 0.39ns 1.51ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 761ns 0.28ns 1.01ns 0.00853 0 0 648 B
master StartFinishScope net472 1.06μs 0.391ns 1.41ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 919ns 0.448ns 1.68ns 0.0102 0 0 768 B
#2800 StartFinishSpan net472 810ns 0.262ns 0.98ns 0.104 0 0 658 B
#2800 StartFinishSpan netcoreapp3.1 725ns 0.248ns 0.893ns 0.00871 0 0 648 B
#2800 StartFinishScope net472 1.05μs 0.284ns 1.1ns 0.117 0 0 738 B
#2800 StartFinishScope netcoreapp3.1 914ns 0.234ns 0.907ns 0.0105 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.25μs 0.99ns 3.83ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1μs 0.303ns 1.13ns 0.0104 0 0 768 B
#2800 RunOnMethodBegin net472 1.27μs 0.243ns 0.875ns 0.117 0 0 738 B
#2800 RunOnMethodBegin netcoreapp3.1 1.06μs 0.494ns 1.91ns 0.0105 0 0 768 B

andrewlock avatar Jul 28 '22 12:07 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2800 into master will not change line coverage :heavy_check_mark: Merging #2800 into master will not change branch coverage :no_entry: Merging #2800 into master will will increase complexity by 37

master #2800 Change
Lines 16775 / 22847 16790 / 22887
Lines % 73% 73% 0% :heavy_check_mark:
Branches 9997 / 14174 10023 / 14204
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 15236 15273 37 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2800 Change
Lines % 73% 73% 0% :heavy_check_mark:
Branches % 71% 71% 0% :heavy_check_mark:
Complexity 15236 15273 37 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.GitInfo -17% :no_entry: -11% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.Telemetry.TelemetryFactory -2% :warning: -25% :no_entry: 2 :no_entry:
Datadog.Trace.Agent.Transports.SocketHandlerRequestFactory 7% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Ci.CIVisibility 7% :heavy_check_mark: 7% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Agent.Transports.HttpStreamRequestFactory 12% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:

The following classes were added in #2800:

File Line coverage Branch coverage Complexity
Datadog.Trace.Telemetry.TelemetryTransportManager 100% 96% 24
Datadog.Trace.Telemetry.Transports.AgentlessTelemetryTransport 100% 100% 2
Datadog.Trace.Telemetry.Transports.AgentTelemetryTransport 100% 100% 2

1 classes were removed from Datadog.Trace in #2800

View the full reports for further details:

andrewlock avatar Jul 28 '22 14:07 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 721μs 430ns 1.61μs 0.361 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 462μs 104ns 388ns 0 0 0 2.58 KB
#2800 WriteAndFlushEnrichedTraces net472 714μs 426ns 1.65μs 0.359 0 0 3.18 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 483μs 223ns 802ns 0 0 0 2.59 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 199ns 0.285ns 1.1ns 0.0726 0 0 457 B
master AllCycleSimpleBody netcoreapp3.1 264ns 1.28ns 4.94ns 0.00631 0 0 456 B
master AllCycleMoreComplexBody net472 192ns 0.044ns 0.165ns 0.0688 0 0 433 B
master AllCycleMoreComplexBody netcoreapp3.1 250ns 0.0999ns 0.36ns 0.00601 0 0 432 B
master BodyExtractorSimpleBody net472 286ns 0.415ns 1.61ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 234ns 0.097ns 0.363ns 0.00377 0 0 272 B
master BodyExtractorMoreComplexBody net472 16.5μs 8.68ns 31.3ns 1.2 0.0164 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 13.2μs 13.3ns 51.6ns 0.0927 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 197ns 0.0447ns 0.167ns 0.0727 0 0 457 B
#2800 AllCycleSimpleBody netcoreapp3.1 257ns 0.153ns 0.594ns 0.00623 0 0 456 B
#2800 AllCycleMoreComplexBody net472 193ns 0.0426ns 0.159ns 0.0688 0 0 433 B
#2800 AllCycleMoreComplexBody netcoreapp3.1 252ns 0.132ns 0.496ns 0.00595 0 0 432 B
#2800 BodyExtractorSimpleBody net472 280ns 0.528ns 2.04ns 0.0573 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 244ns 0.063ns 0.218ns 0.00382 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 15.7μs 7.3ns 26.3ns 1.21 0.0157 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 12.9μs 9.6ns 35.9ns 0.0908 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 182μs 81.8ns 317ns 0.272 0 0 20.68 KB
#2800 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 181μs 336ns 1.3μs 0.18 0 0 20.68 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.94μs 0.418ns 1.56ns 0.155 0.000967 0 979 B
master ExecuteNonQuery netcoreapp3.1 1.4μs 0.53ns 1.98ns 0.0133 0 0 968 B
#2800 ExecuteNonQuery net472 1.93μs 0.561ns 2.17ns 0.155 0.000964 0 979 B
#2800 ExecuteNonQuery netcoreapp3.1 1.49μs 0.48ns 1.86ns 0.0127 0 0 968 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.51μs 0.931ns 3.61ns 0.188 0 0 1.19 KB
master CallElasticsearch netcoreapp3.1 1.62μs 0.805ns 2.9ns 0.0154 0 0 1.13 KB
master CallElasticsearchAsync net472 2.63μs 1.31ns 4.91ns 0.21 0 0 1.32 KB
master CallElasticsearchAsync netcoreapp3.1 1.65μs 0.551ns 2.06ns 0.0173 0 0 1.25 KB
#2800 CallElasticsearch net472 2.57μs 1.07ns 4.13ns 0.188 0 0 1.19 KB
#2800 CallElasticsearch netcoreapp3.1 1.54μs 0.531ns 1.99ns 0.0149 0 0 1.13 KB
#2800 CallElasticsearchAsync net472 2.76μs 0.763ns 2.95ns 0.209 0 0 1.32 KB
#2800 CallElasticsearchAsync netcoreapp3.1 1.68μs 0.447ns 1.61ns 0.0168 0 0 1.25 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.85μs 0.973ns 3.77ns 0.23 0 0 1.45 KB
master ExecuteAsync netcoreapp3.1 1.78μs 0.75ns 2.81ns 0.0188 0 0 1.38 KB
#2800 ExecuteAsync net472 2.8μs 0.481ns 1.86ns 0.231 0 0 1.45 KB
#2800 ExecuteAsync netcoreapp3.1 1.76μs 0.84ns 3.14ns 0.0186 0 0 1.38 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.9μs 1.81ns 6.76ns 0.444 0 0 2.8 KB
master SendAsync netcoreapp3.1 3.56μs 3.07ns 11.9ns 0.0355 0 0 2.63 KB
#2800 SendAsync net472 5.94μs 1.23ns 4.59ns 0.443 0 0 2.8 KB
#2800 SendAsync netcoreapp3.1 3.73μs 1.39ns 5.2ns 0.0356 0 0 2.63 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.3μs 3.43ns 12.4ns 0.298 0 0 1.88 KB
master EnrichedLog netcoreapp3.1 2.54μs 1.24ns 4.64ns 0.0265 0 0 1.91 KB
#2800 EnrichedLog net472 3.23μs 1.99ns 7.72ns 0.298 0 0 1.88 KB
#2800 EnrichedLog netcoreapp3.1 2.48μs 2.74ns 10.2ns 0.0259 0 0 1.91 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 153μs 281ns 1.09μs 0.688 0.229 0 4.72 KB
master EnrichedLog netcoreapp3.1 116μs 145ns 544ns 0.0576 0 0 4.55 KB
#2800 EnrichedLog net472 154μs 116ns 451ns 0.694 0.231 0 4.72 KB
#2800 EnrichedLog netcoreapp3.1 118μs 145ns 562ns 0.0582 0 0 4.55 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.77μs 2ns 7.49ns 0.578 0.00289 0 3.65 KB
master EnrichedLog netcoreapp3.1 4.7μs 1.93ns 7.49ns 0.0544 0 0 3.98 KB
#2800 EnrichedLog net472 5.93μs 2.29ns 8.56ns 0.578 0.00296 0 3.65 KB
#2800 EnrichedLog netcoreapp3.1 4.37μs 1.29ns 4.82ns 0.0526 0 0 3.98 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.32μs 1.76ns 6.33ns 0.222 0 0 1.4 KB
master SendReceive netcoreapp3.1 1.88μs 0.949ns 3.68ns 0.0185 0 0 1.35 KB
#2800 SendReceive net472 2.34μs 2.49ns 9.66ns 0.223 0 0 1.4 KB
#2800 SendReceive netcoreapp3.1 1.8μs 0.764ns 2.86ns 0.018 0 0 1.35 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.1μs 1.55ns 5.81ns 0.364 0 0 2.3 KB
master EnrichedLog netcoreapp3.1 4.36μs 2.31ns 8.96ns 0.0242 0 0 1.86 KB
#2800 EnrichedLog net472 5.11μs 1.35ns 5.04ns 0.363 0 0 2.3 KB
#2800 EnrichedLog netcoreapp3.1 4.22μs 2.36ns 9.14ns 0.0254 0 0 1.86 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 1.18μs 0.399ns 1.49ns 0.134 0 0 842 B
master StartFinishSpan netcoreapp3.1 931ns 0.367ns 1.37ns 0.0106 0 0 792 B
master StartFinishScope net472 1.43μs 1.08ns 4.03ns 0.147 0 0 923 B
master StartFinishScope netcoreapp3.1 1.12μs 0.281ns 1.01ns 0.0123 0 0 912 B
#2800 StartFinishSpan net472 1.15μs 0.312ns 1.17ns 0.134 0 0 842 B
#2800 StartFinishSpan netcoreapp3.1 946ns 0.458ns 1.65ns 0.0106 0 0 792 B
#2800 StartFinishScope net472 1.41μs 0.45ns 1.68ns 0.146 0 0 923 B
#2800 StartFinishScope netcoreapp3.1 1.04μs 0.516ns 2ns 0.0126 0 0 912 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.46μs 0.414ns 1.6ns 0.146 0 0 923 B
master RunOnMethodBegin netcoreapp3.1 1.2μs 0.661ns 2.38ns 0.012 0 0 912 B
#2800 RunOnMethodBegin net472 1.5μs 0.855ns 3.2ns 0.146 0 0 923 B
#2800 RunOnMethodBegin netcoreapp3.1 1.2μs 0.52ns 1.95ns 0.0126 0 0 912 B

andrewlock avatar Oct 03 '22 16:10 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 721μs 430ns 1.61μs 0.361 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 462μs 104ns 388ns 0 0 0 2.58 KB
#2800 WriteAndFlushEnrichedTraces net472 716μs 414ns 1.55μs 0.355 0 0 3.18 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 466μs 252ns 978ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 199ns 0.285ns 1.1ns 0.0726 0 0 457 B
master AllCycleSimpleBody netcoreapp3.1 264ns 1.28ns 4.94ns 0.00631 0 0 456 B
master AllCycleMoreComplexBody net472 192ns 0.044ns 0.165ns 0.0688 0 0 433 B
master AllCycleMoreComplexBody netcoreapp3.1 250ns 0.0999ns 0.36ns 0.00601 0 0 432 B
master BodyExtractorSimpleBody net472 286ns 0.415ns 1.61ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 234ns 0.097ns 0.363ns 0.00377 0 0 272 B
master BodyExtractorMoreComplexBody net472 16.5μs 8.68ns 31.3ns 1.2 0.0164 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 13.2μs 13.3ns 51.6ns 0.0927 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 195ns 0.0497ns 0.186ns 0.0726 0 0 457 B
#2800 AllCycleSimpleBody netcoreapp3.1 257ns 0.243ns 0.908ns 0.00633 0 0 456 B
#2800 AllCycleMoreComplexBody net472 194ns 0.098ns 0.379ns 0.0688 0 0 433 B
#2800 AllCycleMoreComplexBody netcoreapp3.1 251ns 0.287ns 1.11ns 0.00592 0 0 432 B
#2800 BodyExtractorSimpleBody net472 279ns 0.19ns 0.711ns 0.0573 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 237ns 0.227ns 0.849ns 0.0037 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 16μs 16.3ns 63.3ns 1.21 0.0159 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 13.3μs 6.18ns 23.1ns 0.0931 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 182μs 81.8ns 317ns 0.272 0 0 20.68 KB
#2800 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 181μs 232ns 897ns 0.181 0 0 20.68 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.94μs 0.418ns 1.56ns 0.155 0.000967 0 979 B
master ExecuteNonQuery netcoreapp3.1 1.4μs 0.53ns 1.98ns 0.0133 0 0 968 B
#2800 ExecuteNonQuery net472 1.89μs 0.454ns 1.76ns 0.155 0.000944 0 979 B
#2800 ExecuteNonQuery netcoreapp3.1 1.41μs 0.781ns 2.92ns 0.0134 0 0 968 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.51μs 0.931ns 3.61ns 0.188 0 0 1.19 KB
master CallElasticsearch netcoreapp3.1 1.62μs 0.805ns 2.9ns 0.0154 0 0 1.13 KB
master CallElasticsearchAsync net472 2.63μs 1.31ns 4.91ns 0.21 0 0 1.32 KB
master CallElasticsearchAsync netcoreapp3.1 1.65μs 0.551ns 2.06ns 0.0173 0 0 1.25 KB
#2800 CallElasticsearch net472 2.58μs 0.892ns 3.45ns 0.188 0 0 1.19 KB
#2800 CallElasticsearch netcoreapp3.1 1.6μs 1.31ns 5.07ns 0.0151 0 0 1.13 KB
#2800 CallElasticsearchAsync net472 2.6μs 0.676ns 2.62ns 0.21 0 0 1.32 KB
#2800 CallElasticsearchAsync netcoreapp3.1 1.62μs 0.83ns 3.1ns 0.017 0 0 1.25 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.85μs 0.973ns 3.77ns 0.23 0 0 1.45 KB
master ExecuteAsync netcoreapp3.1 1.78μs 0.75ns 2.81ns 0.0188 0 0 1.38 KB
#2800 ExecuteAsync net472 2.75μs 0.823ns 3.19ns 0.23 0 0 1.45 KB
#2800 ExecuteAsync netcoreapp3.1 1.74μs 0.466ns 1.68ns 0.0183 0 0 1.38 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.9μs 1.81ns 6.76ns 0.444 0 0 2.8 KB
master SendAsync netcoreapp3.1 3.56μs 3.07ns 11.9ns 0.0355 0 0 2.63 KB
#2800 SendAsync net472 5.84μs 1.86ns 6.95ns 0.442 0 0 2.8 KB
#2800 SendAsync netcoreapp3.1 3.57μs 1.53ns 5.5ns 0.0363 0 0 2.63 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.3μs 3.43ns 12.4ns 0.298 0 0 1.88 KB
master EnrichedLog netcoreapp3.1 2.54μs 1.24ns 4.64ns 0.0265 0 0 1.91 KB
#2800 EnrichedLog net472 3.2μs 1.75ns 6.77ns 0.298 0 0 1.88 KB
#2800 EnrichedLog netcoreapp3.1 2.48μs 1.29ns 4.99ns 0.0261 0 0 1.91 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 153μs 281ns 1.09μs 0.688 0.229 0 4.72 KB
master EnrichedLog netcoreapp3.1 116μs 145ns 544ns 0.0576 0 0 4.55 KB
#2800 EnrichedLog net472 152μs 142ns 551ns 0.685 0.228 0 4.72 KB
#2800 EnrichedLog netcoreapp3.1 118μs 249ns 963ns 0 0 0 4.55 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.77μs 2ns 7.49ns 0.578 0.00289 0 3.65 KB
master EnrichedLog netcoreapp3.1 4.7μs 1.93ns 7.49ns 0.0544 0 0 3.98 KB
#2800 EnrichedLog net472 5.94μs 1.46ns 5.46ns 0.579 0.00297 0 3.65 KB
#2800 EnrichedLog netcoreapp3.1 4.53μs 1.24ns 4.8ns 0.054 0 0 3.98 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.32μs 1.76ns 6.33ns 0.222 0 0 1.4 KB
master SendReceive netcoreapp3.1 1.88μs 0.949ns 3.68ns 0.0185 0 0 1.35 KB
#2800 SendReceive net472 2.33μs 2.62ns 10.2ns 0.223 0 0 1.4 KB
#2800 SendReceive netcoreapp3.1 1.85μs 1.13ns 4.38ns 0.0177 0 0 1.35 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.1μs 1.55ns 5.81ns 0.364 0 0 2.3 KB
master EnrichedLog netcoreapp3.1 4.36μs 2.31ns 8.96ns 0.0242 0 0 1.86 KB
#2800 EnrichedLog net472 5.1μs 1.99ns 7.44ns 0.364 0 0 2.3 KB
#2800 EnrichedLog netcoreapp3.1 4.41μs 1.66ns 6.42ns 0.0242 0 0 1.86 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 1.18μs 0.399ns 1.49ns 0.134 0 0 842 B
master StartFinishSpan netcoreapp3.1 931ns 0.367ns 1.37ns 0.0106 0 0 792 B
master StartFinishScope net472 1.43μs 1.08ns 4.03ns 0.147 0 0 923 B
master StartFinishScope netcoreapp3.1 1.12μs 0.281ns 1.01ns 0.0123 0 0 912 B
#2800 StartFinishSpan net472 1.21μs 0.71ns 2.75ns 0.134 0 0 842 B
#2800 StartFinishSpan netcoreapp3.1 916ns 0.283ns 1.1ns 0.0107 0 0 792 B
#2800 StartFinishScope net472 1.39μs 0.371ns 1.34ns 0.146 0 0 923 B
#2800 StartFinishScope netcoreapp3.1 1.11μs 0.572ns 2.21ns 0.0122 0 0 912 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.46μs 0.414ns 1.6ns 0.146 0 0 923 B
master RunOnMethodBegin netcoreapp3.1 1.2μs 0.661ns 2.38ns 0.012 0 0 912 B
#2800 RunOnMethodBegin net472 1.48μs 0.617ns 2.31ns 0.146 0 0 923 B
#2800 RunOnMethodBegin netcoreapp3.1 1.19μs 0.365ns 1.37ns 0.0125 0 0 912 B

andrewlock avatar Oct 03 '22 18:10 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2800 into master will not change line coverage :heavy_check_mark: Merging #2800 into master will not change branch coverage :no_entry: Merging #2800 into master will will increase complexity by 41

master #2800 Change
Lines 18342 / 25593 18467 / 25619
Lines % 72% 72% 0% :heavy_check_mark:
Branches 10626 / 15583 10680 / 15615
Branches % 68% 68% 0% :heavy_check_mark:
Complexity 16844 16885 41 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2800 Change
Lines % 72% 72% 0% :heavy_check_mark:
Branches % 68% 68% 0% :heavy_check_mark:
Complexity 16844 16885 41 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Telemetry.TelemetryFactory -2% :warning: -25% :no_entry: 2 :no_entry:
Datadog.Trace.Agent.Transports.SocketHandlerRequestFactory 7% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Agent.Transports.HttpStreamRequestFactory 12% :heavy_check_mark: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.Ci.GitInfo 38% :heavy_check_mark: 30% :heavy_check_mark: 0 :heavy_check_mark:

The following classes were added in #2800:

File Line coverage Branch coverage Complexity
Datadog.Trace.Telemetry.TelemetryTransportManager 100% 89% 38
Datadog.Trace.Telemetry.Transports.AgentlessTelemetryTransport 100% 100% 2
Datadog.Trace.Telemetry.Transports.AgentTelemetryTransport 100% 100% 2

1 classes were removed from Datadog.Trace in #2800

View the full reports for further details:

andrewlock avatar Oct 03 '22 19:10 andrewlock

Benchmarks Report :snail:

Benchmarks for #2800 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 723μs 634ns 2.46μs 0.361 0 0 3.17 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 462μs 86.6ns 324ns 0 0 0 2.59 KB
#2800 WriteAndFlushEnrichedTraces net472 728μs 566ns 2.19μs 0.359 0 0 3.18 KB
#2800 WriteAndFlushEnrichedTraces netcoreapp3.1 473μs 200ns 775ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 198ns 0.0965ns 0.348ns 0.0727 0 0 457 B
master AllCycleSimpleBody netcoreapp3.1 251ns 0.166ns 0.642ns 0.00632 0 0 456 B
master AllCycleMoreComplexBody net472 214ns 0.148ns 0.554ns 0.0688 0 0 433 B
master AllCycleMoreComplexBody netcoreapp3.1 255ns 0.15ns 0.582ns 0.00599 0 0 432 B
master BodyExtractorSimpleBody net472 276ns 0.21ns 0.757ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 243ns 0.0938ns 0.363ns 0.00368 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.8μs 10.1ns 39.1ns 1.2 0.0157 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.8μs 5.8ns 22.5ns 0.0893 0 0 6.75 KB
#2800 AllCycleSimpleBody net472 196ns 0.0397ns 0.149ns 0.0727 0 0 457 B
#2800 AllCycleSimpleBody netcoreapp3.1 255ns 0.288ns 1.12ns 0.00616 0 0 456 B
#2800 AllCycleMoreComplexBody net472 207ns 0.384ns 1.44ns 0.0689 0 0 433 B
#2800 AllCycleMoreComplexBody netcoreapp3.1 249ns 0.267ns 1.04ns 0.00599 0 0 432 B
#2800 BodyExtractorSimpleBody net472 274ns 0.138ns 0.516ns 0.0574 0 0 361 B
#2800 BodyExtractorSimpleBody netcoreapp3.1 228ns 0.132ns 0.511ns 0.00368 0 0 272 B
#2800 BodyExtractorMoreComplexBody net472 15.7μs 4.74ns 17.1ns 1.2 0.0157 0 7.62 KB
#2800 BodyExtractorMoreComplexBody netcoreapp3.1 12.9μs 5.73ns 22.2ns 0.0907 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 179μs 190ns 737ns 0.269 0 0 20.68 KB
#2800 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2800 SendRequest netcoreapp3.1 179μs 194ns 750ns 0.267 0 0 20.68 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.89μs 0.767ns 2.87ns 0.155 0.000944 0 979 B
master ExecuteNonQuery netcoreapp3.1 1.44μs 0.635ns 2.37ns 0.013 0 0 968 B
#2800 ExecuteNonQuery net472 1.85μs 0.746ns 2.79ns 0.155 0.000929 0 979 B
#2800 ExecuteNonQuery netcoreapp3.1 1.39μs 0.476ns 1.84ns 0.0132 0 0 968 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.56μs 1.22ns 4.73ns 0.188 0 0 1.19 KB
master CallElasticsearch netcoreapp3.1 1.54μs 0.404ns 1.56ns 0.0154 0 0 1.13 KB
master CallElasticsearchAsync net472 2.66μs 0.848ns 3.29ns 0.209 0 0 1.32 KB
master CallElasticsearchAsync netcoreapp3.1 1.65μs 0.634ns 2.45ns 0.0166 0 0 1.25 KB
#2800 CallElasticsearch net472 2.44μs 1.52ns 5.9ns 0.188 0 0 1.19 KB
#2800 CallElasticsearch netcoreapp3.1 1.62μs 0.377ns 1.36ns 0.0146 0 0 1.13 KB
#2800 CallElasticsearchAsync net472 2.63μs 0.472ns 1.77ns 0.209 0 0 1.32 KB
#2800 CallElasticsearchAsync netcoreapp3.1 1.57μs 0.299ns 1.08ns 0.0165 0 0 1.25 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.83μs 1.19ns 4.61ns 0.231 0 0 1.45 KB
master ExecuteAsync netcoreapp3.1 1.68μs 0.691ns 2.58ns 0.0184 0 0 1.38 KB
#2800 ExecuteAsync net472 2.9μs 0.793ns 3.07ns 0.231 0 0 1.45 KB
#2800 ExecuteAsync netcoreapp3.1 1.72μs 0.915ns 3.54ns 0.0188 0 0 1.38 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.88μs 2.57ns 9.61ns 0.445 0 0 2.8 KB
master SendAsync netcoreapp3.1 3.76μs 1.29ns 4.81ns 0.0358 0 0 2.63 KB
#2800 SendAsync net472 5.81μs 1.74ns 6.5ns 0.443 0 0 2.8 KB
#2800 SendAsync netcoreapp3.1 3.66μs 2.47ns 9.25ns 0.0366 0 0 2.63 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.16μs 2.03ns 7.6ns 0.297 0 0 1.88 KB
master EnrichedLog netcoreapp3.1 2.64μs 1.17ns 4.55ns 0.0264 0 0 1.91 KB
#2800 EnrichedLog net472 3.29μs 4.34ns 16.3ns 0.297 0 0 1.88 KB
#2800 EnrichedLog netcoreapp3.1 2.42μs 1.92ns 7.45ns 0.0254 0 0 1.91 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 153μs 159ns 616ns 0.687 0.229 0 4.72 KB
master EnrichedLog netcoreapp3.1 117μs 124ns 479ns 0.0583 0 0 4.55 KB
#2800 EnrichedLog net472 151μs 176ns 680ns 0.683 0.228 0 4.72 KB
#2800 EnrichedLog netcoreapp3.1 117μs 123ns 477ns 0.0577 0 0 4.55 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.89μs 2.92ns 11.3ns 0.58 0.00293 0 3.65 KB
master EnrichedLog netcoreapp3.1 4.44μs 2.51ns 9.4ns 0.0531 0 0 3.98 KB
#2800 EnrichedLog net472 5.86μs 3.53ns 13.7ns 0.577 0.00295 0 3.65 KB
#2800 EnrichedLog netcoreapp3.1 4.4μs 1.51ns 5.46ns 0.055 0 0 3.98 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.22μs 2.03ns 7.88ns 0.222 0 0 1.4 KB
master SendReceive netcoreapp3.1 1.83μs 0.687ns 2.66ns 0.0183 0 0 1.35 KB
#2800 SendReceive net472 2.35μs 1.67ns 6.48ns 0.223 0 0 1.4 KB
#2800 SendReceive netcoreapp3.1 1.81μs 0.56ns 2.17ns 0.0181 0 0 1.35 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.02μs 1.49ns 5.37ns 0.364 0 0 2.3 KB
master EnrichedLog netcoreapp3.1 4.34μs 1.41ns 5.45ns 0.0238 0 0 1.86 KB
#2800 EnrichedLog net472 5.01μs 1.4ns 5.25ns 0.364 0 0 2.3 KB
#2800 EnrichedLog netcoreapp3.1 4.31μs 1.21ns 4.51ns 0.0237 0 0 1.86 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 1.17μs 0.311ns 1.16ns 0.134 0 0 842 B
master StartFinishSpan netcoreapp3.1 922ns 0.308ns 1.07ns 0.0111 0 0 792 B
master StartFinishScope net472 1.45μs 0.518ns 2.01ns 0.146 0 0 923 B
master StartFinishScope netcoreapp3.1 1.05μs 0.434ns 1.62ns 0.0126 0 0 912 B
#2800 StartFinishSpan net472 1.13μs 0.298ns 1.15ns 0.134 0 0 842 B
#2800 StartFinishSpan netcoreapp3.1 995ns 0.717ns 2.78ns 0.0109 0 0 792 B
#2800 StartFinishScope net472 1.43μs 1.25ns 4.86ns 0.146 0 0 923 B
#2800 StartFinishScope netcoreapp3.1 1.09μs 0.26ns 0.938ns 0.0121 0 0 912 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.6μs 0.347ns 1.34ns 0.146 0 0 923 B
master RunOnMethodBegin netcoreapp3.1 1.21μs 0.615ns 2.38ns 0.0121 0 0 912 B
#2800 RunOnMethodBegin net472 1.46μs 0.407ns 1.58ns 0.147 0 0 923 B
#2800 RunOnMethodBegin netcoreapp3.1 1.24μs 0.387ns 1.5ns 0.0125 0 0 912 B

andrewlock avatar Oct 07 '22 10:10 andrewlock

Code Coverage Report :bar_chart:

:heavy_check_mark: Merging #2800 into master will not change line coverage :heavy_check_mark: Merging #2800 into master will will increase branch coverage by 1% :no_entry: Merging #2800 into master will will increase complexity by 973

master #2800 Change
Lines 18475 / 25654 19229 / 26611
Lines % 72% 72% 0% :heavy_check_mark:
Branches 10686 / 15617 11442 / 16529
Branches % 68% 69% 1% :heavy_check_mark:
Complexity 16884 17857 973 :no_entry:

View the full report for further details:

Datadog.Trace Breakdown :heavy_check_mark:

master #2800 Change
Lines % 72% 72% 0% :heavy_check_mark:
Branches % 68% 69% 1% :heavy_check_mark:
Complexity 16884 17857 973 :no_entry:

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.ClrProfiler.AutoInstrumentation.Testing.MsTestV2.UnitTestRunnerRunCleanupIntegration -33% :no_entry: -50% :no_entry: 1 :no_entry:
Datadog.Trace.ClrProfiler.AutoInstrumentation.Testing.Common -26% :no_entry: -13% :no_entry: -19 :heavy_check_mark:
Datadog.Trace.Ci.GitInfo -17% :no_entry: -11% :no_entry: 0 :heavy_check_mark:
Datadog.Trace.ClrProfiler.AutoInstrumentation.Testing.XUnit.XUnitIntegration -16% :no_entry: -17% :no_entry: 18 :no_entry:
Datadog.Trace.ClrProfiler.AutoInstrumentation.Testing.NUnit.NUnitIntegration -11% :no_entry: -4% :warning: 18 :no_entry:
Datadog.Trace.ClrProfiler.AutoInstrumentation.Testing.NUnit.NUnitSkipCommandExecuteIntegration -11% :no_entry: 0% :heavy_check_mark: 0 :heavy_check_mark:
Datadog.Trace.ClrProfiler.AutoInstrumentation.Testing.MsTestV2.TestMethodAttributeExecuteIntegration -9% :no_entry: -8% :no_entry: 2 :no_entry:
Datadog.Trace.ClrProfiler.AutoInstrumentation.Testing.XUnit.XUnitTestInvokerRunAsyncIntegration -5% :no_entry: -12% :no_entry: 4 :no_entry:
Datadog.Trace.Telemetry.TelemetryFactory -2% :warning: -25% :no_entry: 2 :no_entry:
Datadog.Trace.ClrProfiler.AutoInstrumentation.Testing.XUnit.XUnitTestRunnerRunAsyncIntegration 0% :heavy_check_mark: -36% :no_entry: 8 :no_entry:
...And 5 more

The following classes were added in #2800:

File Line coverage Branch coverage Complexity
Datadog.Trace.Ci.Agent.MessagePack.CIVisibilityEventMessagePackFormatter`1 0% 0% 2
Datadog.Trace.Ci.EventModel.CIVisibilityEventsFactory 0% 0% 10
Datadog.Trace.Ci.EventModel.TestModuleEvent 0% 100% 1
Datadog.Trace.Ci.EventModel.TestSessionEvent 0% 100% 1
Datadog.Trace.Ci.EventModel.TestSuiteEvent 0% 100% 1
...And 21 more

6 classes were removed from Datadog.Trace in #2800

View the full reports for further details:

andrewlock avatar Oct 07 '22 11:10 andrewlock