dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard

More build tidying up

Open andrewlock opened this issue 3 years ago • 3 comments

Summary of changes

  • Replace the deprecated ubuntu-18.04 hosted runners with ubuntu-20.04
  • Tidy up the serverless build by not building when it's not required
  • Reduce the size of the self-contained dd-trace executables
  • Replaced some confusing usages of TracerHomeDirectory

Reason for change

Replace the deprecated ubuntu-18.04 hosted runners with ubuntu-20.04 We're getting build warnings about these in AzDo. As we run in Docker, the underlying VM doesn't really matter, so should be fine.

Tidy up the serverless build by not building when it's not required Currently the serverless build only runs on .NET Core 3.1, yet we have to download and build the dockerfiles for every platform.

In this PR we split out the serverless-specific docker-compose services into a separate file, and conditionally use them on netcore3.1 only. That also means we can remove the annoying placeholder.txt I added in #3060. Supersedes #2365.

Note that with these changes the other docker stages are much shorter than the .NET Core 3.1 stage. We could look at splitting the serverless tests only into a separate stage, but I don't think this is in the hot path currently, so probably not worth the effort (I'll take another look after this is merged)

Reduce the size of the self-contained dd-trace executables We build self-contained versions of the dd-trace tool for each platform (linux-x64, win-x64 etc) which are essentially the same as the .NET tool NuGet. However, as the self-contained executables will only run on a single platform, there's not much point including all the other native files for the other platforms too. This PR strips out all those extra dlls, reducing the executable size by ~30MB, and the compressed size by ~20MB (40%)

Replaced some confusing usages of TracerHomeDirectory #3060 largely replaced the concepts of TracerHomeDirectory and ProfilerHomeDirectory with MonitoringHomeDirectory. This PR does some more cleanup around this, and replaces the remaining usages.

Test coverage

Mostly covered by existing tests. I'll do a dedicated run for testing the standalone dd-trace artifacts.

Other details

Rebased on top of #3071, so we should merge that one first

andrewlock avatar Aug 12 '22 15:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3076 compared to master:

  • 3 benchmarks are faster, with geometric mean 1.221
  • 1 benchmarks are slower, with geometric mean 1.121
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #3076

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 1.145 527,143.46 460,441.46

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 792μs 4.32μs 24.4μs 0.391 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 529μs 3.02μs 22.6μs 0 0 0 2.58 KB
#3076 WriteAndFlushEnrichedTraces net472 726μs 484ns 1.88μs 0.363 0 0 3.18 KB
#3076 WriteAndFlushEnrichedTraces netcoreapp3.1 460μs 366ns 1.37μs 0 0 0 2.59 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 205ns 0.137ns 0.493ns 0.0676 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 234ns 0.27ns 1.05ns 0.00594 0 0 424 B
master AllCycleMoreComplexBody net472 182ns 0.151ns 0.566ns 0.0638 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 236ns 0.314ns 1.22ns 0.00542 0 0 400 B
master BodyExtractorSimpleBody net472 259ns 0.152ns 0.549ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 216ns 0.355ns 1.37ns 0.0038 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.2μs 9.8ns 37.9ns 1.21 0.0215 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.5μs 20.2ns 72.8ns 0.0933 0 0 6.75 KB
#3076 AllCycleSimpleBody net472 186ns 0.171ns 0.64ns 0.0676 9.18E-05 0 425 B
#3076 AllCycleSimpleBody netcoreapp3.1 236ns 0.279ns 1.04ns 0.00575 0 0 424 B
#3076 AllCycleMoreComplexBody net472 182ns 0.193ns 0.747ns 0.0637 0 0 401 B
#3076 AllCycleMoreComplexBody netcoreapp3.1 237ns 0.386ns 1.5ns 0.00552 0 0 400 B
#3076 BodyExtractorSimpleBody net472 265ns 0.379ns 1.42ns 0.0573 0 0 361 B
#3076 BodyExtractorSimpleBody netcoreapp3.1 227ns 0.297ns 1.15ns 0.00379 0 0 272 B
#3076 BodyExtractorMoreComplexBody net472 14.2μs 7.35ns 28.5ns 1.21 0.0212 0 7.62 KB
#3076 BodyExtractorMoreComplexBody netcoreapp3.1 12.1μs 15.3ns 57.2ns 0.0905 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0.0624ns 0.0151ns 0.0918ns 0 0 0 0 b
master SendRequest netcoreapp3.1 189μs 77.1ns 267ns 0.188 0 0 20.33 KB
#3076 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3076 SendRequest netcoreapp3.1 180μs 144ns 558ns 0.18 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #3076

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑netcoreapp3.1 1.421 1,762.88 1,240.96 bimodal

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.69μs 9.36ns 54.6ns 0.126 0.000827 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.77μs 15.2ns 152ns 0.0112 0 0 824 B
#3076 ExecuteNonQuery net472 1.56μs 0.834ns 3.23ns 0.126 0.000778 0 794 B
#3076 ExecuteNonQuery netcoreapp3.1 1.24μs 0.428ns 1.66ns 0.0111 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.17μs 0.518ns 1.87ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.4μs 0.68ns 2.64ns 0.0131 0 0 984 B
master CallElasticsearchAsync net472 2.28μs 4.84ns 18.8ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.43μs 0.529ns 1.98ns 0.015 0 0 1.1 KB
#3076 CallElasticsearch net472 2.19μs 0.602ns 2.25ns 0.159 0 0 1 KB
#3076 CallElasticsearch netcoreapp3.1 1.47μs 1.27ns 4.74ns 0.0132 0 0 984 B
#3076 CallElasticsearchAsync net472 2.26μs 2.87ns 11.1ns 0.181 0 0 1.14 KB
#3076 CallElasticsearchAsync netcoreapp3.1 1.41μs 0.451ns 1.75ns 0.0147 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.48μs 3.1ns 12ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.56μs 1.66ns 6.2ns 0.0163 0 0 1.22 KB
#3076 ExecuteAsync net472 2.41μs 1.93ns 7.47ns 0.199 0 0 1.26 KB
#3076 ExecuteAsync netcoreapp3.1 1.54μs 2.51ns 9.4ns 0.0162 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.06μs 6.98ns 26.1ns 0.392 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.3μs 4.05ns 15.7ns 0.031 0 0 2.36 KB
#3076 SendAsync net472 5.01μs 2.24ns 8.67ns 0.393 0 0 2.48 KB
#3076 SendAsync netcoreapp3.1 3.26μs 1.8ns 6.75ns 0.031 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.83μs 0.751ns 2.81ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.29μs 0.476ns 1.72ns 0.0231 0 0 1.73 KB
#3076 EnrichedLog net472 2.93μs 1.65ns 6.4ns 0.263 0 0 1.66 KB
#3076 EnrichedLog netcoreapp3.1 2.4μs 1.29ns 4.83ns 0.0229 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 149μs 77.9ns 302ns 0.675 0.225 0 4.5 KB
master EnrichedLog netcoreapp3.1 113μs 107ns 399ns 0.0566 0 0 4.38 KB
#3076 EnrichedLog net472 148μs 138ns 534ns 0.674 0.225 0 4.5 KB
#3076 EnrichedLog netcoreapp3.1 114μs 177ns 685ns 0 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.62μs 8.3ns 32.1ns 0.545 0.00281 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.31μs 5.04ns 19.5ns 0.0514 0 0 3.8 KB
#3076 EnrichedLog net472 5.57μs 7.31ns 28.3ns 0.545 0.00279 0 3.43 KB
#3076 EnrichedLog netcoreapp3.1 4.25μs 1.77ns 6.87ns 0.0504 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.99μs 1.03ns 4ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.73μs 0.707ns 2.65ns 0.0165 0 0 1.21 KB
#3076 SendReceive net472 1.93μs 0.719ns 2.69ns 0.193 0 0 1.22 KB
#3076 SendReceive netcoreapp3.1 1.61μs 0.512ns 1.92ns 0.0161 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.66μs 1.38ns 5.33ns 0.329 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.09μs 2ns 7.73ns 0.0226 0 0 1.69 KB
#3076 EnrichedLog net472 4.8μs 1.84ns 6.89ns 0.33 0 0 2.08 KB
#3076 EnrichedLog netcoreapp3.1 4.15μs 2.2ns 8.51ns 0.0226 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #3076

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 1.121 1,013.50 1,135.65

Faster :tada: in #3076

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 1.118 929.93 831.47

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 832ns 0.252ns 0.944ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 776ns 1.78ns 6.88ns 0.00885 0 0 648 B
master StartFinishScope net472 1.01μs 0.27ns 1.05ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 930ns 0.337ns 1.26ns 0.0102 0 0 768 B
#3076 StartFinishSpan net472 834ns 0.355ns 1.33ns 0.104 0 0 658 B
#3076 StartFinishSpan netcoreapp3.1 784ns 1.06ns 4.11ns 0.00862 0 0 648 B
#3076 StartFinishScope net472 1.13μs 0.614ns 2.38ns 0.117 0 0 738 B
#3076 StartFinishScope netcoreapp3.1 831ns 0.243ns 0.875ns 0.0104 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.22μs 0.858ns 3.32ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.01μs 0.329ns 1.19ns 0.0102 0 0 768 B
#3076 RunOnMethodBegin net472 1.29μs 0.339ns 1.31ns 0.117 0 0 738 B
#3076 RunOnMethodBegin netcoreapp3.1 1E+03ns 0.318ns 1.19ns 0.0105 0 0 768 B

andrewlock avatar Aug 12 '22 16:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3076 compared to master:

  • 2 benchmarks are slower, with geometric mean 1.181
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 727μs 478ns 1.85μs 0.363 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 464μs 190ns 736ns 0 0 0 2.58 KB
#3076 WriteAndFlushEnrichedTraces net472 726μs 615ns 2.38μs 0.361 0 0 3.18 KB
#3076 WriteAndFlushEnrichedTraces netcoreapp3.1 452μs 264ns 988ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #3076

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net472 1.224 185.58 227.13

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 183ns 0.138ns 0.497ns 0.0676 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 242ns 0.307ns 1.19ns 0.00584 0 0 424 B
master AllCycleMoreComplexBody net472 186ns 0.993ns 4.86ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 236ns 0.253ns 0.948ns 0.00557 0 0 400 B
master BodyExtractorSimpleBody net472 257ns 0.2ns 0.776ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 219ns 0.222ns 0.831ns 0.00374 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.3μs 8.28ns 29.9ns 1.2 0.0214 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 10.7ns 41.2ns 0.0914 0 0 6.75 KB
#3076 AllCycleSimpleBody net472 183ns 0.123ns 0.444ns 0.0675 0 0 425 B
#3076 AllCycleSimpleBody netcoreapp3.1 236ns 0.353ns 1.37ns 0.0059 0 0 424 B
#3076 AllCycleMoreComplexBody net472 227ns 0.121ns 0.452ns 0.0637 0 0 401 B
#3076 AllCycleMoreComplexBody netcoreapp3.1 235ns 0.207ns 0.8ns 0.00548 0 0 400 B
#3076 BodyExtractorSimpleBody net472 268ns 0.172ns 0.645ns 0.0573 0 0 361 B
#3076 BodyExtractorSimpleBody netcoreapp3.1 233ns 0.428ns 1.6ns 0.00371 0 0 272 B
#3076 BodyExtractorMoreComplexBody net472 14.2μs 11.6ns 44.9ns 1.21 0.0213 0 7.62 KB
#3076 BodyExtractorMoreComplexBody netcoreapp3.1 12.3μs 7.81ns 29.2ns 0.0866 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 179μs 149ns 576ns 0.268 0 0 20.33 KB
#3076 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3076 SendRequest netcoreapp3.1 179μs 166ns 645ns 0.268 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.58μs 0.528ns 1.98ns 0.126 0.000785 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.3μs 0.329ns 1.23ns 0.011 0 0 824 B
#3076 ExecuteNonQuery net472 1.49μs 0.365ns 1.37ns 0.126 0.000744 0 794 B
#3076 ExecuteNonQuery netcoreapp3.1 1.28μs 0.41ns 1.54ns 0.0108 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #3076

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net472 1.139 2,052.99 2,338.73

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.05μs 0.45ns 1.68ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.39μs 0.707ns 2.65ns 0.0132 0 0 984 B
master CallElasticsearchAsync net472 2.35μs 0.63ns 2.36ns 0.18 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.42μs 0.59ns 2.21ns 0.0149 0 0 1.1 KB
#3076 CallElasticsearch net472 2.34μs 0.716ns 2.68ns 0.159 0 0 1 KB
#3076 CallElasticsearch netcoreapp3.1 1.41μs 0.516ns 2ns 0.0133 0 0 984 B
#3076 CallElasticsearchAsync net472 2.36μs 0.669ns 2.41ns 0.181 0 0 1.14 KB
#3076 CallElasticsearchAsync netcoreapp3.1 1.48μs 0.826ns 3.2ns 0.0148 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.4μs 4.98ns 19.3ns 0.199 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.53μs 3.04ns 11.8ns 0.0169 0 0 1.22 KB
#3076 ExecuteAsync net472 2.33μs 4.79ns 18.6ns 0.2 0 0 1.26 KB
#3076 ExecuteAsync netcoreapp3.1 1.55μs 2.37ns 8.54ns 0.0165 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.88μs 9.66ns 37.4ns 0.393 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.15μs 7.82ns 29.3ns 0.032 0 0 2.36 KB
#3076 SendAsync net472 4.94μs 13.3ns 51.5ns 0.394 0 0 2.48 KB
#3076 SendAsync netcoreapp3.1 3.21μs 7.99ns 31ns 0.0323 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.81μs 0.661ns 2.56ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.42μs 0.764ns 2.86ns 0.0229 0 0 1.73 KB
#3076 EnrichedLog net472 3.01μs 1.24ns 4.64ns 0.263 0 0 1.66 KB
#3076 EnrichedLog netcoreapp3.1 2.22μs 0.721ns 2.7ns 0.0232 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 150μs 116ns 433ns 0.665 0.222 0 4.5 KB
master EnrichedLog netcoreapp3.1 113μs 230ns 893ns 0 0 0 4.38 KB
#3076 EnrichedLog net472 152μs 186ns 719ns 0.677 0.226 0 4.5 KB
#3076 EnrichedLog netcoreapp3.1 114μs 184ns 713ns 0.0567 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.36μs 11.8ns 45.7ns 0.544 0.00265 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.13μs 4.23ns 15.8ns 0.0522 0 0 3.8 KB
#3076 EnrichedLog net472 5.33μs 13.8ns 53.6ns 0.546 0.00266 0 3.43 KB
#3076 EnrichedLog netcoreapp3.1 4.21μs 9.27ns 35.9ns 0.0522 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.97μs 0.712ns 2.76ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.7μs 0.415ns 1.5ns 0.0162 0 0 1.21 KB
#3076 SendReceive net472 2.07μs 0.893ns 3.46ns 0.193 0 0 1.22 KB
#3076 SendReceive netcoreapp3.1 1.65μs 0.568ns 2.05ns 0.0165 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.81μs 1.34ns 5.18ns 0.328 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.18μs 3.64ns 14.1ns 0.0229 0 0 1.69 KB
#3076 EnrichedLog net472 4.99μs 1.36ns 5.27ns 0.328 0 0 2.08 KB
#3076 EnrichedLog netcoreapp3.1 4.21μs 1.87ns 7.01ns 0.0231 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 812ns 0.828ns 3.21ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 771ns 0.652ns 2.52ns 0.00883 0 0 648 B
master StartFinishScope net472 1.1μs 1.41ns 5.28ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 931ns 1.21ns 4.54ns 0.0101 0 0 768 B
#3076 StartFinishSpan net472 783ns 0.573ns 2.07ns 0.104 0 0 658 B
#3076 StartFinishSpan netcoreapp3.1 720ns 0.654ns 2.53ns 0.00865 0 0 648 B
#3076 StartFinishScope net472 1.14μs 0.971ns 3.76ns 0.117 0 0 738 B
#3076 StartFinishScope netcoreapp3.1 924ns 0.766ns 2.87ns 0.0102 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.2μs 1.4ns 5.44ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.03μs 6.85ns 63.9ns 0.0103 0 0 768 B
#3076 RunOnMethodBegin net472 1.21μs 1.28ns 4.96ns 0.117 0 0 738 B
#3076 RunOnMethodBegin netcoreapp3.1 1.08μs 0.834ns 3.23ns 0.0102 0 0 768 B

andrewlock avatar Aug 15 '22 10:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3076 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 727μs 478ns 1.85μs 0.363 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 464μs 190ns 736ns 0 0 0 2.58 KB
#3076 WriteAndFlushEnrichedTraces net472 730μs 595ns 2.3μs 0.365 0 0 3.18 KB
#3076 WriteAndFlushEnrichedTraces netcoreapp3.1 464μs 268ns 1μs 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 183ns 0.138ns 0.497ns 0.0676 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 242ns 0.307ns 1.19ns 0.00584 0 0 424 B
master AllCycleMoreComplexBody net472 186ns 0.993ns 4.86ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 236ns 0.253ns 0.948ns 0.00557 0 0 400 B
master BodyExtractorSimpleBody net472 257ns 0.2ns 0.776ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 219ns 0.222ns 0.831ns 0.00374 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.3μs 8.28ns 29.9ns 1.2 0.0214 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 10.7ns 41.2ns 0.0914 0 0 6.75 KB
#3076 AllCycleSimpleBody net472 182ns 0.183ns 0.707ns 0.0675 9.2E-05 0 425 B
#3076 AllCycleSimpleBody netcoreapp3.1 237ns 0.212ns 0.82ns 0.00588 0 0 424 B
#3076 AllCycleMoreComplexBody net472 183ns 0.2ns 0.774ns 0.0637 0 0 401 B
#3076 AllCycleMoreComplexBody netcoreapp3.1 236ns 0.335ns 1.3ns 0.0054 0 0 400 B
#3076 BodyExtractorSimpleBody net472 264ns 0.623ns 2.41ns 0.0573 0 0 361 B
#3076 BodyExtractorSimpleBody netcoreapp3.1 222ns 0.24ns 0.898ns 0.00368 0 0 272 B
#3076 BodyExtractorMoreComplexBody net472 14.3μs 10.6ns 39.6ns 1.21 0.0215 0 7.62 KB
#3076 BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 11.2ns 41.9ns 0.0907 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 179μs 149ns 576ns 0.268 0 0 20.33 KB
#3076 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3076 SendRequest netcoreapp3.1 179μs 160ns 621ns 0.268 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.58μs 0.528ns 1.98ns 0.126 0.000785 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.3μs 0.329ns 1.23ns 0.011 0 0 824 B
#3076 ExecuteNonQuery net472 1.47μs 0.403ns 1.51ns 0.126 0.000732 0 794 B
#3076 ExecuteNonQuery netcoreapp3.1 1.26μs 0.573ns 2.14ns 0.0114 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.05μs 0.45ns 1.68ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.39μs 0.707ns 2.65ns 0.0132 0 0 984 B
master CallElasticsearchAsync net472 2.35μs 0.63ns 2.36ns 0.18 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.42μs 0.59ns 2.21ns 0.0149 0 0 1.1 KB
#3076 CallElasticsearch net472 2.19μs 0.406ns 1.57ns 0.159 0 0 1 KB
#3076 CallElasticsearch netcoreapp3.1 1.36μs 1.14ns 4.41ns 0.0129 0 0 984 B
#3076 CallElasticsearchAsync net472 2.35μs 0.843ns 3.27ns 0.181 0 0 1.14 KB
#3076 CallElasticsearchAsync netcoreapp3.1 1.44μs 0.412ns 1.54ns 0.0152 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.4μs 4.98ns 19.3ns 0.199 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.53μs 3.04ns 11.8ns 0.0169 0 0 1.22 KB
#3076 ExecuteAsync net472 2.45μs 6.96ns 27ns 0.199 0 0 1.26 KB
#3076 ExecuteAsync netcoreapp3.1 1.54μs 3.15ns 12.2ns 0.0166 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.88μs 9.66ns 37.4ns 0.393 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.15μs 7.82ns 29.3ns 0.032 0 0 2.36 KB
#3076 SendAsync net472 5.02μs 4.06ns 15.7ns 0.393 0 0 2.48 KB
#3076 SendAsync netcoreapp3.1 3.25μs 4.85ns 18.8ns 0.0312 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.81μs 0.661ns 2.56ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.42μs 0.764ns 2.86ns 0.0229 0 0 1.73 KB
#3076 EnrichedLog net472 2.82μs 1.92ns 7.17ns 0.263 0 0 1.66 KB
#3076 EnrichedLog netcoreapp3.1 2.45μs 0.888ns 3.32ns 0.0234 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 150μs 116ns 433ns 0.665 0.222 0 4.5 KB
master EnrichedLog netcoreapp3.1 113μs 230ns 893ns 0 0 0 4.38 KB
#3076 EnrichedLog net472 147μs 221ns 856ns 0.666 0.222 0 4.5 KB
#3076 EnrichedLog netcoreapp3.1 115μs 154ns 575ns 0.0564 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.36μs 11.8ns 45.7ns 0.544 0.00265 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.13μs 4.23ns 15.8ns 0.0522 0 0 3.8 KB
#3076 EnrichedLog net472 5.49μs 6.75ns 26.1ns 0.544 0.00275 0 3.43 KB
#3076 EnrichedLog netcoreapp3.1 4.25μs 6.85ns 26.5ns 0.0497 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.97μs 0.712ns 2.76ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.7μs 0.415ns 1.5ns 0.0162 0 0 1.21 KB
#3076 SendReceive net472 1.99μs 1.98ns 7.39ns 0.193 0 0 1.22 KB
#3076 SendReceive netcoreapp3.1 1.71μs 0.716ns 2.68ns 0.0163 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.81μs 1.34ns 5.18ns 0.328 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.18μs 3.64ns 14.1ns 0.0229 0 0 1.69 KB
#3076 EnrichedLog net472 4.71μs 0.882ns 3.42ns 0.329 0 0 2.08 KB
#3076 EnrichedLog netcoreapp3.1 4.18μs 2.48ns 9.26ns 0.0229 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 812ns 0.828ns 3.21ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 771ns 0.652ns 2.52ns 0.00883 0 0 648 B
master StartFinishScope net472 1.1μs 1.41ns 5.28ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 931ns 1.21ns 4.54ns 0.0101 0 0 768 B
#3076 StartFinishSpan net472 812ns 0.426ns 1.65ns 0.105 0 0 658 B
#3076 StartFinishSpan netcoreapp3.1 825ns 0.596ns 2.31ns 0.00873 0 0 648 B
#3076 StartFinishScope net472 1.11μs 2.67ns 10.3ns 0.117 0 0 738 B
#3076 StartFinishScope netcoreapp3.1 838ns 0.618ns 2.23ns 0.0105 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.2μs 1.4ns 5.44ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.03μs 6.85ns 63.9ns 0.0103 0 0 768 B
#3076 RunOnMethodBegin net472 1.29μs 0.806ns 3.01ns 0.117 0 0 738 B
#3076 RunOnMethodBegin netcoreapp3.1 955ns 0.237ns 0.82ns 0.0105 0 0 768 B

andrewlock avatar Aug 15 '22 17:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3076 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 730μs 176ns 635ns 0.363 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 458μs 98ns 367ns 0 0 0 2.58 KB
#3076 WriteAndFlushEnrichedTraces net472 728μs 353ns 1.37μs 0.363 0 0 3.18 KB
#3076 WriteAndFlushEnrichedTraces netcoreapp3.1 464μs 217ns 812ns 0 0 0 2.59 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 182ns 0.151ns 0.524ns 0.0675 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 236ns 0.216ns 0.807ns 0.00582 0 0 424 B
master AllCycleMoreComplexBody net472 227ns 0.126ns 0.435ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 238ns 0.33ns 1.28ns 0.00545 0 0 400 B
master BodyExtractorSimpleBody net472 261ns 0.229ns 0.887ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 223ns 0.273ns 1.02ns 0.00367 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.4μs 9.68ns 36.2ns 1.21 0.0215 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.4μs 15.3ns 59.4ns 0.0921 0 0 6.75 KB
#3076 AllCycleSimpleBody net472 182ns 0.184ns 0.689ns 0.0675 0 0 425 B
#3076 AllCycleSimpleBody netcoreapp3.1 234ns 0.339ns 1.31ns 0.00584 0 0 424 B
#3076 AllCycleMoreComplexBody net472 205ns 0.145ns 0.561ns 0.0637 0 0 401 B
#3076 AllCycleMoreComplexBody netcoreapp3.1 238ns 0.676ns 2.62ns 0.00535 0 0 400 B
#3076 BodyExtractorSimpleBody net472 279ns 0.43ns 1.67ns 0.0573 0 0 361 B
#3076 BodyExtractorSimpleBody netcoreapp3.1 219ns 0.39ns 1.46ns 0.00374 0 0 272 B
#3076 BodyExtractorMoreComplexBody net472 14.2μs 9.79ns 37.9ns 1.21 0.0213 0 7.62 KB
#3076 BodyExtractorMoreComplexBody netcoreapp3.1 12.4μs 16.3ns 63.2ns 0.0934 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 181μs 271ns 1.05μs 0.271 0 0 20.33 KB
#3076 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3076 SendRequest netcoreapp3.1 180μs 134ns 517ns 0.269 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.54μs 0.406ns 1.57ns 0.126 0.000771 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.28μs 0.408ns 1.58ns 0.0109 0 0 824 B
#3076 ExecuteNonQuery net472 1.5μs 0.699ns 2.71ns 0.126 0.000749 0 794 B
#3076 ExecuteNonQuery netcoreapp3.1 1.23μs 0.39ns 1.51ns 0.0111 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.17μs 0.595ns 2.3ns 0.158 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.45μs 1.01ns 3.9ns 0.0136 0 0 984 B
master CallElasticsearchAsync net472 2.32μs 0.846ns 3.28ns 0.18 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.42μs 0.344ns 1.29ns 0.015 0 0 1.1 KB
#3076 CallElasticsearch net472 2.25μs 0.443ns 1.72ns 0.159 0 0 1 KB
#3076 CallElasticsearch netcoreapp3.1 1.43μs 0.691ns 2.67ns 0.0129 0 0 984 B
#3076 CallElasticsearchAsync net472 2.34μs 0.555ns 2.08ns 0.18 0 0 1.14 KB
#3076 CallElasticsearchAsync netcoreapp3.1 1.47μs 0.517ns 1.93ns 0.0147 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.33μs 4.65ns 18ns 0.199 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.48μs 2.78ns 10.8ns 0.016 0 0 1.22 KB
#3076 ExecuteAsync net472 2.39μs 5.66ns 21.9ns 0.199 0 0 1.26 KB
#3076 ExecuteAsync netcoreapp3.1 1.58μs 4.39ns 16.4ns 0.0162 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.08μs 9.96ns 38.6ns 0.394 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.18μs 5.39ns 20.9ns 0.032 0 0 2.36 KB
#3076 SendAsync net472 4.89μs 7.33ns 25.4ns 0.394 0 0 2.48 KB
#3076 SendAsync netcoreapp3.1 3.18μs 5.43ns 21ns 0.0312 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.67μs 2.44ns 9.14ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.33μs 0.898ns 3.48ns 0.0233 0 0 1.73 KB
#3076 EnrichedLog net472 2.77μs 1.34ns 5.02ns 0.264 0 0 1.66 KB
#3076 EnrichedLog netcoreapp3.1 2.37μs 0.694ns 2.5ns 0.0237 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 149μs 162ns 626ns 0.674 0.225 0 4.5 KB
master EnrichedLog netcoreapp3.1 114μs 118ns 456ns 0.057 0 0 4.38 KB
#3076 EnrichedLog net472 148μs 89.3ns 346ns 0.668 0.223 0 4.5 KB
#3076 EnrichedLog netcoreapp3.1 113μs 103ns 371ns 0.0565 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.42μs 7.23ns 27.1ns 0.543 0.00269 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.16μs 5.1ns 19.7ns 0.0521 0 0 3.8 KB
#3076 EnrichedLog net472 5.41μs 17.1ns 64ns 0.545 0.00265 0 3.43 KB
#3076 EnrichedLog netcoreapp3.1 4.16μs 9.63ns 37.3ns 0.0517 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.08μs 1.91ns 7.41ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.61μs 0.474ns 1.77ns 0.0161 0 0 1.21 KB
#3076 SendReceive net472 2.05μs 0.434ns 1.63ns 0.193 0 0 1.22 KB
#3076 SendReceive netcoreapp3.1 1.71μs 0.659ns 2.38ns 0.0162 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.75μs 0.755ns 2.61ns 0.33 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4μs 1.66ns 6.44ns 0.0221 0 0 1.69 KB
#3076 EnrichedLog net472 4.67μs 2.36ns 8.84ns 0.33 0 0 2.08 KB
#3076 EnrichedLog netcoreapp3.1 4.18μs 3.91ns 15.2ns 0.0231 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 840ns 0.546ns 2.12ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 745ns 0.309ns 1.16ns 0.00858 0 0 648 B
master StartFinishScope net472 1.1μs 0.753ns 2.72ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 932ns 0.958ns 3.71ns 0.0103 0 0 768 B
#3076 StartFinishSpan net472 892ns 0.807ns 3.02ns 0.104 0 0 658 B
#3076 StartFinishSpan netcoreapp3.1 720ns 0.344ns 1.24ns 0.00869 0 0 648 B
#3076 StartFinishScope net472 1.11μs 0.801ns 3.1ns 0.117 0 0 738 B
#3076 StartFinishScope netcoreapp3.1 896ns 0.628ns 2.35ns 0.0102 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.3μs 0.7ns 2.62ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.01μs 0.553ns 2.14ns 0.0106 0 0 768 B
#3076 RunOnMethodBegin net472 1.2μs 0.971ns 3.63ns 0.117 0 0 738 B
#3076 RunOnMethodBegin netcoreapp3.1 1.06μs 1.38ns 5.35ns 0.0107 0 0 768 B

andrewlock avatar Aug 16 '22 09:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3076 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 736μs 419ns 1.62μs 0.361 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 460μs 182ns 657ns 0 0 0 2.58 KB
#3076 WriteAndFlushEnrichedTraces net472 738μs 488ns 1.83μs 0.372 0 0 3.18 KB
#3076 WriteAndFlushEnrichedTraces netcoreapp3.1 471μs 685ns 2.65μs 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 182ns 0.203ns 0.786ns 0.0676 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 237ns 0.242ns 0.935ns 0.00582 0 0 424 B
master AllCycleMoreComplexBody net472 183ns 0.262ns 1.01ns 0.0638 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 236ns 0.169ns 0.633ns 0.00546 0 0 400 B
master BodyExtractorSimpleBody net472 252ns 0.236ns 0.914ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 227ns 0.356ns 1.33ns 0.00365 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.3μs 9.69ns 37.5ns 1.2 0.0215 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 14.2ns 53.3ns 0.0927 0 0 6.75 KB
#3076 AllCycleSimpleBody net472 182ns 0.189ns 0.732ns 0.0675 0 0 425 B
#3076 AllCycleSimpleBody netcoreapp3.1 234ns 0.54ns 2.09ns 0.00577 0 0 424 B
#3076 AllCycleMoreComplexBody net472 196ns 0.262ns 0.979ns 0.0637 0 0 401 B
#3076 AllCycleMoreComplexBody netcoreapp3.1 235ns 0.178ns 0.668ns 0.00554 0 0 400 B
#3076 BodyExtractorSimpleBody net472 260ns 0.361ns 1.35ns 0.0574 0 0 361 B
#3076 BodyExtractorSimpleBody netcoreapp3.1 229ns 0.277ns 0.998ns 0.00368 0 0 272 B
#3076 BodyExtractorMoreComplexBody net472 14.8μs 8.34ns 31.2ns 1.21 0.0148 0 7.62 KB
#3076 BodyExtractorMoreComplexBody netcoreapp3.1 12.4μs 10.6ns 40.9ns 0.0922 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 179μs 126ns 488ns 0.267 0 0 20.33 KB
#3076 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3076 SendRequest netcoreapp3.1 178μs 134ns 484ns 0.268 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.56μs 0.464ns 1.74ns 0.126 0.000776 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.33μs 0.425ns 1.64ns 0.0106 0 0 824 B
#3076 ExecuteNonQuery net472 1.62μs 0.848ns 3.28ns 0.126 0.000798 0 794 B
#3076 ExecuteNonQuery netcoreapp3.1 1.22μs 0.73ns 2.63ns 0.011 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.15μs 0.526ns 2.04ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.42μs 0.626ns 2.34ns 0.0135 0 0 984 B
master CallElasticsearchAsync net472 2.4μs 1.2ns 4.63ns 0.18 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.49μs 0.711ns 2.56ns 0.0148 0 0 1.1 KB
#3076 CallElasticsearch net472 2.14μs 0.485ns 1.88ns 0.159 0 0 1 KB
#3076 CallElasticsearch netcoreapp3.1 1.34μs 0.483ns 1.87ns 0.0134 0 0 984 B
#3076 CallElasticsearchAsync net472 2.27μs 1.95ns 7.56ns 0.18 0 0 1.14 KB
#3076 CallElasticsearchAsync netcoreapp3.1 1.4μs 0.304ns 1.09ns 0.0147 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.41μs 9.49ns 36.8ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.57μs 3.83ns 14.3ns 0.0158 0 0 1.22 KB
#3076 ExecuteAsync net472 2.43μs 7.33ns 26.4ns 0.2 0 0 1.26 KB
#3076 ExecuteAsync netcoreapp3.1 1.53μs 1.5ns 5.62ns 0.0169 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.09μs 16.5ns 61.8ns 0.394 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.17μs 8.34ns 32.3ns 0.0315 0 0 2.36 KB
#3076 SendAsync net472 5μs 14.9ns 57.5ns 0.393 0 0 2.48 KB
#3076 SendAsync netcoreapp3.1 3.15μs 9.39ns 36.4ns 0.0325 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.77μs 2.2ns 8.52ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.39μs 0.768ns 2.97ns 0.0227 0 0 1.73 KB
#3076 EnrichedLog net472 2.86μs 2.05ns 7.95ns 0.263 0 0 1.66 KB
#3076 EnrichedLog netcoreapp3.1 2.39μs 1.42ns 5.33ns 0.0229 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 147μs 104ns 375ns 0.67 0.223 0 4.5 KB
master EnrichedLog netcoreapp3.1 116μs 316ns 1.14μs 0.0579 0 0 4.38 KB
#3076 EnrichedLog net472 150μs 151ns 584ns 0.671 0.224 0 4.5 KB
#3076 EnrichedLog netcoreapp3.1 115μs 70.7ns 255ns 0.0575 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.3μs 11.2ns 43.5ns 0.545 0.00265 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.04μs 8.04ns 31.1ns 0.0505 0 0 3.8 KB
#3076 EnrichedLog net472 5.42μs 7.29ns 28.2ns 0.546 0.00269 0 3.43 KB
#3076 EnrichedLog netcoreapp3.1 4.19μs 3.21ns 12ns 0.0522 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.99μs 1.28ns 4.97ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.65μs 4.47ns 16.7ns 0.0163 0 0 1.21 KB
#3076 SendReceive net472 2.09μs 1.74ns 6.28ns 0.193 0 0 1.22 KB
#3076 SendReceive netcoreapp3.1 1.7μs 1.09ns 4.06ns 0.017 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.79μs 2.24ns 8.67ns 0.33 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.16μs 2.12ns 8.23ns 0.0228 0 0 1.69 KB
#3076 EnrichedLog net472 4.72μs 2.05ns 7.4ns 0.33 0 0 2.08 KB
#3076 EnrichedLog netcoreapp3.1 4.2μs 3.29ns 12.7ns 0.0231 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 878ns 0.418ns 1.62ns 0.105 0 0 658 B
master StartFinishSpan netcoreapp3.1 751ns 0.304ns 1.14ns 0.00865 0 0 648 B
master StartFinishScope net472 1.04μs 0.545ns 2.04ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 862ns 0.397ns 1.49ns 0.0103 0 0 768 B
#3076 StartFinishSpan net472 860ns 0.652ns 2.53ns 0.105 0 0 658 B
#3076 StartFinishSpan netcoreapp3.1 772ns 0.154ns 0.576ns 0.00846 0 0 648 B
#3076 StartFinishScope net472 1.02μs 0.194ns 0.752ns 0.117 0 0 738 B
#3076 StartFinishScope netcoreapp3.1 843ns 0.557ns 2.16ns 0.0101 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.29μs 0.358ns 1.38ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.09μs 0.64ns 2.39ns 0.0103 0 0 768 B
#3076 RunOnMethodBegin net472 1.22μs 0.332ns 1.29ns 0.117 0 0 738 B
#3076 RunOnMethodBegin netcoreapp3.1 1.07μs 0.458ns 1.72ns 0.0103 0 0 768 B

andrewlock avatar Aug 16 '22 15:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3076 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.268
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 736μs 419ns 1.62μs 0.361 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 460μs 182ns 657ns 0 0 0 2.58 KB
#3076 WriteAndFlushEnrichedTraces net472 730μs 334ns 1.25μs 0.363 0 0 3.18 KB
#3076 WriteAndFlushEnrichedTraces netcoreapp3.1 463μs 93.1ns 336ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #3076

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AppSecBodyBenchmark.AllCycleSimpleBody‑net472 1.268 181.82 230.47

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 182ns 0.203ns 0.786ns 0.0676 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 237ns 0.242ns 0.935ns 0.00582 0 0 424 B
master AllCycleMoreComplexBody net472 183ns 0.262ns 1.01ns 0.0638 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 236ns 0.169ns 0.633ns 0.00546 0 0 400 B
master BodyExtractorSimpleBody net472 252ns 0.236ns 0.914ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 227ns 0.356ns 1.33ns 0.00365 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.3μs 9.69ns 37.5ns 1.2 0.0215 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 14.2ns 53.3ns 0.0927 0 0 6.75 KB
#3076 AllCycleSimpleBody net472 231ns 0.2ns 0.776ns 0.0675 0 0 425 B
#3076 AllCycleSimpleBody netcoreapp3.1 237ns 0.281ns 1.09ns 0.00592 0 0 424 B
#3076 AllCycleMoreComplexBody net472 184ns 0.14ns 0.541ns 0.0637 0 0 401 B
#3076 AllCycleMoreComplexBody netcoreapp3.1 237ns 0.221ns 0.857ns 0.00547 0 0 400 B
#3076 BodyExtractorSimpleBody net472 261ns 0.18ns 0.648ns 0.0573 0 0 361 B
#3076 BodyExtractorSimpleBody netcoreapp3.1 250ns 0.316ns 1.18ns 0.00366 0 0 272 B
#3076 BodyExtractorMoreComplexBody net472 14.5μs 7.91ns 29.6ns 1.21 0.0217 0 7.62 KB
#3076 BodyExtractorMoreComplexBody netcoreapp3.1 12.4μs 17.7ns 68.7ns 0.092 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 179μs 126ns 488ns 0.267 0 0 20.33 KB
#3076 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3076 SendRequest netcoreapp3.1 180μs 149ns 556ns 0.179 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.56μs 0.464ns 1.74ns 0.126 0.000776 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.33μs 0.425ns 1.64ns 0.0106 0 0 824 B
#3076 ExecuteNonQuery net472 1.53μs 0.725ns 2.71ns 0.126 0.000765 0 794 B
#3076 ExecuteNonQuery netcoreapp3.1 1.22μs 0.232ns 0.803ns 0.011 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.15μs 0.526ns 2.04ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.42μs 0.626ns 2.34ns 0.0135 0 0 984 B
master CallElasticsearchAsync net472 2.4μs 1.2ns 4.63ns 0.18 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.49μs 0.711ns 2.56ns 0.0148 0 0 1.1 KB
#3076 CallElasticsearch net472 2.18μs 0.597ns 2.23ns 0.158 0 0 1 KB
#3076 CallElasticsearch netcoreapp3.1 1.39μs 0.572ns 2.22ns 0.0131 0 0 984 B
#3076 CallElasticsearchAsync net472 2.31μs 0.756ns 2.73ns 0.181 0 0 1.14 KB
#3076 CallElasticsearchAsync netcoreapp3.1 1.47μs 0.817ns 3.06ns 0.0147 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.41μs 9.49ns 36.8ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.57μs 3.83ns 14.3ns 0.0158 0 0 1.22 KB
#3076 ExecuteAsync net472 2.42μs 2.53ns 9.8ns 0.199 0 0 1.26 KB
#3076 ExecuteAsync netcoreapp3.1 1.53μs 1.33ns 4.78ns 0.0169 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.09μs 16.5ns 61.8ns 0.394 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.17μs 8.34ns 32.3ns 0.0315 0 0 2.36 KB
#3076 SendAsync net472 4.88μs 4.17ns 16.2ns 0.393 0 0 2.48 KB
#3076 SendAsync netcoreapp3.1 3.22μs 3.91ns 15.2ns 0.0307 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.77μs 2.2ns 8.52ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.39μs 0.768ns 2.97ns 0.0227 0 0 1.73 KB
#3076 EnrichedLog net472 2.92μs 2.72ns 10.5ns 0.263 0 0 1.66 KB
#3076 EnrichedLog netcoreapp3.1 2.39μs 0.941ns 3.39ns 0.024 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 147μs 104ns 375ns 0.67 0.223 0 4.5 KB
master EnrichedLog netcoreapp3.1 116μs 316ns 1.14μs 0.0579 0 0 4.38 KB
#3076 EnrichedLog net472 149μs 209ns 808ns 0.674 0.225 0 4.5 KB
#3076 EnrichedLog netcoreapp3.1 115μs 355ns 1.33μs 0.0573 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.3μs 11.2ns 43.5ns 0.545 0.00265 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.04μs 8.04ns 31.1ns 0.0505 0 0 3.8 KB
#3076 EnrichedLog net472 5.41μs 8.4ns 32.5ns 0.544 0.00271 0 3.43 KB
#3076 EnrichedLog netcoreapp3.1 4.24μs 3.39ns 13.1ns 0.0525 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.99μs 1.28ns 4.97ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.65μs 4.47ns 16.7ns 0.0163 0 0 1.21 KB
#3076 SendReceive net472 2.05μs 0.984ns 3.68ns 0.193 0 0 1.22 KB
#3076 SendReceive netcoreapp3.1 1.58μs 6.51ns 25.2ns 0.0163 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.79μs 2.24ns 8.67ns 0.33 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.16μs 2.12ns 8.23ns 0.0228 0 0 1.69 KB
#3076 EnrichedLog net472 4.76μs 1.64ns 6.37ns 0.328 0 0 2.08 KB
#3076 EnrichedLog netcoreapp3.1 3.9μs 1.54ns 5.97ns 0.0217 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 878ns 0.418ns 1.62ns 0.105 0 0 658 B
master StartFinishSpan netcoreapp3.1 751ns 0.304ns 1.14ns 0.00865 0 0 648 B
master StartFinishScope net472 1.04μs 0.545ns 2.04ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 862ns 0.397ns 1.49ns 0.0103 0 0 768 B
#3076 StartFinishSpan net472 803ns 0.214ns 0.829ns 0.104 0 0 658 B
#3076 StartFinishSpan netcoreapp3.1 718ns 0.619ns 2.4ns 0.00894 0 0 648 B
#3076 StartFinishScope net472 1.05μs 0.208ns 0.752ns 0.117 0 0 738 B
#3076 StartFinishScope netcoreapp3.1 871ns 0.553ns 2.07ns 0.0104 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.29μs 0.358ns 1.38ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.09μs 0.64ns 2.39ns 0.0103 0 0 768 B
#3076 RunOnMethodBegin net472 1.29μs 0.432ns 1.67ns 0.117 0 0 738 B
#3076 RunOnMethodBegin netcoreapp3.1 1μs 0.345ns 1.29ns 0.01 0 0 768 B

andrewlock avatar Aug 16 '22 17:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3076 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 727μs 259ns 970ns 0.363 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 475μs 749ns 2.59μs 0 0 0 2.58 KB
#3076 WriteAndFlushEnrichedTraces net472 724μs 384ns 1.49μs 0.363 0 0 3.18 KB
#3076 WriteAndFlushEnrichedTraces netcoreapp3.1 474μs 269ns 1.01μs 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 181ns 0.224ns 0.839ns 0.0675 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 236ns 0.282ns 1.09ns 0.00588 0 0 424 B
master AllCycleMoreComplexBody net472 184ns 0.509ns 1.97ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 243ns 0.274ns 0.987ns 0.00541 0 0 400 B
master BodyExtractorSimpleBody net472 272ns 0.36ns 1.39ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 216ns 0.2ns 0.774ns 0.00367 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.3μs 14.3ns 55.3ns 1.21 0.0215 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 14.2ns 55.1ns 0.0904 0 0 6.75 KB
#3076 AllCycleSimpleBody net472 181ns 0.131ns 0.492ns 0.0676 9.14E-05 0 425 B
#3076 AllCycleSimpleBody netcoreapp3.1 235ns 0.417ns 1.5ns 0.00577 0 0 424 B
#3076 AllCycleMoreComplexBody net472 182ns 0.166ns 0.642ns 0.0637 0 0 401 B
#3076 AllCycleMoreComplexBody netcoreapp3.1 237ns 0.288ns 1.12ns 0.00547 0 0 400 B
#3076 BodyExtractorSimpleBody net472 252ns 0.209ns 0.781ns 0.0573 0 0 361 B
#3076 BodyExtractorSimpleBody netcoreapp3.1 217ns 0.361ns 1.4ns 0.00368 0 0 272 B
#3076 BodyExtractorMoreComplexBody net472 14.4μs 12.5ns 48.4ns 1.21 0.0215 0 7.62 KB
#3076 BodyExtractorMoreComplexBody netcoreapp3.1 12.4μs 17.4ns 67.2ns 0.092 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 181μs 164ns 615ns 0.181 0 0 20.33 KB
#3076 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3076 SendRequest netcoreapp3.1 179μs 113ns 438ns 0.268 0 0 20.34 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.54μs 0.423ns 1.58ns 0.126 0.000773 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.32μs 0.395ns 1.53ns 0.0113 0 0 824 B
#3076 ExecuteNonQuery net472 1.48μs 0.472ns 1.83ns 0.126 0.00074 0 794 B
#3076 ExecuteNonQuery netcoreapp3.1 1.25μs 0.335ns 1.21ns 0.0113 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.11μs 0.614ns 2.38ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.4μs 2.43ns 9.4ns 0.0132 0 0 984 B
master CallElasticsearchAsync net472 2.43μs 1.2ns 4.63ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.41μs 0.52ns 2.01ns 0.0147 0 0 1.1 KB
#3076 CallElasticsearch net472 2.16μs 0.769ns 2.98ns 0.159 0 0 1 KB
#3076 CallElasticsearch netcoreapp3.1 1.45μs 1.13ns 4.21ns 0.013 0 0 984 B
#3076 CallElasticsearchAsync net472 2.3μs 0.775ns 3ns 0.18 0 0 1.14 KB
#3076 CallElasticsearchAsync netcoreapp3.1 1.46μs 1.14ns 4.26ns 0.0152 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.37μs 9.46ns 36.6ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.48μs 1.42ns 4.93ns 0.0166 0 0 1.22 KB
#3076 ExecuteAsync net472 2.42μs 6.07ns 23.5ns 0.199 0 0 1.26 KB
#3076 ExecuteAsync netcoreapp3.1 1.53μs 3.93ns 15.2ns 0.0166 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.85μs 14.5ns 56.2ns 0.394 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.18μs 6.53ns 25.3ns 0.032 0 0 2.36 KB
#3076 SendAsync net472 4.85μs 9.26ns 34.6ns 0.392 0 0 2.48 KB
#3076 SendAsync netcoreapp3.1 3.14μs 6.78ns 26.3ns 0.0314 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.79μs 1.05ns 3.94ns 0.264 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.3μs 0.877ns 3.4ns 0.023 0 0 1.73 KB
#3076 EnrichedLog net472 2.86μs 1.4ns 5.43ns 0.263 0 0 1.66 KB
#3076 EnrichedLog netcoreapp3.1 2.41μs 0.593ns 2.22ns 0.0241 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 150μs 168ns 652ns 0.674 0.225 0 4.5 KB
master EnrichedLog netcoreapp3.1 116μs 155ns 599ns 0 0 0 4.38 KB
#3076 EnrichedLog net472 149μs 65.9ns 255ns 0.676 0.225 0 4.5 KB
#3076 EnrichedLog netcoreapp3.1 113μs 255ns 988ns 0 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.28μs 9.7ns 36.3ns 0.545 0.00266 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.18μs 8.2ns 29.6ns 0.0519 0 0 3.8 KB
#3076 EnrichedLog net472 5.32μs 14.3ns 55.4ns 0.545 0.00263 0 3.43 KB
#3076 EnrichedLog netcoreapp3.1 4.26μs 14.1ns 54.7ns 0.0526 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.03μs 0.517ns 2ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.6μs 0.884ns 3.42ns 0.0161 0 0 1.21 KB
#3076 SendReceive net472 2μs 1.93ns 7.48ns 0.194 0 0 1.22 KB
#3076 SendReceive netcoreapp3.1 1.63μs 0.908ns 3.4ns 0.0163 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.73μs 1.48ns 5.73ns 0.328 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.08μs 2.71ns 10.5ns 0.0224 0 0 1.69 KB
#3076 EnrichedLog net472 4.68μs 1.61ns 6.04ns 0.33 0 0 2.08 KB
#3076 EnrichedLog netcoreapp3.1 4.22μs 2.55ns 9.89ns 0.0232 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 855ns 0.547ns 2.12ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 764ns 0.713ns 2.76ns 0.00874 0 0 648 B
master StartFinishScope net472 1.08μs 0.865ns 3.35ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 882ns 0.549ns 2.13ns 0.0101 0 0 768 B
#3076 StartFinishSpan net472 864ns 0.834ns 3.23ns 0.104 0 0 658 B
#3076 StartFinishSpan netcoreapp3.1 749ns 0.687ns 2.66ns 0.00892 0 0 648 B
#3076 StartFinishScope net472 1.02μs 1.01ns 3.89ns 0.117 0 0 738 B
#3076 StartFinishScope netcoreapp3.1 860ns 0.537ns 2.08ns 0.0103 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.29μs 0.811ns 3.14ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.02μs 0.626ns 2.42ns 0.0103 0 0 768 B
#3076 RunOnMethodBegin net472 1.2μs 1.14ns 4.41ns 0.117 0 0 738 B
#3076 RunOnMethodBegin netcoreapp3.1 985ns 0.493ns 1.84ns 0.0103 0 0 768 B

andrewlock avatar Aug 17 '22 08:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3076 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.262
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 727μs 259ns 970ns 0.363 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 475μs 749ns 2.59μs 0 0 0 2.58 KB
#3076 WriteAndFlushEnrichedTraces net472 726μs 640ns 2.48μs 0.363 0 0 3.18 KB
#3076 WriteAndFlushEnrichedTraces netcoreapp3.1 470μs 243ns 909ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #3076

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net472 1.262 183.49 231.50

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 181ns 0.224ns 0.839ns 0.0675 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 236ns 0.282ns 1.09ns 0.00588 0 0 424 B
master AllCycleMoreComplexBody net472 184ns 0.509ns 1.97ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 243ns 0.274ns 0.987ns 0.00541 0 0 400 B
master BodyExtractorSimpleBody net472 272ns 0.36ns 1.39ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 216ns 0.2ns 0.774ns 0.00367 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.3μs 14.3ns 55.3ns 1.21 0.0215 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 14.2ns 55.1ns 0.0904 0 0 6.75 KB
#3076 AllCycleSimpleBody net472 182ns 0.123ns 0.459ns 0.0676 9.19E-05 0 425 B
#3076 AllCycleSimpleBody netcoreapp3.1 236ns 0.265ns 1.02ns 0.0059 0 0 424 B
#3076 AllCycleMoreComplexBody net472 231ns 0.217ns 0.812ns 0.0637 0 0 401 B
#3076 AllCycleMoreComplexBody netcoreapp3.1 238ns 0.232ns 0.898ns 0.00541 0 0 400 B
#3076 BodyExtractorSimpleBody net472 256ns 0.098ns 0.353ns 0.0574 0 0 361 B
#3076 BodyExtractorSimpleBody netcoreapp3.1 222ns 0.33ns 1.24ns 0.00368 0 0 272 B
#3076 BodyExtractorMoreComplexBody net472 14.2μs 8.91ns 34.5ns 1.21 0.0213 0 7.62 KB
#3076 BodyExtractorMoreComplexBody netcoreapp3.1 12.3μs 16.2ns 62.6ns 0.0897 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 181μs 164ns 615ns 0.181 0 0 20.33 KB
#3076 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3076 SendRequest netcoreapp3.1 179μs 98.3ns 381ns 0.268 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.54μs 0.423ns 1.58ns 0.126 0.000773 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.32μs 0.395ns 1.53ns 0.0113 0 0 824 B
#3076 ExecuteNonQuery net472 1.54μs 0.43ns 1.66ns 0.126 0.000762 0 794 B
#3076 ExecuteNonQuery netcoreapp3.1 1.33μs 0.404ns 1.51ns 0.011 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.11μs 0.614ns 2.38ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.4μs 2.43ns 9.4ns 0.0132 0 0 984 B
master CallElasticsearchAsync net472 2.43μs 1.2ns 4.63ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.41μs 0.52ns 2.01ns 0.0147 0 0 1.1 KB
#3076 CallElasticsearch net472 2.13μs 0.842ns 3.15ns 0.159 0 0 1 KB
#3076 CallElasticsearch netcoreapp3.1 1.39μs 0.507ns 1.96ns 0.0131 0 0 984 B
#3076 CallElasticsearchAsync net472 2.39μs 0.911ns 3.53ns 0.18 0 0 1.14 KB
#3076 CallElasticsearchAsync netcoreapp3.1 1.49μs 0.525ns 1.82ns 0.015 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.37μs 9.46ns 36.6ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.48μs 1.42ns 4.93ns 0.0166 0 0 1.22 KB
#3076 ExecuteAsync net472 2.24μs 3.04ns 11.4ns 0.199 0 0 1.26 KB
#3076 ExecuteAsync netcoreapp3.1 1.53μs 2.42ns 9.05ns 0.0161 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.85μs 14.5ns 56.2ns 0.394 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.18μs 6.53ns 25.3ns 0.032 0 0 2.36 KB
#3076 SendAsync net472 4.77μs 6.55ns 24.5ns 0.393 0 0 2.48 KB
#3076 SendAsync netcoreapp3.1 3.23μs 6.92ns 25.9ns 0.0318 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.79μs 1.05ns 3.94ns 0.264 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.3μs 0.877ns 3.4ns 0.023 0 0 1.73 KB
#3076 EnrichedLog net472 2.9μs 2.28ns 8.82ns 0.263 0 0 1.66 KB
#3076 EnrichedLog netcoreapp3.1 2.46μs 2.53ns 9.79ns 0.0233 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 150μs 168ns 652ns 0.674 0.225 0 4.5 KB
master EnrichedLog netcoreapp3.1 116μs 155ns 599ns 0 0 0 4.38 KB
#3076 EnrichedLog net472 149μs 99.4ns 385ns 0.666 0.222 0 4.5 KB
#3076 EnrichedLog netcoreapp3.1 114μs 268ns 1.04μs 0.0558 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.28μs 9.7ns 36.3ns 0.545 0.00266 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.18μs 8.2ns 29.6ns 0.0519 0 0 3.8 KB
#3076 EnrichedLog net472 5.2μs 13.6ns 50.7ns 0.543 0.00266 0 3.43 KB
#3076 EnrichedLog netcoreapp3.1 4.11μs 12.5ns 48.3ns 0.0512 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.03μs 0.517ns 2ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.6μs 0.884ns 3.42ns 0.0161 0 0 1.21 KB
#3076 SendReceive net472 1.98μs 0.972ns 3.64ns 0.194 0 0 1.22 KB
#3076 SendReceive netcoreapp3.1 1.65μs 1.13ns 3.92ns 0.0165 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.73μs 1.48ns 5.73ns 0.328 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.08μs 2.71ns 10.5ns 0.0224 0 0 1.69 KB
#3076 EnrichedLog net472 4.81μs 3.01ns 11.3ns 0.329 0 0 2.08 KB
#3076 EnrichedLog netcoreapp3.1 4.26μs 1.79ns 6.71ns 0.0233 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 855ns 0.547ns 2.12ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 764ns 0.713ns 2.76ns 0.00874 0 0 648 B
master StartFinishScope net472 1.08μs 0.865ns 3.35ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 882ns 0.549ns 2.13ns 0.0101 0 0 768 B
#3076 StartFinishSpan net472 843ns 0.807ns 3.12ns 0.104 0 0 658 B
#3076 StartFinishSpan netcoreapp3.1 717ns 0.363ns 1.36ns 0.00894 0 0 648 B
#3076 StartFinishScope net472 1.08μs 0.893ns 3.46ns 0.117 0 0 738 B
#3076 StartFinishScope netcoreapp3.1 889ns 0.666ns 2.58ns 0.0102 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.29μs 0.811ns 3.14ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.02μs 0.626ns 2.42ns 0.0103 0 0 768 B
#3076 RunOnMethodBegin net472 1.19μs 1.33ns 4.78ns 0.117 0 0 738 B
#3076 RunOnMethodBegin netcoreapp3.1 1μs 0.686ns 2.66ns 0.0107 0 0 768 B

andrewlock avatar Aug 17 '22 11:08 andrewlock

Benchmarks Report :snail:

Benchmarks for #3076 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.153
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 730μs 572ns 2.22μs 0.363 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 452μs 173ns 671ns 0 0 0 2.59 KB
#3076 WriteAndFlushEnrichedTraces net472 728μs 553ns 2μs 0.363 0 0 3.18 KB
#3076 WriteAndFlushEnrichedTraces netcoreapp3.1 482μs 154ns 595ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #3076

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net472 1.153 180.64 208.20

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 183ns 0.159ns 0.593ns 0.0675 9.2E-05 0 425 B
master AllCycleSimpleBody netcoreapp3.1 235ns 0.203ns 0.784ns 0.00594 0 0 424 B
master AllCycleMoreComplexBody net472 181ns 0.467ns 1.81ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 236ns 0.334ns 1.29ns 0.00545 0 0 400 B
master BodyExtractorSimpleBody net472 267ns 0.277ns 1.07ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 228ns 0.402ns 1.5ns 0.00379 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.3μs 6.35ns 24.6ns 1.2 0.0214 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 13.2μs 14.2ns 49ns 0.0915 0 0 6.75 KB
#3076 AllCycleSimpleBody net472 182ns 0.187ns 0.724ns 0.0675 0 0 425 B
#3076 AllCycleSimpleBody netcoreapp3.1 238ns 0.212ns 0.821ns 0.00586 0 0 424 B
#3076 AllCycleMoreComplexBody net472 208ns 0.223ns 0.835ns 0.0637 0 0 401 B
#3076 AllCycleMoreComplexBody netcoreapp3.1 236ns 0.147ns 0.548ns 0.00542 0 0 400 B
#3076 BodyExtractorSimpleBody net472 260ns 0.267ns 1ns 0.0573 0 0 361 B
#3076 BodyExtractorSimpleBody netcoreapp3.1 231ns 0.32ns 1.24ns 0.00373 0 0 272 B
#3076 BodyExtractorMoreComplexBody net472 14.3μs 10.1ns 39ns 1.2 0.0215 0 7.62 KB
#3076 BodyExtractorMoreComplexBody netcoreapp3.1 12μs 15.9ns 61.7ns 0.0897 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 180μs 133ns 516ns 0.267 0 0 20.33 KB
#3076 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3076 SendRequest netcoreapp3.1 179μs 120ns 467ns 0.267 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.54μs 0.503ns 1.88ns 0.126 0.000774 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.3μs 0.447ns 1.67ns 0.0111 0 0 824 B
#3076 ExecuteNonQuery net472 1.64μs 0.233ns 0.839ns 0.126 0.000814 0 794 B
#3076 ExecuteNonQuery netcoreapp3.1 1.29μs 0.491ns 1.9ns 0.0109 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.19μs 0.58ns 2.09ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.41μs 0.599ns 2.32ns 0.0133 0 0 984 B
master CallElasticsearchAsync net472 2.38μs 0.675ns 2.52ns 0.181 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.42μs 0.411ns 1.42ns 0.0149 0 0 1.1 KB
#3076 CallElasticsearch net472 2.25μs 0.806ns 3.12ns 0.159 0 0 1 KB
#3076 CallElasticsearch netcoreapp3.1 1.35μs 0.712ns 2.67ns 0.0129 0 0 984 B
#3076 CallElasticsearchAsync net472 2.23μs 0.953ns 3.69ns 0.181 0 0 1.14 KB
#3076 CallElasticsearchAsync netcoreapp3.1 1.44μs 0.354ns 1.28ns 0.015 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.49μs 5.22ns 20.2ns 0.2 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.54μs 1.51ns 5.83ns 0.0162 0 0 1.22 KB
#3076 ExecuteAsync net472 2.36μs 6.63ns 23.9ns 0.2 0 0 1.26 KB
#3076 ExecuteAsync netcoreapp3.1 1.55μs 2.6ns 10.1ns 0.0171 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5μs 5.43ns 21ns 0.392 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.19μs 2.43ns 9.42ns 0.0321 0 0 2.36 KB
#3076 SendAsync net472 4.98μs 6.27ns 23.4ns 0.393 0 0 2.48 KB
#3076 SendAsync netcoreapp3.1 3.24μs 8.29ns 32.1ns 0.0307 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.83μs 2.33ns 8.71ns 0.264 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.31μs 2.39ns 8.96ns 0.0239 0 0 1.73 KB
#3076 EnrichedLog net472 2.95μs 2.49ns 9.66ns 0.263 0 0 1.66 KB
#3076 EnrichedLog netcoreapp3.1 2.31μs 1.29ns 4.84ns 0.0232 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 151μs 118ns 459ns 0.686 0.229 0 4.5 KB
master EnrichedLog netcoreapp3.1 115μs 88.3ns 342ns 0 0 0 4.38 KB
#3076 EnrichedLog net472 151μs 183ns 710ns 0.674 0.225 0 4.5 KB
#3076 EnrichedLog netcoreapp3.1 114μs 212ns 822ns 0.0567 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.36μs 8.85ns 34.3ns 0.545 0.00271 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.39μs 1.69ns 6.08ns 0.0525 0 0 3.8 KB
#3076 EnrichedLog net472 5.46μs 9.84ns 38.1ns 0.544 0.00274 0 3.43 KB
#3076 EnrichedLog netcoreapp3.1 4.38μs 6.93ns 26.8ns 0.0501 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.09μs 2.08ns 8.05ns 0.193 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.68μs 0.802ns 3.11ns 0.0162 0 0 1.21 KB
#3076 SendReceive net472 2.12μs 1.74ns 6.29ns 0.194 0 0 1.22 KB
#3076 SendReceive netcoreapp3.1 1.69μs 1.09ns 4.24ns 0.0161 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.66μs 1.53ns 5.53ns 0.328 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.03μs 3.08ns 11.9ns 0.0223 0 0 1.69 KB
#3076 EnrichedLog net472 4.76μs 1.8ns 6.98ns 0.328 0 0 2.08 KB
#3076 EnrichedLog netcoreapp3.1 4.13μs 1.76ns 6.82ns 0.0227 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 848ns 0.18ns 0.698ns 0.104 0 0 658 B
master StartFinishSpan netcoreapp3.1 776ns 0.209ns 0.753ns 0.00895 0 0 648 B
master StartFinishScope net472 1.05μs 0.192ns 0.72ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 898ns 0.378ns 1.41ns 0.0103 0 0 768 B
#3076 StartFinishSpan net472 806ns 0.348ns 1.35ns 0.104 0 0 658 B
#3076 StartFinishSpan netcoreapp3.1 726ns 0.49ns 1.83ns 0.0087 0 0 648 B
#3076 StartFinishScope net472 1.14μs 1.07ns 4ns 0.117 0 0 738 B
#3076 StartFinishScope netcoreapp3.1 828ns 0.41ns 1.54ns 0.0104 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.19μs 0.6ns 2.32ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.06μs 0.451ns 1.75ns 0.0105 0 0 768 B
#3076 RunOnMethodBegin net472 1.31μs 0.566ns 2.12ns 0.117 0 0 738 B
#3076 RunOnMethodBegin netcoreapp3.1 988ns 0.406ns 1.57ns 0.0105 0 0 768 B

andrewlock avatar Aug 17 '22 16:08 andrewlock