dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard
verified publisher icon DataDog

Small tidy up of the build for serverless tests

Open andrewlock opened this issue 3 years ago • 4 comments

The new serverless docker files were adding benign errors to the build logs, and were doing additional downloads of the tracer home directory. This tidies up the build slightly, without touching any functionality.

Includes an auto-fix for incorrect sln file configurations

@DataDog/apm-dotnet

andrewlock avatar Jan 28 '22 16:01 andrewlock

Benchmarks Report :snail:

Benchmarks for #2365 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.173
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 860μs 4.02μs 15.6μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 672μs 2.98μs 11.5μs 0 0 0 2.57 KB
#2365 WriteAndFlushEnrichedTraces net472 884μs 4.14μs 15.5μs 0 0 0 3.16 KB
#2365 WriteAndFlushEnrichedTraces netcoreapp3.1 668μs 3.24μs 13.7μs 0 0 0 2.57 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 302μs 1.51μs 6.41μs 0.148 0 0 19.67 KB
#2365 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2365 SendRequest netcoreapp3.1 294μs 1.2μs 4.51μs 0.148 0 0 19.67 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.58μs 6.21ns 24ns 0.0928 0.00078 0 586 B
master ExecuteNonQuery netcoreapp3.1 1.48μs 7.39ns 33.9ns 0.00874 0 0 624 B
#2365 ExecuteNonQuery net472 1.61μs 8.07ns 37ns 0.0923 0.000831 0 586 B
#2365 ExecuteNonQuery netcoreapp3.1 1.45μs 3.9ns 14.1ns 0.00865 0 0 624 B
Benchmarks.Trace.ElasticsearchBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #2365

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net472 1.173 2,528.85 2,967.17

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.38μs 13.5ns 90.8ns 0.124 0 0 794 B
master CallElasticsearch netcoreapp3.1 1.69μs 7.59ns 29.4ns 0.011 0 0 784 B
master CallElasticsearchAsync net472 2.53μs 11.2ns 42ns 0.146 0 0 931 B
master CallElasticsearchAsync netcoreapp3.1 1.71μs 8.22ns 45ns 0.0122 0 0 904 B
#2365 CallElasticsearch net472 2.3μs 9.82ns 36.7ns 0.124 0 0 794 B
#2365 CallElasticsearch netcoreapp3.1 1.74μs 9.52ns 68ns 0.0112 0 0 784 B
#2365 CallElasticsearchAsync net472 2.97μs 11.2ns 43.3ns 0.146 0 0 931 B
#2365 CallElasticsearchAsync netcoreapp3.1 1.82μs 8.9ns 37.8ns 0.0117 0 0 904 B
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.69μs 12.7ns 50.8ns 0.164 0 0 1.05 KB
master ExecuteAsync netcoreapp3.1 1.98μs 9.46ns 37.8ns 0.0136 0 0 1.02 KB
#2365 ExecuteAsync net472 2.68μs 8.75ns 32.7ns 0.164 0 0 1.05 KB
#2365 ExecuteAsync netcoreapp3.1 1.9μs 5.62ns 21.8ns 0.0141 0 0 1.02 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 6.04μs 28.8ns 112ns 0.352 0 0 2.26 KB
master SendAsync netcoreapp3.1 4.36μs 12.4ns 44.7ns 0.0303 0 0 2.16 KB
#2365 SendAsync net472 6.11μs 27.8ns 104ns 0.353 0 0 2.26 KB
#2365 SendAsync netcoreapp3.1 4.47μs 15.4ns 59.5ns 0.0307 0 0 2.16 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.4μs 16.5ns 68ns 0.225 0 0 1.44 KB
master EnrichedLog netcoreapp3.1 3.14μs 12.4ns 60.9ns 0.0202 0 0 1.52 KB
#2365 EnrichedLog net472 3.42μs 12.4ns 47.9ns 0.224 0 0 1.44 KB
#2365 EnrichedLog netcoreapp3.1 3.09μs 11.6ns 43.2ns 0.0201 0 0 1.52 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 294μs 1.24μs 4.78μs 0.448 0.149 0 4.31 KB
master EnrichedLog netcoreapp3.1 235μs 998ns 3.74μs 0 0 0 4.19 KB
#2365 EnrichedLog net472 290μs 605ns 2.18μs 0.444 0.148 0 4.31 KB
#2365 EnrichedLog netcoreapp3.1 237μs 1.08μs 4.18μs 0 0 0 4.19 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 7.62μs 29.5ns 110ns 0.503 0 0 3.21 KB
master EnrichedLog netcoreapp3.1 6.61μs 37.5ns 286ns 0.0478 0 0 3.58 KB
#2365 EnrichedLog net472 7.72μs 37.3ns 154ns 0.503 0 0 3.21 KB
#2365 EnrichedLog netcoreapp3.1 6.36μs 29.1ns 109ns 0.0505 0 0 3.58 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.2μs 11.8ns 66.8ns 0.157 0 0 1 KB
master SendReceive netcoreapp3.1 1.97μs 6.89ns 26.7ns 0.0139 0 0 1 KB
#2365 SendReceive net472 2.28μs 10.5ns 40.6ns 0.157 0 0 1 KB
#2365 SendReceive netcoreapp3.1 2.04μs 11ns 63.4ns 0.0138 0 0 1 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.86μs 27.3ns 106ns 0.288 0 0 1.85 KB
master EnrichedLog netcoreapp3.1 5.06μs 13.1ns 49.1ns 0.0199 0 0 1.47 KB
#2365 EnrichedLog net472 6.2μs 32.1ns 161ns 0.288 0 0 1.85 KB
#2365 EnrichedLog netcoreapp3.1 5.25μs 27.3ns 128ns 0.0211 0 0 1.47 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 928ns 3.39ns 13.1ns 0.0703 0 0 449 B
master StartFinishSpan netcoreapp3.1 886ns 3.04ns 11.8ns 0.00617 0 0 448 B
master StartFinishScope net472 1.1μs 3.53ns 13.2ns 0.0831 0 0 530 B
master StartFinishScope netcoreapp3.1 1.05μs 5.46ns 36.6ns 0.00766 0 0 568 B
#2365 StartFinishSpan net472 916ns 3.29ns 12.7ns 0.0705 0 0 449 B
#2365 StartFinishSpan netcoreapp3.1 874ns 2.5ns 9ns 0.0059 0 0 448 B
#2365 StartFinishScope net472 1.12μs 5.54ns 24.8ns 0.083 0 0 530 B
#2365 StartFinishScope netcoreapp3.1 1.08μs 4.91ns 19ns 0.00762 0 0 568 B

andrewlock avatar Jan 28 '22 19:01 andrewlock

Benchmarks Report :snail:

Benchmarks for #2365 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 885μs 3.54μs 12.3μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 667μs 3.59μs 19.4μs 0 0 0 2.57 KB
#2365 WriteAndFlushEnrichedTraces net472 875μs 4.1μs 15.4μs 0 0 0 3.16 KB
#2365 WriteAndFlushEnrichedTraces netcoreapp3.1 695μs 3.84μs 24.3μs 0 0 0 2.57 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 293μs 1.29μs 5.16μs 0.146 0 0 19.67 KB
#2365 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2365 SendRequest netcoreapp3.1 294μs 1.43μs 5.74μs 0.23 0 0 19.67 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.58μs 7.62ns 29.5ns 0.0928 0.000786 0 586 B
master ExecuteNonQuery netcoreapp3.1 1.42μs 2.49ns 8.61ns 0.00845 0 0 624 B
#2365 ExecuteNonQuery net472 1.55μs 6.74ns 26.1ns 0.0929 0.00078 0 586 B
#2365 ExecuteNonQuery netcoreapp3.1 1.46μs 6.35ns 23.7ns 0.00815 0 0 624 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.24μs 8.77ns 32.8ns 0.124 0 0 794 B
master CallElasticsearch netcoreapp3.1 1.66μs 8.56ns 38.3ns 0.0108 0 0 784 B
master CallElasticsearchAsync net472 2.45μs 12.3ns 55.1ns 0.146 0 0 931 B
master CallElasticsearchAsync netcoreapp3.1 1.7μs 6.89ns 24.8ns 0.0129 0 0 904 B
#2365 CallElasticsearch net472 2.4μs 10.9ns 40.9ns 0.124 0 0 794 B
#2365 CallElasticsearch netcoreapp3.1 1.67μs 7.81ns 30.2ns 0.0107 0 0 784 B
#2365 CallElasticsearchAsync net472 2.4μs 9.35ns 36.2ns 0.145 0 0 931 B
#2365 CallElasticsearchAsync netcoreapp3.1 1.84μs 8.79ns 34ns 0.0128 0 0 904 B
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.78μs 19.4ns 187ns 0.164 0 0 1.05 KB
master ExecuteAsync netcoreapp3.1 1.98μs 11.1ns 75.5ns 0.0136 0 0 1.02 KB
#2365 ExecuteAsync net472 2.7μs 11.7ns 43.8ns 0.164 0 0 1.05 KB
#2365 ExecuteAsync netcoreapp3.1 1.94μs 5.81ns 21.7ns 0.0145 0 0 1.02 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 6.25μs 33.6ns 174ns 0.35 0 0 2.26 KB
master SendAsync netcoreapp3.1 4.52μs 22.5ns 103ns 0.0293 0 0 2.16 KB
#2365 SendAsync net472 6.04μs 18.3ns 81.8ns 0.354 0 0 2.26 KB
#2365 SendAsync netcoreapp3.1 4.74μs 25.4ns 130ns 0.0299 0 0 2.16 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.34μs 15.7ns 62.6ns 0.225 0 0 1.44 KB
master EnrichedLog netcoreapp3.1 3.15μs 8.28ns 32.1ns 0.0209 0 0 1.52 KB
#2365 EnrichedLog net472 3.46μs 17.9ns 83.9ns 0.225 0 0 1.44 KB
#2365 EnrichedLog netcoreapp3.1 3.11μs 12.7ns 47.5ns 0.0214 0 0 1.52 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 292μs 912ns 3.53μs 0.443 0.148 0 4.31 KB
master EnrichedLog netcoreapp3.1 235μs 995ns 3.85μs 0 0 0 4.19 KB
#2365 EnrichedLog net472 295μs 1.27μs 4.75μs 0.434 0.145 0 4.31 KB
#2365 EnrichedLog netcoreapp3.1 237μs 813ns 3.15μs 0 0 0 4.19 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 7.75μs 36.6ns 146ns 0.501 0 0 3.21 KB
master EnrichedLog netcoreapp3.1 6.5μs 28.2ns 105ns 0.0512 0 0 3.58 KB
#2365 EnrichedLog net472 7.79μs 36.4ns 141ns 0.503 0 0 3.21 KB
#2365 EnrichedLog netcoreapp3.1 6.28μs 27.2ns 105ns 0.0509 0 0 3.58 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.39μs 9.58ns 40.6ns 0.157 0 0 1 KB
master SendReceive netcoreapp3.1 1.98μs 7.87ns 29.5ns 0.0139 0 0 1 KB
#2365 SendReceive net472 2.26μs 11.2ns 51.5ns 0.157 0 0 1 KB
#2365 SendReceive netcoreapp3.1 1.98μs 5.39ns 20.2ns 0.0139 0 0 1 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.89μs 31.8ns 191ns 0.29 0 0 1.85 KB
master EnrichedLog netcoreapp3.1 5.25μs 22.5ns 87ns 0.0187 0 0 1.47 KB
#2365 EnrichedLog net472 5.98μs 30.4ns 136ns 0.289 0 0 1.85 KB
#2365 EnrichedLog netcoreapp3.1 5.41μs 15.7ns 60.8ns 0.019 0 0 1.47 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 952ns 5.26ns 40.8ns 0.0705 0 0 449 B
master StartFinishSpan netcoreapp3.1 898ns 2.89ns 10.8ns 0.00621 0 0 448 B
master StartFinishScope net472 1.11μs 5.75ns 26.4ns 0.0829 0 0 530 B
master StartFinishScope netcoreapp3.1 1.06μs 3.3ns 12.8ns 0.00784 0 0 568 B
#2365 StartFinishSpan net472 930ns 4.5ns 17.4ns 0.0705 0 0 449 B
#2365 StartFinishSpan netcoreapp3.1 986ns 4.56ns 17.1ns 0.00598 0 0 448 B
#2365 StartFinishScope net472 1.09μs 4.62ns 17.9ns 0.0828 0 0 530 B
#2365 StartFinishScope netcoreapp3.1 1.04μs 5.68ns 32.1ns 0.00789 0 0 568 B

andrewlock avatar Jan 31 '22 11:01 andrewlock

Benchmarks Report :snail:

Benchmarks for #2365 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.173
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 956μs 5.54μs 45.7μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 682μs 2.82μs 10.5μs 0 0 0 2.57 KB
#2365 WriteAndFlushEnrichedTraces net472 912μs 5.26μs 41.5μs 0 0 0 3.16 KB
#2365 WriteAndFlushEnrichedTraces netcoreapp3.1 717μs 4.13μs 32.5μs 0 0 0 2.57 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0.00235ns 0.00235ns 0.0159ns 0 0 0 0 b
master SendRequest netcoreapp3.1 336μs 1.21μs 4.51μs 0.164 0 0 19.67 KB
#2365 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2365 SendRequest netcoreapp3.1 337μs 828ns 3.21μs 0.167 0 0 19.67 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.74μs 8.78ns 42.1ns 0.0927 0.000874 0 586 B
master ExecuteNonQuery netcoreapp3.1 1.45μs 5.17ns 17.9ns 0.00873 0 0 624 B
#2365 ExecuteNonQuery net472 1.73μs 4.03ns 14.5ns 0.0925 0.000857 0 586 B
#2365 ExecuteNonQuery netcoreapp3.1 1.47μs 6.31ns 25.2ns 0.00865 0 0 624 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.45μs 10.3ns 37.1ns 0.124 0 0 794 B
master CallElasticsearch netcoreapp3.1 1.81μs 10ns 71.7ns 0.0109 0 0 784 B
master CallElasticsearchAsync net472 2.68μs 15.5ns 119ns 0.146 0 0 931 B
master CallElasticsearchAsync netcoreapp3.1 1.77μs 8.28ns 33.1ns 0.0123 0 0 904 B
#2365 CallElasticsearch net472 2.46μs 12ns 51ns 0.124 0 0 794 B
#2365 CallElasticsearch netcoreapp3.1 1.73μs 8.6ns 36.5ns 0.0111 0 0 784 B
#2365 CallElasticsearchAsync net472 2.63μs 12.2ns 45.8ns 0.146 0 0 931 B
#2365 CallElasticsearchAsync netcoreapp3.1 1.78μs 7.13ns 26.7ns 0.0127 0 0 904 B
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.82μs 10.9ns 42.4ns 0.165 0 0 1.05 KB
master ExecuteAsync netcoreapp3.1 2.02μs 8.39ns 33.6ns 0.0138 0 0 1.02 KB
#2365 ExecuteAsync net472 2.85μs 14.3ns 63.8ns 0.164 0 0 1.05 KB
#2365 ExecuteAsync netcoreapp3.1 1.93μs 8.99ns 34.8ns 0.0143 0 0 1.02 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 6.37μs 17ns 63.7ns 0.351 0 0 2.26 KB
master SendAsync netcoreapp3.1 4.45μs 20.9ns 81.1ns 0.0296 0 0 2.16 KB
#2365 SendAsync net472 6.67μs 38.5ns 336ns 0.351 0 0 2.26 KB
#2365 SendAsync netcoreapp3.1 4.59μs 25.5ns 175ns 0.0306 0 0 2.16 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.49μs 16.4ns 67.6ns 0.224 0 0 1.44 KB
master EnrichedLog netcoreapp3.1 3.13μs 10.9ns 40.7ns 0.0211 0 0 1.52 KB
#2365 EnrichedLog net472 3.65μs 20.7ns 137ns 0.225 0 0 1.44 KB
#2365 EnrichedLog netcoreapp3.1 3.19μs 15.2ns 62.8ns 0.0208 0 0 1.52 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 302μs 1.22μs 4.73μs 0.447 0.149 0 4.31 KB
master EnrichedLog netcoreapp3.1 242μs 826ns 3.09μs 0 0 0 4.19 KB
#2365 EnrichedLog net472 302μs 1.4μs 5.43μs 0.305 0.153 0 4.31 KB
#2365 EnrichedLog netcoreapp3.1 243μs 634ns 2.46μs 0 0 0 4.19 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 7.8μs 40.8ns 258ns 0.501 0 0 3.21 KB
master EnrichedLog netcoreapp3.1 6.97μs 46.5ns 461ns 0.0487 0 0 3.58 KB
#2365 EnrichedLog net472 7.95μs 18.6ns 69.5ns 0.503 0 0 3.21 KB
#2365 EnrichedLog netcoreapp3.1 6.72μs 21.6ns 80.8ns 0.0496 0 0 3.58 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.43μs 7.31ns 26.3ns 0.158 0 0 1 KB
master SendReceive netcoreapp3.1 2μs 6.61ns 24.7ns 0.0136 0 0 1 KB
#2365 SendReceive net472 2.46μs 13.4ns 77.1ns 0.157 0 0 1 KB
#2365 SendReceive netcoreapp3.1 2.01μs 6.6ns 25.6ns 0.0136 0 0 1 KB
Benchmarks.Trace.SerilogBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #2365

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑net472 1.173 6,067.01 7,114.87 bimodal

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 6.07μs 22ns 79.5ns 0.289 0 0 1.85 KB
master EnrichedLog netcoreapp3.1 5.18μs 17ns 65.8ns 0.021 0 0 1.47 KB
#2365 EnrichedLog net472 7.14μs 41.3ns 370ns 0.289 0 0 1.85 KB
#2365 EnrichedLog netcoreapp3.1 5.33μs 16.3ns 61.1ns 0.0209 0 0 1.47 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 944ns 4.48ns 17.9ns 0.0705 0 0 449 B
master StartFinishSpan netcoreapp3.1 927ns 2.06ns 7.69ns 0.00604 0 0 448 B
master StartFinishScope net472 1.14μs 3.86ns 15ns 0.083 0 0 530 B
master StartFinishScope netcoreapp3.1 1.06μs 2.85ns 10.7ns 0.00788 0 0 568 B
#2365 StartFinishSpan net472 948ns 4.6ns 19ns 0.0707 0 0 449 B
#2365 StartFinishSpan netcoreapp3.1 897ns 2.73ns 10.2ns 0.00596 0 0 448 B
#2365 StartFinishScope net472 1.13μs 4.22ns 16.4ns 0.0832 0 0 530 B
#2365 StartFinishScope netcoreapp3.1 1.08μs 5.6ns 26.3ns 0.00782 0 0 568 B

andrewlock avatar Feb 08 '22 18:02 andrewlock

Benchmarks Report :snail:

Benchmarks for #2365 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 948μs 5.46μs 41.9μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 747μs 4.2μs 30μs 0 0 0 2.57 KB
#2365 WriteAndFlushEnrichedTraces net472 912μs 3.07μs 11.5μs 0 0 0 3.16 KB
#2365 WriteAndFlushEnrichedTraces netcoreapp3.1 694μs 1.77μs 6.38μs 0 0 0 2.57 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 326μs 1.49μs 5.96μs 0.161 0 0 19.67 KB
#2365 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2365 SendRequest netcoreapp3.1 337μs 1.53μs 5.93μs 0.168 0 0 19.67 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.63μs 3.75ns 13.5ns 0.0929 0.000815 0 586 B
master ExecuteNonQuery netcoreapp3.1 1.49μs 7.62ns 36.6ns 0.00865 0 0 624 B
#2365 ExecuteNonQuery net472 1.79μs 10.4ns 82.9ns 0.093 0.000894 0 586 B
#2365 ExecuteNonQuery netcoreapp3.1 1.53μs 5.77ns 22.3ns 0.009 0 0 624 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.41μs 10.9ns 46.2ns 0.124 0 0 794 B
master CallElasticsearch netcoreapp3.1 1.62μs 6.68ns 25.9ns 0.0112 0 0 784 B
master CallElasticsearchAsync net472 2.63μs 11.9ns 46.3ns 0.145 0 0 931 B
master CallElasticsearchAsync netcoreapp3.1 1.76μs 8.62ns 37.6ns 0.0122 0 0 904 B
#2365 CallElasticsearch net472 2.64μs 15.3ns 131ns 0.124 0 0 794 B
#2365 CallElasticsearch netcoreapp3.1 1.74μs 4.7ns 17.6ns 0.0106 0 0 784 B
#2365 CallElasticsearchAsync net472 2.8μs 15.7ns 128ns 0.145 0 0 931 B
#2365 CallElasticsearchAsync netcoreapp3.1 1.75μs 8.07ns 31.3ns 0.0128 0 0 904 B
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.85μs 13.5ns 57.1ns 0.165 0 0 1.05 KB
master ExecuteAsync netcoreapp3.1 1.92μs 7.15ns 27.7ns 0.0144 0 0 1.02 KB
#2365 ExecuteAsync net472 2.87μs 8.64ns 31.2ns 0.164 0 0 1.05 KB
#2365 ExecuteAsync netcoreapp3.1 1.97μs 4.96ns 19.2ns 0.014 0 0 1.02 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 6.23μs 15ns 55.9ns 0.352 0 0 2.26 KB
master SendAsync netcoreapp3.1 4.78μs 26.8ns 190ns 0.0294 0 0 2.16 KB
#2365 SendAsync net472 6.32μs 30.1ns 117ns 0.354 0 0 2.26 KB
#2365 SendAsync netcoreapp3.1 4.44μs 16.1ns 57.9ns 0.0298 0 0 2.16 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.4μs 18.6ns 125ns 0.225 0 0 1.44 KB
master EnrichedLog netcoreapp3.1 3.16μs 14.2ns 53.2ns 0.0216 0 0 1.52 KB
#2365 EnrichedLog net472 3.48μs 16.1ns 66.4ns 0.224 0 0 1.44 KB
#2365 EnrichedLog netcoreapp3.1 3.23μs 10.9ns 40.7ns 0.0213 0 0 1.52 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 293μs 800ns 2.99μs 0.44 0.147 0 4.31 KB
master EnrichedLog netcoreapp3.1 234μs 767ns 2.87μs 0 0 0 4.19 KB
#2365 EnrichedLog net472 295μs 808ns 3.13μs 0.445 0.148 0 4.31 KB
#2365 EnrichedLog netcoreapp3.1 246μs 1.26μs 9.11μs 0 0 0 4.19 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 7.77μs 35.1ns 161ns 0.502 0 0 3.21 KB
master EnrichedLog netcoreapp3.1 6.41μs 32.6ns 142ns 0.0504 0 0 3.58 KB
#2365 EnrichedLog net472 8.04μs 46.6ns 417ns 0.504 0 0 3.21 KB
#2365 EnrichedLog netcoreapp3.1 6.48μs 23.8ns 89ns 0.0506 0 0 3.58 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.28μs 11.1ns 45.9ns 0.157 0 0 1 KB
master SendReceive netcoreapp3.1 2.01μs 5.89ns 21.2ns 0.0137 0 0 1 KB
#2365 SendReceive net472 2.3μs 11ns 43.8ns 0.157 0 0 1 KB
#2365 SendReceive netcoreapp3.1 2.07μs 7.86ns 30.4ns 0.0135 0 0 1 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 6.1μs 26.7ns 103ns 0.287 0 0 1.85 KB
master EnrichedLog netcoreapp3.1 5.41μs 13.9ns 52.2ns 0.0185 0 0 1.47 KB
#2365 EnrichedLog net472 6.22μs 27.7ns 104ns 0.287 0 0 1.85 KB
#2365 EnrichedLog netcoreapp3.1 5.26μs 8.77ns 34ns 0.0206 0 0 1.47 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 926ns 3.93ns 14.2ns 0.0707 0 0 449 B
master StartFinishSpan netcoreapp3.1 921ns 5.23ns 36.6ns 0.00628 0 0 448 B
master StartFinishScope net472 1.18μs 6.68ns 44.8ns 0.0833 0 0 530 B
master StartFinishScope netcoreapp3.1 1.05μs 3.95ns 15.3ns 0.00774 0 0 568 B
#2365 StartFinishSpan net472 929ns 3.63ns 13.6ns 0.0707 0 0 449 B
#2365 StartFinishSpan netcoreapp3.1 922ns 1.68ns 6.29ns 0.00631 0 0 448 B
#2365 StartFinishScope net472 1.12μs 3.45ns 13.4ns 0.0832 0 0 530 B
#2365 StartFinishScope netcoreapp3.1 1.08μs 3.63ns 14.1ns 0.00813 0 0 568 B

andrewlock avatar Feb 09 '22 19:02 andrewlock

This has been reworked elsewhere

andrewlock avatar Aug 22 '22 12:08 andrewlock