Summary of changes
- Run system tests with a pinned commit SHA
- Automation for upgrade system tests version and open a pull request.
Reason for change
Inconsistent version using HEAD of main from system tests.
Implementation details
In order for the automated upgrade to work:
- Install Github App
- Add Secrets
- Allow certain Github Actions.
Execution-Time Benchmarks Report :stopwatch:
Execution-time results for samples comparing the following branches/commits:
Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
- Welch test with statistical test for significance of 5%
- Only results indicating a difference greater than 5% and 5 ms are considered.
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.
Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).
gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (7067) - mean (77ms) : 71, 84
. : milestone, 77,
master - mean (76ms) : 70, 83
. : milestone, 76,
section CallTarget+Inlining+NGEN
This PR (7067) - mean (1,082ms) : 1038, 1126
. : milestone, 1082,
master - mean (1,085ms) : 1039, 1131
. : milestone, 1085,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (7067) - mean (114ms) : 109, 119
. : milestone, 114,
master - mean (114ms) : 106, 121
. : milestone, 114,
section CallTarget+Inlining+NGEN
This PR (7067) - mean (763ms) : 743, 782
. : milestone, 763,
master - mean (755ms) : 724, 786
. : milestone, 755,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (7067) - mean (101ms) : 92, 109
. : milestone, 101,
master - mean (100ms) : 94, 106
. : milestone, 100,
section CallTarget+Inlining+NGEN
This PR (7067) - mean (718ms) : 690, 746
. : milestone, 718,
master - mean (728ms) : 701, 755
. : milestone, 728,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (7067) - mean (190ms) : 186, 194
. : milestone, 190,
master - mean (191ms) : 186, 195
. : milestone, 191,
section CallTarget+Inlining+NGEN
This PR (7067) - mean (1,168ms) : 1151, 1185
. : milestone, 1168,
master - mean (1,166ms) : 1145, 1187
. : milestone, 1166,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (7067) - mean (269ms) : 265, 273
. : milestone, 269,
master - mean (269ms) : 265, 272
. : milestone, 269,
section CallTarget+Inlining+NGEN
This PR (7067) - mean (932ms) : 915, 948
. : milestone, 932,
master - mean (932ms) : 919, 944
. : milestone, 932,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (7067) - mean (262ms) : 259, 265
. : milestone, 262,
master - mean (262ms) : 259, 265
. : milestone, 262,
section CallTarget+Inlining+NGEN
This PR (7067) - mean (927ms) : 910, 944
. : milestone, 927,
master - mean (927ms) : 913, 942
. : milestone, 927,
Datadog Report
Branch report: tonycthsu/update-system-tests
Commit report: d7b93c2
Test service: dd-trace-dotnet
:white_check_mark: 0 Failed, 244996 Passed, 2460 Skipped, 15h 25m 53.09s Total Time
:hourglass: 1 Performance Regression
:hourglass: Performance Regressions vs Default Branch (1)
Profiler_allocation_cpu_walltime - scenarios 2.22s (+64.68ms, +3%) - Details
Benchmarks
Benchmarks Report for benchmark platform :snail:
Benchmarks for #7067 compared to master:
- 2 benchmarks are slower, with geometric mean 1.198
- 3 benchmarks have fewer allocations
- 7 benchmarks have more allocations
The following thresholds were used for comparing the benchmark speeds:
- Mann–Whitney U test with statistical test for significance of 5%
- Only results indicating a difference greater than 10% and 0.3 ns are considered.
Allocation changes below 0.5% are ignored.
Benchmark details
Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: More allocations :warning:
More allocations :warning: in #7067
| Benchmark |
Base Allocated |
Diff Allocated |
Change |
Change % |
| Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 |
6.09 KB |
6.14 KB |
52 B |
0.85% |
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
StartStopWithChild |
net6.0 |
11.2μs |
59.7ns |
322ns |
0 |
0 |
0 |
5.58 KB |
| master |
StartStopWithChild |
netcoreapp3.1 |
14.1μs |
67.5ns |
286ns |
0 |
0 |
0 |
5.75 KB |
| master |
StartStopWithChild |
net472 |
22.1μs |
119ns |
651ns |
1.03 |
0.411 |
0.103 |
6.09 KB |
| #7067 |
StartStopWithChild |
net6.0 |
10.5μs |
32.8ns |
127ns |
0 |
0 |
0 |
5.56 KB |
| #7067 |
StartStopWithChild |
netcoreapp3.1 |
13.8μs |
66.9ns |
276ns |
0 |
0 |
0 |
5.75 KB |
| #7067 |
StartStopWithChild |
net472 |
21.3μs |
115ns |
609ns |
0.987 |
0.329 |
0.11 |
6.14 KB |
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
WriteAndFlushEnrichedTraces |
net6.0 |
927μs |
26.6ns |
103ns |
0 |
0 |
0 |
2.71 KB |
| master |
WriteAndFlushEnrichedTraces |
netcoreapp3.1 |
1.04ms |
404ns |
1.56μs |
0 |
0 |
0 |
2.7 KB |
| master |
WriteAndFlushEnrichedTraces |
net472 |
1.25ms |
357ns |
1.38μs |
0 |
0 |
0 |
3.33 KB |
| #7067 |
WriteAndFlushEnrichedTraces |
net6.0 |
923μs |
76.9ns |
288ns |
0 |
0 |
0 |
2.71 KB |
| #7067 |
WriteAndFlushEnrichedTraces |
netcoreapp3.1 |
1.02ms |
38.4ns |
138ns |
0 |
0 |
0 |
2.7 KB |
| #7067 |
WriteAndFlushEnrichedTraces |
net472 |
1.26ms |
444ns |
1.72μs |
0 |
0 |
0 |
3.33 KB |
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
AllCycleSimpleBody |
net6.0 |
331μs |
1.76μs |
8.42μs |
0 |
0 |
0 |
197.06 KB |
| master |
AllCycleSimpleBody |
netcoreapp3.1 |
510μs |
1.45μs |
5.61μs |
0 |
0 |
0 |
204.77 KB |
| master |
AllCycleSimpleBody |
net472 |
436μs |
119ns |
460ns |
36.6 |
2.16 |
0 |
236.35 KB |
| master |
AllCycleMoreComplexBody |
net6.0 |
338μs |
1.76μs |
8.8μs |
0 |
0 |
0 |
200.56 KB |
| master |
AllCycleMoreComplexBody |
netcoreapp3.1 |
495μs |
987ns |
3.56μs |
0 |
0 |
0 |
208.18 KB |
| master |
AllCycleMoreComplexBody |
net472 |
446μs |
106ns |
412ns |
36.6 |
2.16 |
0 |
239.87 KB |
| master |
ObjectExtractorSimpleBody |
net6.0 |
311ns |
1.77ns |
12.3ns |
0 |
0 |
0 |
280 B |
| master |
ObjectExtractorSimpleBody |
netcoreapp3.1 |
409ns |
1.93ns |
8.2ns |
0 |
0 |
0 |
272 B |
| master |
ObjectExtractorSimpleBody |
net472 |
303ns |
0.175ns |
0.676ns |
0.0442 |
0 |
0 |
281 B |
| master |
ObjectExtractorMoreComplexBody |
net6.0 |
6.52μs |
29.5ns |
110ns |
0 |
0 |
0 |
3.78 KB |
| master |
ObjectExtractorMoreComplexBody |
netcoreapp3.1 |
7.76μs |
36.2ns |
140ns |
0 |
0 |
0 |
3.69 KB |
| master |
ObjectExtractorMoreComplexBody |
net472 |
6.66μs |
0.89ns |
3.33ns |
0.599 |
0 |
0 |
3.8 KB |
| #7067 |
AllCycleSimpleBody |
net6.0 |
331μs |
265ns |
1.03μs |
0 |
0 |
0 |
197.41 KB |
| #7067 |
AllCycleSimpleBody |
netcoreapp3.1 |
483μs |
878ns |
3.4μs |
0 |
0 |
0 |
205.17 KB |
| #7067 |
AllCycleSimpleBody |
net472 |
437μs |
126ns |
490ns |
36.6 |
2.16 |
0 |
236.95 KB |
| #7067 |
AllCycleMoreComplexBody |
net6.0 |
342μs |
770ns |
2.88μs |
0 |
0 |
0 |
200.91 KB |
| #7067 |
AllCycleMoreComplexBody |
netcoreapp3.1 |
512μs |
262ns |
909ns |
0 |
0 |
0 |
208.58 KB |
| #7067 |
AllCycleMoreComplexBody |
net472 |
448μs |
132ns |
510ns |
37.9 |
2.23 |
0 |
240.46 KB |
| #7067 |
ObjectExtractorSimpleBody |
net6.0 |
321ns |
1.7ns |
9.17ns |
0 |
0 |
0 |
280 B |
| #7067 |
ObjectExtractorSimpleBody |
netcoreapp3.1 |
393ns |
2.18ns |
13.4ns |
0 |
0 |
0 |
272 B |
| #7067 |
ObjectExtractorSimpleBody |
net472 |
304ns |
0.0644ns |
0.249ns |
0.0444 |
0 |
0 |
281 B |
| #7067 |
ObjectExtractorMoreComplexBody |
net6.0 |
6.33μs |
14.6ns |
56.5ns |
0 |
0 |
0 |
3.78 KB |
| #7067 |
ObjectExtractorMoreComplexBody |
netcoreapp3.1 |
7.85μs |
37.7ns |
151ns |
0 |
0 |
0 |
3.69 KB |
| #7067 |
ObjectExtractorMoreComplexBody |
net472 |
6.84μs |
2.47ns |
9.55ns |
0.581 |
0 |
0 |
3.8 KB |
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:
Fewer allocations :tada: in #7067
| Benchmark |
Base Allocated |
Diff Allocated |
Change |
Change % |
| Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs‑net6.0 |
2.16 KB |
2.14 KB |
-12 B |
-0.56% |
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
EncodeArgs |
net6.0 |
73.3μs |
263ns |
984ns |
0 |
0 |
0 |
32.41 KB |
| master |
EncodeArgs |
netcoreapp3.1 |
95.7μs |
36.2ns |
130ns |
0 |
0 |
0 |
32.4 KB |
| master |
EncodeArgs |
net472 |
107μs |
17.6ns |
65.8ns |
4.82 |
0 |
0 |
32.51 KB |
| master |
EncodeLegacyArgs |
net6.0 |
143μs |
122ns |
472ns |
0 |
0 |
0 |
2.16 KB |
| master |
EncodeLegacyArgs |
netcoreapp3.1 |
197μs |
42.5ns |
147ns |
0 |
0 |
0 |
2.14 KB |
| master |
EncodeLegacyArgs |
net472 |
261μs |
66.8ns |
250ns |
0 |
0 |
0 |
2.16 KB |
| #7067 |
EncodeArgs |
net6.0 |
74.5μs |
160ns |
599ns |
0 |
0 |
0 |
32.41 KB |
| #7067 |
EncodeArgs |
netcoreapp3.1 |
95.1μs |
93.7ns |
363ns |
0 |
0 |
0 |
32.4 KB |
| #7067 |
EncodeArgs |
net472 |
104μs |
15.2ns |
57ns |
4.67 |
0 |
0 |
32.51 KB |
| #7067 |
EncodeLegacyArgs |
net6.0 |
147μs |
12ns |
46.4ns |
0 |
0 |
0 |
2.14 KB |
| #7067 |
EncodeLegacyArgs |
netcoreapp3.1 |
197μs |
189ns |
732ns |
0 |
0 |
0 |
2.14 KB |
| #7067 |
EncodeLegacyArgs |
net472 |
264μs |
38.8ns |
140ns |
0 |
0 |
0 |
2.16 KB |
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
RunWafRealisticBenchmark |
net6.0 |
271μs |
133ns |
499ns |
0 |
0 |
0 |
4.55 KB |
| master |
RunWafRealisticBenchmark |
netcoreapp3.1 |
294μs |
264ns |
989ns |
0 |
0 |
0 |
4.48 KB |
| master |
RunWafRealisticBenchmark |
net472 |
307μs |
38.1ns |
147ns |
0 |
0 |
0 |
4.66 KB |
| master |
RunWafRealisticBenchmarkWithAttack |
net6.0 |
181μs |
80.1ns |
300ns |
0 |
0 |
0 |
2.24 KB |
| master |
RunWafRealisticBenchmarkWithAttack |
netcoreapp3.1 |
198μs |
104ns |
404ns |
0 |
0 |
0 |
2.22 KB |
| master |
RunWafRealisticBenchmarkWithAttack |
net472 |
207μs |
45.3ns |
169ns |
0 |
0 |
0 |
2.28 KB |
| #7067 |
RunWafRealisticBenchmark |
net6.0 |
270μs |
69ns |
267ns |
0 |
0 |
0 |
4.55 KB |
| #7067 |
RunWafRealisticBenchmark |
netcoreapp3.1 |
291μs |
414ns |
1.6μs |
0 |
0 |
0 |
4.48 KB |
| #7067 |
RunWafRealisticBenchmark |
net472 |
306μs |
25.5ns |
98.8ns |
0 |
0 |
0 |
4.66 KB |
| #7067 |
RunWafRealisticBenchmarkWithAttack |
net6.0 |
182μs |
93.8ns |
351ns |
0 |
0 |
0 |
2.24 KB |
| #7067 |
RunWafRealisticBenchmarkWithAttack |
netcoreapp3.1 |
194μs |
66.5ns |
249ns |
0 |
0 |
0 |
2.22 KB |
| #7067 |
RunWafRealisticBenchmarkWithAttack |
net472 |
206μs |
56ns |
217ns |
0 |
0 |
0 |
2.28 KB |
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
SendRequest |
net6.0 |
61.1μs |
29ns |
104ns |
0 |
0 |
0 |
14.53 KB |
| master |
SendRequest |
netcoreapp3.1 |
69.9μs |
118ns |
440ns |
0 |
0 |
0 |
17.42 KB |
| master |
SendRequest |
net472 |
0.0171ns |
0.00147ns |
0.00551ns |
0 |
0 |
0 |
0 b |
| #7067 |
SendRequest |
net6.0 |
61.6μs |
68.2ns |
264ns |
0 |
0 |
0 |
14.53 KB |
| #7067 |
SendRequest |
netcoreapp3.1 |
72μs |
367ns |
1.76μs |
0 |
0 |
0 |
17.42 KB |
| #7067 |
SendRequest |
net472 |
0.0238ns |
0.00178ns |
0.00691ns |
0 |
0 |
0 |
0 b |
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Slower :warning: More allocations :warning:
Slower :warning: in #7067
| Benchmark |
diff/base |
Base Median (ns) |
Diff Median (ns) |
Modality |
| Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 |
1.166 |
631,437.50 |
736,559.72 |
several? |
More allocations :warning: in #7067
| Benchmark |
Base Allocated |
Diff Allocated |
Change |
Change % |
| Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 |
55.75 KB |
56.39 KB |
636 B |
1.14% |
| Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 |
41.89 KB |
42.27 KB |
387 B |
0.92% |
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
WriteAndFlushEnrichedTraces |
net6.0 |
645μs |
512ns |
1.98μs |
0 |
0 |
0 |
41.73 KB |
| master |
WriteAndFlushEnrichedTraces |
netcoreapp3.1 |
631μs |
1.67μs |
6.67μs |
0 |
0 |
0 |
41.89 KB |
| master |
WriteAndFlushEnrichedTraces |
net472 |
923μs |
1.93μs |
7.2μs |
4.46 |
0 |
0 |
55.75 KB |
| #7067 |
WriteAndFlushEnrichedTraces |
net6.0 |
665μs |
1.05μs |
4.07μs |
0 |
0 |
0 |
41.69 KB |
| #7067 |
WriteAndFlushEnrichedTraces |
netcoreapp3.1 |
724μs |
3.83μs |
19.9μs |
0 |
0 |
0 |
42.27 KB |
| #7067 |
WriteAndFlushEnrichedTraces |
net472 |
907μs |
2.7μs |
10.5μs |
8.33 |
0 |
0 |
56.39 KB |
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
ExecuteNonQuery |
net6.0 |
1.99μs |
10.2ns |
46.8ns |
0 |
0 |
0 |
1.03 KB |
| master |
ExecuteNonQuery |
netcoreapp3.1 |
2.53μs |
3.43ns |
13.3ns |
0 |
0 |
0 |
1.02 KB |
| master |
ExecuteNonQuery |
net472 |
2.7μs |
2.41ns |
9.34ns |
0.147 |
0.0134 |
0 |
995 B |
| #7067 |
ExecuteNonQuery |
net6.0 |
1.98μs |
2.65ns |
9.17ns |
0 |
0 |
0 |
1.03 KB |
| #7067 |
ExecuteNonQuery |
netcoreapp3.1 |
2.5μs |
2.23ns |
8.63ns |
0 |
0 |
0 |
1.02 KB |
| #7067 |
ExecuteNonQuery |
net472 |
2.74μs |
3.39ns |
13.1ns |
0.15 |
0.0136 |
0 |
995 B |
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
CallElasticsearch |
net6.0 |
1.83μs |
5.85ns |
22.7ns |
0 |
0 |
0 |
1.04 KB |
| master |
CallElasticsearch |
netcoreapp3.1 |
2.3μs |
11.8ns |
56.6ns |
0 |
0 |
0 |
1.04 KB |
| master |
CallElasticsearch |
net472 |
3.55μs |
3.05ns |
11.8ns |
0.159 |
0 |
0 |
1.05 KB |
| master |
CallElasticsearchAsync |
net6.0 |
1.81μs |
3.89ns |
14.5ns |
0 |
0 |
0 |
1.02 KB |
| master |
CallElasticsearchAsync |
netcoreapp3.1 |
2.35μs |
7.26ns |
28.1ns |
0 |
0 |
0 |
1.09 KB |
| master |
CallElasticsearchAsync |
net472 |
3.84μs |
3.19ns |
12.4ns |
0.169 |
0 |
0 |
1.11 KB |
| #7067 |
CallElasticsearch |
net6.0 |
1.8μs |
9.5ns |
46.6ns |
0 |
0 |
0 |
1.04 KB |
| #7067 |
CallElasticsearch |
netcoreapp3.1 |
2.27μs |
7.77ns |
26.9ns |
0 |
0 |
0 |
1.04 KB |
| #7067 |
CallElasticsearch |
net472 |
3.63μs |
5.58ns |
21.6ns |
0.166 |
0 |
0 |
1.05 KB |
| #7067 |
CallElasticsearchAsync |
net6.0 |
1.83μs |
6.21ns |
22.4ns |
0 |
0 |
0 |
1.02 KB |
| #7067 |
CallElasticsearchAsync |
netcoreapp3.1 |
2.41μs |
9.63ns |
37.3ns |
0 |
0 |
0 |
1.09 KB |
| #7067 |
CallElasticsearchAsync |
net472 |
3.85μs |
1.61ns |
6.03ns |
0.173 |
0 |
0 |
1.11 KB |
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
ExecuteAsync |
net6.0 |
1.78μs |
9.27ns |
43.5ns |
0 |
0 |
0 |
960 B |
| master |
ExecuteAsync |
netcoreapp3.1 |
2.29μs |
7.45ns |
25.8ns |
0 |
0 |
0 |
960 B |
| master |
ExecuteAsync |
net472 |
2.58μs |
1.78ns |
6.88ns |
0.143 |
0 |
0 |
923 B |
| #7067 |
ExecuteAsync |
net6.0 |
1.82μs |
1.16ns |
4.33ns |
0 |
0 |
0 |
960 B |
| #7067 |
ExecuteAsync |
netcoreapp3.1 |
2.3μs |
9.38ns |
36.3ns |
0 |
0 |
0 |
960 B |
| #7067 |
ExecuteAsync |
net472 |
2.58μs |
0.921ns |
3.57ns |
0.141 |
0 |
0 |
923 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
SendAsync |
net6.0 |
7.1μs |
10.4ns |
39ns |
0 |
0 |
0 |
2.37 KB |
| master |
SendAsync |
netcoreapp3.1 |
8.69μs |
14.1ns |
54.6ns |
0 |
0 |
0 |
2.9 KB |
| master |
SendAsync |
net472 |
12.5μs |
10.3ns |
38.5ns |
0.498 |
0 |
0 |
3.19 KB |
| #7067 |
SendAsync |
net6.0 |
7.1μs |
8.49ns |
32.9ns |
0 |
0 |
0 |
2.37 KB |
| #7067 |
SendAsync |
netcoreapp3.1 |
8.49μs |
11.8ns |
45.5ns |
0 |
0 |
0 |
2.9 KB |
| #7067 |
SendAsync |
net472 |
12.5μs |
10.7ns |
41.6ns |
0.501 |
0 |
0 |
3.19 KB |
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower :warning: More allocations :warning:
Slower :warning: in #7067
| Benchmark |
diff/base |
Base Median (ns) |
Diff Median (ns) |
Modality |
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 |
1.230 |
413,700.00 |
508,850.00 |
|
More allocations :warning: in #7067
| Benchmark |
Base Allocated |
Diff Allocated |
Change |
Change % |
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 |
57.34 KB |
73.73 KB |
16.38 KB |
28.57% |
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 |
259.96 KB |
276.5 KB |
16.54 KB |
6.36% |
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 |
42.64 KB |
43.19 KB |
552 B |
1.29% |
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 |
274.93 KB |
277.29 KB |
2.36 KB |
0.86% |
Fewer allocations :tada: in #7067
| Benchmark |
Base Allocated |
Diff Allocated |
Change |
Change % |
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 |
43.83 KB |
43.1 KB |
-728 B |
-1.66% |
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 |
286.72 KB |
278.53 KB |
-8.19 KB |
-2.86% |
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
StringConcatBenchmark |
net6.0 |
44.6μs |
212ns |
1.6μs |
0 |
0 |
0 |
43.83 KB |
| master |
StringConcatBenchmark |
netcoreapp3.1 |
47.2μs |
224ns |
838ns |
0 |
0 |
0 |
42.64 KB |
| master |
StringConcatBenchmark |
net472 |
56.8μs |
259ns |
968ns |
0 |
0 |
0 |
57.34 KB |
| master |
StringConcatAspectBenchmark |
net6.0 |
458μs |
1.08μs |
3.89μs |
0 |
0 |
0 |
259.96 KB |
| master |
StringConcatAspectBenchmark |
netcoreapp3.1 |
447μs |
6.44μs |
63.7μs |
0 |
0 |
0 |
274.93 KB |
| master |
StringConcatAspectBenchmark |
net472 |
410μs |
2.07μs |
9.27μs |
0 |
0 |
0 |
286.72 KB |
| #7067 |
StringConcatBenchmark |
net6.0 |
47.3μs |
233ns |
1.38μs |
0 |
0 |
0 |
43.1 KB |
| #7067 |
StringConcatBenchmark |
netcoreapp3.1 |
46.5μs |
225ns |
983ns |
0 |
0 |
0 |
43.19 KB |
| #7067 |
StringConcatBenchmark |
net472 |
56.9μs |
244ns |
947ns |
0 |
0 |
0 |
73.73 KB |
| #7067 |
StringConcatAspectBenchmark |
net6.0 |
504μs |
2.32μs |
9.28μs |
0 |
0 |
0 |
276.5 KB |
| #7067 |
StringConcatAspectBenchmark |
netcoreapp3.1 |
513μs |
2.28μs |
8.53μs |
0 |
0 |
0 |
277.29 KB |
| #7067 |
StringConcatAspectBenchmark |
net472 |
406μs |
2.16μs |
15.1μs |
0 |
0 |
0 |
278.53 KB |
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
EnrichedLog |
net6.0 |
2.63μs |
2.6ns |
10.1ns |
0 |
0 |
0 |
1.76 KB |
| master |
EnrichedLog |
netcoreapp3.1 |
3.44μs |
4.38ns |
17ns |
0 |
0 |
0 |
1.76 KB |
| master |
EnrichedLog |
net472 |
4.05μs |
4.83ns |
18.7ns |
0.265 |
0 |
0 |
1.69 KB |
| #7067 |
EnrichedLog |
net6.0 |
2.59μs |
12.3ns |
47.7ns |
0 |
0 |
0 |
1.76 KB |
| #7067 |
EnrichedLog |
netcoreapp3.1 |
3.51μs |
6.32ns |
24.5ns |
0 |
0 |
0 |
1.76 KB |
| #7067 |
EnrichedLog |
net472 |
4.08μs |
3.98ns |
15.4ns |
0.267 |
0 |
0 |
1.69 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
EnrichedLog |
net6.0 |
123μs |
145ns |
563ns |
0 |
0 |
0 |
4.37 KB |
| master |
EnrichedLog |
netcoreapp3.1 |
126μs |
330ns |
1.24μs |
0 |
0 |
0 |
4.37 KB |
| master |
EnrichedLog |
net472 |
167μs |
161ns |
603ns |
0 |
0 |
0 |
4.57 KB |
| #7067 |
EnrichedLog |
net6.0 |
122μs |
125ns |
483ns |
0 |
0 |
0 |
4.37 KB |
| #7067 |
EnrichedLog |
netcoreapp3.1 |
127μs |
181ns |
627ns |
0 |
0 |
0 |
4.37 KB |
| #7067 |
EnrichedLog |
net472 |
167μs |
260ns |
1.01μs |
0 |
0 |
0 |
4.57 KB |
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
EnrichedLog |
net6.0 |
4.94μs |
19.3ns |
74.6ns |
0 |
0 |
0 |
2.32 KB |
| master |
EnrichedLog |
netcoreapp3.1 |
6.77μs |
22.2ns |
79.9ns |
0 |
0 |
0 |
2.32 KB |
| master |
EnrichedLog |
net472 |
7.45μs |
8.04ns |
31.1ns |
0.335 |
0 |
0 |
2.14 KB |
| #7067 |
EnrichedLog |
net6.0 |
4.99μs |
15.8ns |
61ns |
0 |
0 |
0 |
2.32 KB |
| #7067 |
EnrichedLog |
netcoreapp3.1 |
6.64μs |
3.73ns |
14ns |
0 |
0 |
0 |
2.32 KB |
| #7067 |
EnrichedLog |
net472 |
7.5μs |
3.99ns |
14.9ns |
0.337 |
0 |
0 |
2.14 KB |
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
SendReceive |
net6.0 |
2.04μs |
10.5ns |
49.4ns |
0 |
0 |
0 |
1.21 KB |
| master |
SendReceive |
netcoreapp3.1 |
2.53μs |
11.8ns |
47.4ns |
0 |
0 |
0 |
1.21 KB |
| master |
SendReceive |
net472 |
3.28μs |
2.71ns |
10.5ns |
0.178 |
0 |
0 |
1.21 KB |
| #7067 |
SendReceive |
net6.0 |
2.01μs |
9.56ns |
40.6ns |
0 |
0 |
0 |
1.21 KB |
| #7067 |
SendReceive |
netcoreapp3.1 |
2.51μs |
8.33ns |
32.3ns |
0 |
0 |
0 |
1.21 KB |
| #7067 |
SendReceive |
net472 |
3.18μs |
2.67ns |
10.3ns |
0.191 |
0 |
0 |
1.21 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
EnrichedLog |
net6.0 |
4.18μs |
0.92ns |
3.44ns |
0 |
0 |
0 |
1.64 KB |
| master |
EnrichedLog |
netcoreapp3.1 |
5.64μs |
23ns |
89.3ns |
0 |
0 |
0 |
1.69 KB |
| master |
EnrichedLog |
net472 |
6.67μs |
7.28ns |
27.3ns |
0.298 |
0 |
0 |
2.08 KB |
| #7067 |
EnrichedLog |
net6.0 |
4.31μs |
5.17ns |
19.3ns |
0 |
0 |
0 |
1.64 KB |
| #7067 |
EnrichedLog |
netcoreapp3.1 |
5.59μs |
6.42ns |
24.8ns |
0 |
0 |
0 |
1.69 KB |
| #7067 |
EnrichedLog |
net472 |
6.53μs |
6.89ns |
26.7ns |
0.325 |
0 |
0 |
2.08 KB |
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
StartFinishSpan |
net6.0 |
751ns |
3.97ns |
22.4ns |
0 |
0 |
0 |
584 B |
| master |
StartFinishSpan |
netcoreapp3.1 |
953ns |
4.42ns |
17.7ns |
0 |
0 |
0 |
584 B |
| master |
StartFinishSpan |
net472 |
914ns |
0.809ns |
3.13ns |
0.0912 |
0 |
0 |
586 B |
| master |
StartFinishScope |
net6.0 |
918ns |
0.479ns |
1.73ns |
0 |
0 |
0 |
704 B |
| master |
StartFinishScope |
netcoreapp3.1 |
1.15μs |
6.24ns |
34.2ns |
0 |
0 |
0 |
704 B |
| master |
StartFinishScope |
net472 |
1.09μs |
0.174ns |
0.652ns |
0.104 |
0 |
0 |
666 B |
| #7067 |
StartFinishSpan |
net6.0 |
748ns |
3.85ns |
18.1ns |
0 |
0 |
0 |
584 B |
| #7067 |
StartFinishSpan |
netcoreapp3.1 |
929ns |
4.94ns |
24.7ns |
0 |
0 |
0 |
584 B |
| #7067 |
StartFinishSpan |
net472 |
914ns |
0.223ns |
0.864ns |
0.092 |
0 |
0 |
586 B |
| #7067 |
StartFinishScope |
net6.0 |
895ns |
4.29ns |
17.1ns |
0 |
0 |
0 |
704 B |
| #7067 |
StartFinishScope |
netcoreapp3.1 |
1.18μs |
1.27ns |
4.76ns |
0 |
0 |
0 |
705 B |
| #7067 |
StartFinishScope |
net472 |
1.12μs |
0.14ns |
0.506ns |
0.101 |
0 |
0 |
666 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch |
Method |
Toolchain |
Mean |
StdError |
StdDev |
Gen 0 |
Gen 1 |
Gen 2 |
Allocated |
| master |
RunOnMethodBegin |
net6.0 |
1.02μs |
5.36ns |
26.8ns |
0 |
0 |
0 |
704 B |
| master |
RunOnMethodBegin |
netcoreapp3.1 |
1.38μs |
2.28ns |
8.82ns |
0 |
0 |
0 |
704 B |
| master |
RunOnMethodBegin |
net472 |
1.36μs |
0.141ns |
0.544ns |
0.102 |
0 |
0 |
666 B |
| #7067 |
RunOnMethodBegin |
net6.0 |
1.03μs |
4.74ns |
17.7ns |
0 |
0 |
0 |
704 B |
| #7067 |
RunOnMethodBegin |
netcoreapp3.1 |
1.33μs |
3.27ns |
12.7ns |
0 |
0 |
0 |
704 B |
| #7067 |
RunOnMethodBegin |
net472 |
1.38μs |
0.395ns |
1.53ns |
0.104 |
0 |
0 |
666 B |
Closing as stale, likely something to consider though in the future if we want to pin system-tests but I'm not terribly motivated to do so at the time being
dd-trace-dotnet copied to clipboard
DataDog
![datadog-datadog-prod-us1[bot] avatar](https://avatars.githubusercontent.com/u/365230?v=4 "Published by a pub.dev verified publisher"){kind=small}
