dd-trace-dotnet
dd-trace-dotnet copied to clipboard
DataDog
[Tests] Retry Kafka and ES6 tests on telemetry assertion failure
Summary of changes
Retry tests that fail on telemetry assertion failure, only when we miss the app closing event.
Reason for change
Kafka and ES6 tests have failed 6 pipelines this week. There's a shutdown bug in telemetry, which fails during the final flush. We've grabbed memory dumps from it and don't know how to fix it. So I assume retrying is acceptable in that case.
Implementation details
Just use the retry logic already in place that report metrics for retries.
Test coverage
Other details
Datadog Report
Branch report: pierre/test-logs
Commit report: 25659f3
:white_check_mark: dd-trace-dotnet: 0 Failed, 0 New Flaky, 299688 Passed, 937 Skipped, 31m 3.91s Wall Time
![datadog-ddstaging[bot] avatar](https://avatars.githubusercontent.com/u/365230?v=4 "Published by a pub.dev verified publisher"){kind=small}
Execution-Time Benchmarks Report :stopwatch:
Execution-time results for samples comparing the following branches/commits:
Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
- Welch test with statistical test for significance of 5%
- Only results indicating a difference greater than 5% and 5 ms are considered.
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.
Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).
gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4927) - mean (72ms) : 61, 82
. : milestone, 72,
master - mean (71ms) : 62, 80
. : milestone, 71,
section CallTarget+Inlining+NGEN
This PR (4927) - mean (994ms) : 969, 1019
. : milestone, 994,
master - mean (984ms) : 957, 1011
. : milestone, 984,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4927) - mean (106ms) : 102, 109
. : milestone, 106,
master - mean (105ms) : 100, 109
. : milestone, 105,
section CallTarget+Inlining+NGEN
This PR (4927) - mean (684ms) : 660, 707
. : milestone, 684,
master - mean (684ms) : 665, 703
. : milestone, 684,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4927) - mean (90ms) : 87, 93
. : milestone, 90,
master - mean (89ms) : 84, 94
. : milestone, 89,
section CallTarget+Inlining+NGEN
This PR (4927) - mean (660ms) : 629, 690
. : milestone, 660,
master - mean (659ms) : 629, 689
. : milestone, 659,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4927) - mean (188ms) : 184, 192
. : milestone, 188,
master - mean (188ms) : 184, 192
. : milestone, 188,
section CallTarget+Inlining+NGEN
This PR (4927) - mean (1,132ms) : 1113, 1151
. : milestone, 1132,
master - mean (1,132ms) : 1109, 1154
. : milestone, 1132,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4927) - mean (274ms) : 268, 279
. : milestone, 274,
master - mean (271ms) : 267, 275
. : milestone, 271,
section CallTarget+Inlining+NGEN
This PR (4927) - mean (1,098ms) : 1070, 1125
. : milestone, 1098,
master - mean (1,085ms) : 1061, 1110
. : milestone, 1085,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4927) - mean (261ms) : 257, 265
. : milestone, 261,
master - mean (261ms) : 258, 265
. : milestone, 261,
section CallTarget+Inlining+NGEN
This PR (4927) - mean (1,058ms) : 1030, 1085
. : milestone, 1058,
master - mean (1,056ms) : 1032, 1079
. : milestone, 1056,
Throughput/Crank Report:zap:
Throughput results for AspNetCoreSimpleController comparing the following branches/commits:
Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.
Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!
gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4927) (11.115M) : 0, 11114892
master (11.399M) : 0, 11399143
benchmarks/2.9.0 (11.363M) : 0, 11362982
section Automatic
This PR (4927) (7.630M) : 0, 7629598
master (7.766M) : 0, 7765795
benchmarks/2.9.0 (8.259M) : 0, 8258618
section Trace stats
This PR (4927) (8.033M) : 0, 8033317
master (8.042M) : 0, 8042224
section Manual
This PR (4927) (9.560M) : 0, 9559699
master (9.522M) : 0, 9522223
section Manual + Automatic
This PR (4927) (7.202M) : 0, 7202005
master (7.447M) : 0, 7447383
section Version Conflict
This PR (4927) (6.479M) : 0, 6479063
master (6.775M) : 0, 6775133
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4927) (9.583M) : 0, 9583430
master (9.789M) : 0, 9789391
benchmarks/2.9.0 (9.602M) : 0, 9602224
section Automatic
This PR (4927) (6.576M) : 0, 6576139
master (6.529M) : 0, 6529231
section Trace stats
This PR (4927) (6.881M) : 0, 6880965
master (6.918M) : 0, 6917556
section Manual
This PR (4927) (8.325M) : 0, 8325015
master (8.224M) : 0, 8223914
section Manual + Automatic
This PR (4927) (6.107M) : 0, 6107125
master (6.223M) : 0, 6222722
section Version Conflict
This PR (4927) (5.570M) : 0, 5569724
master (5.659M) : 0, 5658699
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4927) (10.286M) : 0, 10286373
master (9.896M) : 0, 9895779
benchmarks/2.9.0 (10.259M) : 0, 10259311
section Automatic
This PR (4927) (7.267M) : 0, 7266922
master (7.153M) : 0, 7153186
benchmarks/2.9.0 (7.710M) : 0, 7709529
section Trace stats
This PR (4927) (7.531M) : 0, 7531301
master (7.475M) : 0, 7475105
section Manual
This PR (4927) (8.983M) : 0, 8982559
master (8.673M) : 0, 8672626
section Manual + Automatic
This PR (4927) (6.925M) : 0, 6925337
master (6.808M) : 0, 6807618
section Version Conflict
This PR (4927) (6.254M) : 0, 6254130
master (6.119M) : 0, 6119052
gantt
title Throughput Linux x64 (ASM) (Total requests)
dateFormat X
axisFormat %s
section Baseline
master (7.421M) : 0, 7420648
benchmarks/2.9.0 (7.830M) : 0, 7830154
section No attack
master (1.988M) : 0, 1987679
benchmarks/2.9.0 (3.253M) : 0, 3253296
section Attack
master (1.584M) : 0, 1583698
benchmarks/2.9.0 (2.541M) : 0, 2540898
section Blocking
master (3.198M) : 0, 3197539
section IAST default
master (6.469M) : 0, 6468524
section IAST full
master (5.804M) : 0, 5803735
section Base vuln
master (0.938M) : 0, 937552
section IAST vuln
master (0.878M) : 0, 877600
Benchmarks Report :snail:
Benchmarks for #4927 compared to master:
- 3 benchmarks are faster, with geometric mean 1.129
- 2 benchmarks are slower, with geometric mean 1.396
- 1 benchmarks have more allocations
The following thresholds were used for comparing the benchmark speeds:
- Mann–Whitney U test with statistical test for significance of 5%
- Only results indicating a difference greater than 10% and 0.3 ns are considered.
Allocation changes below 0.5% are ignored.
Benchmark details
Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartStopWithChild |
net6.0 | 9.1μs | 50.2ns | 318ns | 0.0219 | 0.00876 | 0 | 7.48 KB |
| master | StartStopWithChild |
netcoreapp3.1 | 11.3μs | 63.3ns | 410ns | 0.0233 | 0.0117 | 0 | 7.58 KB |
| master | StartStopWithChild |
net472 | 17.3μs | 43ns | 167ns | 1.33 | 0.361 | 0.103 | 7.94 KB |
| #4927 | StartStopWithChild |
net6.0 | 8.52μs | 45.7ns | 250ns | 0.0255 | 0.00849 | 0 | 7.48 KB |
| #4927 | StartStopWithChild |
netcoreapp3.1 | 10.7μs | 51.6ns | 219ns | 0.0211 | 0.0106 | 0 | 7.58 KB |
| #4927 | StartStopWithChild |
net472 | 17.1μs | 55.4ns | 207ns | 1.35 | 0.381 | 0.127 | 7.95 KB |
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | WriteAndFlushEnrichedTraces |
net6.0 | 461μs | 190ns | 736ns | 0 | 0 | 0 | 2.7 KB |
| master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 632μs | 111ns | 401ns | 0 | 0 | 0 | 2.7 KB |
| master | WriteAndFlushEnrichedTraces |
net472 | 776μs | 179ns | 692ns | 0.388 | 0 | 0 | 3.3 KB |
| #4927 | WriteAndFlushEnrichedTraces |
net6.0 | 459μs | 515ns | 1.99μs | 0 | 0 | 0 | 2.7 KB |
| #4927 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 641μs | 296ns | 1.15μs | 0 | 0 | 0 | 2.7 KB |
| #4927 | WriteAndFlushEnrichedTraces |
net472 | 782μs | 106ns | 396ns | 0.388 | 0 | 0 | 3.3 KB |
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster :tada: Same allocations :heavy_check_mark:
Faster :tada: in #4927
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1
1.152
208.57
181.04
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | AllCycleSimpleBody |
net6.0 | 43.3μs | 27ns | 104ns | 0.0221 | 0 | 0 | 1.77 KB |
| master | AllCycleSimpleBody |
netcoreapp3.1 | 46.6μs | 61.1ns | 236ns | 0.0236 | 0 | 0 | 1.74 KB |
| master | AllCycleSimpleBody |
net472 | 49.9μs | 108ns | 390ns | 0.268 | 0 | 0 | 1.81 KB |
| master | AllCycleMoreComplexBody |
net6.0 | 215μs | 67.3ns | 243ns | 0.107 | 0 | 0 | 9.25 KB |
| master | AllCycleMoreComplexBody |
netcoreapp3.1 | 226μs | 109ns | 394ns | 0.112 | 0 | 0 | 9.14 KB |
| master | AllCycleMoreComplexBody |
net472 | 239μs | 350ns | 1.35μs | 1.43 | 0 | 0 | 9.32 KB |
| master | ObjectExtractorSimpleBody |
net6.0 | 132ns | 0.0946ns | 0.366ns | 0.00394 | 0 | 0 | 280 B |
| master | ObjectExtractorSimpleBody |
netcoreapp3.1 | 209ns | 0.314ns | 1.22ns | 0.00375 | 0 | 0 | 272 B |
| master | ObjectExtractorSimpleBody |
net472 | 151ns | 0.0724ns | 0.28ns | 0.0446 | 0 | 0 | 281 B |
| master | ObjectExtractorMoreComplexBody |
net6.0 | 2.77μs | 1.26ns | 4.71ns | 0.0527 | 0 | 0 | 3.78 KB |
| master | ObjectExtractorMoreComplexBody |
netcoreapp3.1 | 3.7μs | 1.25ns | 4.69ns | 0.05 | 0 | 0 | 3.69 KB |
| master | ObjectExtractorMoreComplexBody |
net472 | 3.49μs | 1.92ns | 7.44ns | 0.603 | 0.00524 | 0 | 3.8 KB |
| #4927 | AllCycleSimpleBody |
net6.0 | 44.7μs | 25.2ns | 97.5ns | 0.022 | 0 | 0 | 1.77 KB |
| #4927 | AllCycleSimpleBody |
netcoreapp3.1 | 46.3μs | 74.6ns | 289ns | 0 | 0 | 0 | 1.74 KB |
| #4927 | AllCycleSimpleBody |
net472 | 48.8μs | 111ns | 430ns | 0.268 | 0 | 0 | 1.81 KB |
| #4927 | AllCycleMoreComplexBody |
net6.0 | 211μs | 92.7ns | 321ns | 0.106 | 0 | 0 | 9.25 KB |
| #4927 | AllCycleMoreComplexBody |
netcoreapp3.1 | 224μs | 264ns | 1.02μs | 0.112 | 0 | 0 | 9.14 KB |
| #4927 | AllCycleMoreComplexBody |
net472 | 239μs | 111ns | 429ns | 1.43 | 0 | 0 | 9.32 KB |
| #4927 | ObjectExtractorSimpleBody |
net6.0 | 130ns | 0.0575ns | 0.215ns | 0.00394 | 0 | 0 | 280 B |
| #4927 | ObjectExtractorSimpleBody |
netcoreapp3.1 | 181ns | 0.0509ns | 0.184ns | 0.00375 | 0 | 0 | 272 B |
| #4927 | ObjectExtractorSimpleBody |
net472 | 155ns | 0.779ns | 3.57ns | 0.0446 | 0 | 0 | 281 B |
| #4927 | ObjectExtractorMoreComplexBody |
net6.0 | 2.88μs | 1.36ns | 5.11ns | 0.0535 | 0 | 0 | 3.78 KB |
| #4927 | ObjectExtractorMoreComplexBody |
netcoreapp3.1 | 3.72μs | 0.73ns | 2.83ns | 0.0503 | 0 | 0 | 3.69 KB |
| #4927 | ObjectExtractorMoreComplexBody |
net472 | 3.5μs | 1.56ns | 6.03ns | 0.603 | 0.00527 | 0 | 3.8 KB |
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunWaf(args=NestedMap (10)) |
net6.0 | 52.3μs | 263ns | 1.15μs | 0.214 | 0 | 0 | 16.06 KB |
| master | RunWaf(args=NestedMap (10)) |
netcoreapp3.1 | 67.3μs | 352ns | 1.65μs | 0.208 | 0 | 0 | 16.06 KB |
| master | RunWaf(args=NestedMap (10)) |
net472 | 95.1μs | 63.1ns | 244ns | 2.55 | 0.0946 | 0 | 16.14 KB |
| master | RunWafWithAttack(args=Neste(...)tack) [22]) |
net6.0 | 119μs | 370ns | 1.43μs | 0.292 | 0 | 0 | 22.41 KB |
| master | RunWafWithAttack(args=Neste(...)tack) [22]) |
netcoreapp3.1 | 150μs | 1.61μs | 16.1μs | 0.28 | 0 | 0 | 22.36 KB |
| master | RunWafWithAttack(args=Neste(...)tack) [22]) |
net472 | 169μs | 843ns | 3.58μs | 3.53 | 0.164 | 0 | 22.7 KB |
| master | RunWaf(args=NestedMap (100)) |
net6.0 | 93.8μs | 277ns | 1.07μs | 0.465 | 0 | 0 | 32.76 KB |
| master | RunWaf(args=NestedMap (100)) |
netcoreapp3.1 | 135μs | 168ns | 651ns | 0.401 | 0 | 0 | 33.33 KB |
| master | RunWaf(args=NestedMap (100)) |
net472 | 185μs | 831ns | 3.22μs | 5.31 | 0.366 | 0 | 33.67 KB |
| master | RunWafWithAttack(args=Neste(...)tack) [23]) |
net6.0 | 172μs | 549ns | 2.13μs | 0.49 | 0 | 0 | 39.1 KB |
| master | RunWafWithAttack(args=Neste(...)tack) [23]) |
netcoreapp3.1 | 205μs | 1.07μs | 5.46μs | 0.528 | 0 | 0 | 39.63 KB |
| master | RunWafWithAttack(args=Neste(...)tack) [23]) |
net472 | 267μs | 1.05μs | 4.07μs | 6.34 | 0.518 | 0 | 40.23 KB |
| master | RunWaf(args=NestedMap (20)) |
net6.0 | 100μs | 255ns | 989ns | 0.403 | 0 | 0 | 32.18 KB |
| master | RunWaf(args=NestedMap (20)) |
netcoreapp3.1 | 131μs | 666ns | 3.12μs | 0.405 | 0 | 0 | 32.3 KB |
| master | RunWaf(args=NestedMap (20)) |
net472 | 188μs | 883ns | 3.42μs | 5.14 | 0.367 | 0 | 32.63 KB |
| master | RunWafWithAttack(args=Neste(...)tack) [22]) |
net6.0 | 163μs | 78.4ns | 283ns | 0.517 | 0 | 0 | 38.53 KB |
| master | RunWafWithAttack(args=Neste(...)tack) [22]) |
netcoreapp3.1 | 206μs | 1.01μs | 4.04μs | 0.42 | 0 | 0 | 38.6 KB |
| master | RunWafWithAttack(args=Neste(...)tack) [22]) |
net472 | 262μs | 184ns | 713ns | 6.15 | 0.523 | 0 | 39.2 KB |
| #4927 | RunWaf(args=NestedMap (10)) |
net6.0 | 53μs | 240ns | 928ns | 0.223 | 0 | 0 | 16.06 KB |
| #4927 | RunWaf(args=NestedMap (10)) |
netcoreapp3.1 | 68.5μs | 250ns | 970ns | 0.191 | 0 | 0 | 16.06 KB |
| #4927 | RunWaf(args=NestedMap (10)) |
net472 | 98.3μs | 309ns | 1.16μs | 2.54 | 0.0939 | 0 | 16.14 KB |
| #4927 | RunWafWithAttack(args=Neste(...)tack) [22]) |
net6.0 | 115μs | 73.8ns | 286ns | 0.285 | 0 | 0 | 22.41 KB |
| #4927 | RunWafWithAttack(args=Neste(...)tack) [22]) |
netcoreapp3.1 | 138μs | 730ns | 3.58μs | 0.278 | 0 | 0 | 22.36 KB |
| #4927 | RunWafWithAttack(args=Neste(...)tack) [22]) |
net472 | 167μs | 46.8ns | 169ns | 3.54 | 0.164 | 0 | 22.7 KB |
| #4927 | RunWaf(args=NestedMap (100)) |
net6.0 | 94.1μs | 30.2ns | 113ns | 0.471 | 0 | 0 | 32.76 KB |
| #4927 | RunWaf(args=NestedMap (100)) |
netcoreapp3.1 | 135μs | 701ns | 3.44μs | 0.399 | 0 | 0 | 33.33 KB |
| #4927 | RunWaf(args=NestedMap (100)) |
net472 | 182μs | 65.6ns | 254ns | 5.29 | 0.365 | 0 | 33.67 KB |
| #4927 | RunWafWithAttack(args=Neste(...)tack) [23]) |
net6.0 | 167μs | 779ns | 3.12μs | 0.487 | 0 | 0 | 39.1 KB |
| #4927 | RunWafWithAttack(args=Neste(...)tack) [23]) |
netcoreapp3.1 | 210μs | 275ns | 1.03μs | 0.523 | 0 | 0 | 39.63 KB |
| #4927 | RunWafWithAttack(args=Neste(...)tack) [23]) |
net472 | 261μs | 260ns | 973ns | 6.38 | 0.521 | 0 | 40.23 KB |
| #4927 | RunWaf(args=NestedMap (20)) |
net6.0 | 95.5μs | 24ns | 90ns | 0.424 | 0 | 0 | 32.18 KB |
| #4927 | RunWaf(args=NestedMap (20)) |
netcoreapp3.1 | 125μs | 435ns | 1.68μs | 0.431 | 0 | 0 | 32.3 KB |
| #4927 | RunWaf(args=NestedMap (20)) |
net472 | 181μs | 67.5ns | 261ns | 5.17 | 0.363 | 0 | 32.63 KB |
| #4927 | RunWafWithAttack(args=Neste(...)tack) [22]) |
net6.0 | 170μs | 63.6ns | 246ns | 0.508 | 0 | 0 | 38.53 KB |
| #4927 | RunWafWithAttack(args=Neste(...)tack) [22]) |
netcoreapp3.1 | 211μs | 1.14μs | 6.15μs | 0.504 | 0 | 0 | 38.6 KB |
| #4927 | RunWafWithAttack(args=Neste(...)tack) [22]) |
net472 | 260μs | 195ns | 756ns | 6.15 | 0.523 | 0 | 39.2 KB |
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendRequest |
net6.0 | 173μs | 123ns | 477ns | 0.259 | 0 | 0 | 18.24 KB |
| master | SendRequest |
netcoreapp3.1 | 195μs | 361ns | 1.4μs | 0.193 | 0 | 0 | 20.4 KB |
| master | SendRequest |
net472 | 0ns | 0ns | 0ns | 0 | 0 | 0 | 0 b |
| #4927 | SendRequest |
net6.0 | 172μs | 108ns | 417ns | 0.257 | 0 | 0 | 18.24 KB |
| #4927 | SendRequest |
netcoreapp3.1 | 191μs | 393ns | 1.52μs | 0.19 | 0 | 0 | 20.4 KB |
| #4927 | SendRequest |
net472 | 0.000223ns | 0.000144ns | 0.000539ns | 0 | 0 | 0 | 0 b |
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | WriteAndFlushEnrichedTraces |
net6.0 | 558μs | 1.49μs | 5.75μs | 0.558 | 0 | 0 | 41.41 KB |
| master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 658μs | 852ns | 3.19μs | 0.329 | 0 | 0 | 41.93 KB |
| master | WriteAndFlushEnrichedTraces |
net472 | 831μs | 2.78μs | 10.4μs | 8.13 | 2.57 | 0.428 | 53.22 KB |
| #4927 | WriteAndFlushEnrichedTraces |
net6.0 | 568μs | 279ns | 1.08μs | 0.548 | 0 | 0 | 41.52 KB |
| #4927 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 652μs | 569ns | 2.13μs | 0.321 | 0 | 0 | 41.75 KB |
| #4927 | WriteAndFlushEnrichedTraces |
net472 | 830μs | 3.21μs | 12.4μs | 8.39 | 2.52 | 0.419 | 53.26 KB |
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteNonQuery |
net6.0 | 1.1μs | 0.842ns | 3.26ns | 0.0105 | 0 | 0 | 768 B |
| master | ExecuteNonQuery |
netcoreapp3.1 | 1.41μs | 1.18ns | 4.58ns | 0.0103 | 0 | 0 | 768 B |
| master | ExecuteNonQuery |
net472 | 1.79μs | 0.755ns | 2.92ns | 0.115 | 0 | 0 | 730 B |
| #4927 | ExecuteNonQuery |
net6.0 | 1.03μs | 0.414ns | 1.6ns | 0.0109 | 0 | 0 | 768 B |
| #4927 | ExecuteNonQuery |
netcoreapp3.1 | 1.53μs | 3.98ns | 15.4ns | 0.0104 | 0 | 0 | 768 B |
| #4927 | ExecuteNonQuery |
net472 | 1.82μs | 0.877ns | 3.4ns | 0.116 | 0 | 0 | 730 B |
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | CallElasticsearch |
net6.0 | 1.23μs | 0.554ns | 2.15ns | 0.0129 | 0 | 0 | 936 B |
| master | CallElasticsearch |
netcoreapp3.1 | 1.53μs | 1.83ns | 6.84ns | 0.0125 | 0 | 0 | 936 B |
| master | CallElasticsearch |
net472 | 2.55μs | 1.35ns | 5.23ns | 0.151 | 0 | 0 | 955 B |
| master | CallElasticsearchAsync |
net6.0 | 1.28μs | 0.494ns | 1.91ns | 0.0127 | 0 | 0 | 912 B |
| master | CallElasticsearchAsync |
netcoreapp3.1 | 1.6μs | 0.868ns | 3.13ns | 0.013 | 0 | 0 | 984 B |
| master | CallElasticsearchAsync |
net472 | 2.71μs | 0.922ns | 3.57ns | 0.16 | 0 | 0 | 1.01 KB |
| #4927 | CallElasticsearch |
net6.0 | 1.24μs | 0.705ns | 2.73ns | 0.0127 | 0 | 0 | 936 B |
| #4927 | CallElasticsearch |
netcoreapp3.1 | 1.52μs | 1.12ns | 4.34ns | 0.0127 | 0 | 0 | 936 B |
| #4927 | CallElasticsearch |
net472 | 2.64μs | 0.835ns | 3.23ns | 0.151 | 0.00126 | 0 | 955 B |
| #4927 | CallElasticsearchAsync |
net6.0 | 1.32μs | 0.799ns | 2.99ns | 0.0126 | 0 | 0 | 912 B |
| #4927 | CallElasticsearchAsync |
netcoreapp3.1 | 1.53μs | 0.419ns | 1.57ns | 0.0129 | 0 | 0 | 984 B |
| #4927 | CallElasticsearchAsync |
net472 | 2.74μs | 1.43ns | 5.54ns | 0.159 | 0 | 0 | 1.01 KB |
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteAsync |
net6.0 | 1.31μs | 0.492ns | 1.78ns | 0.0125 | 0 | 0 | 912 B |
| master | ExecuteAsync |
netcoreapp3.1 | 1.61μs | 1.81ns | 6.99ns | 0.012 | 0 | 0 | 912 B |
| master | ExecuteAsync |
net472 | 1.78μs | 1.02ns | 3.96ns | 0.138 | 0 | 0 | 875 B |
| #4927 | ExecuteAsync |
net6.0 | 1.32μs | 0.661ns | 2.47ns | 0.0125 | 0 | 0 | 912 B |
| #4927 | ExecuteAsync |
netcoreapp3.1 | 1.58μs | 0.974ns | 3.77ns | 0.0127 | 0 | 0 | 912 B |
| #4927 | ExecuteAsync |
net472 | 1.82μs | 0.878ns | 3.29ns | 0.139 | 0 | 0 | 875 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendAsync |
net6.0 | 4.15μs | 0.798ns | 2.99ns | 0.0291 | 0 | 0 | 2.1 KB |
| master | SendAsync |
netcoreapp3.1 | 5.06μs | 2.07ns | 8.03ns | 0.0357 | 0 | 0 | 2.63 KB |
| master | SendAsync |
net472 | 7.65μs | 1.66ns | 6.23ns | 0.523 | 0 | 0 | 3.31 KB |
| #4927 | SendAsync |
net6.0 | 4.12μs | 2.43ns | 9.1ns | 0.0288 | 0 | 0 | 2.1 KB |
| #4927 | SendAsync |
netcoreapp3.1 | 4.87μs | 1.8ns | 6.96ns | 0.0341 | 0 | 0 | 2.63 KB |
| #4927 | SendAsync |
net472 | 7.76μs | 4.09ns | 15.9ns | 0.523 | 0 | 0 | 3.31 KB |
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower :warning: More allocations :warning:
Slower :warning: in #4927
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net6.0
1.503
65,200.00
98,000.00
multimodal
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472
1.296
48,600.00
63,000.00
More allocations :warning: in #4927
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472
58.5 KB
59.66 KB
1.17 KB
2.00%
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StringConcatBenchmark(parameters=Syste(...)ring] [48]) |
net6.0 | 51.3μs | 240ns | 898ns | 0 | 0 | 0 | 43.44 KB |
| master | StringConcatBenchmark(parameters=Syste(...)ring] [48]) |
netcoreapp3.1 | 53.3μs | 164ns | 636ns | 0 | 0 | 0 | 42.64 KB |
| master | StringConcatBenchmark(parameters=Syste(...)ring] [48]) |
net472 | 38μs | 108ns | 402ns | 0 | 0 | 0 | 58.5 KB |
| master | StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) |
net6.0 | 65.3μs | 337ns | 1.54μs | 0 | 0 | 0 | 43.29 KB |
| master | StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) |
netcoreapp3.1 | 104μs | 4.01μs | 39.9μs | 0 | 0 | 0 | 42.64 KB |
| master | StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) |
net472 | 48.5μs | 194ns | 671ns | 0 | 0 | 0 | 57.34 KB |
| #4927 | StringConcatBenchmark(parameters=Syste(...)ring] [48]) |
net6.0 | 57.6μs | 626ns | 6.19μs | 0 | 0 | 0 | 43.44 KB |
| #4927 | StringConcatBenchmark(parameters=Syste(...)ring] [48]) |
netcoreapp3.1 | 52.8μs | 150ns | 543ns | 0 | 0 | 0 | 42.64 KB |
| #4927 | StringConcatBenchmark(parameters=Syste(...)ring] [48]) |
net472 | 37.7μs | 50.5ns | 175ns | 0 | 0 | 0 | 59.66 KB |
| #4927 | StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) |
net6.0 | 98.5μs | 2.35μs | 23.5μs | 0 | 0 | 0 | 43.29 KB |
| #4927 | StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) |
netcoreapp3.1 | 107μs | 3.8μs | 37.6μs | 0 | 0 | 0 | 42.64 KB |
| #4927 | StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) |
net472 | 63.2μs | 349ns | 2.01μs | 0 | 0 | 0 | 57.34 KB |
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 1.55μs | 1.47ns | 5.68ns | 0.0217 | 0 | 0 | 1.57 KB |
| master | EnrichedLog |
netcoreapp3.1 | 2.22μs | 0.796ns | 2.98ns | 0.021 | 0 | 0 | 1.57 KB |
| master | EnrichedLog |
net472 | 2.51μs | 1.24ns | 4.64ns | 0.238 | 0 | 0 | 1.5 KB |
| #4927 | EnrichedLog |
net6.0 | 1.48μs | 0.487ns | 1.82ns | 0.0216 | 0 | 0 | 1.57 KB |
| #4927 | EnrichedLog |
netcoreapp3.1 | 2.31μs | 0.857ns | 3.09ns | 0.0209 | 0 | 0 | 1.57 KB |
| #4927 | EnrichedLog |
net472 | 2.62μs | 1.84ns | 6.62ns | 0.237 | 0 | 0 | 1.5 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 114μs | 144ns | 559ns | 0.0566 | 0 | 0 | 4.21 KB |
| master | EnrichedLog |
netcoreapp3.1 | 118μs | 129ns | 501ns | 0 | 0 | 0 | 4.21 KB |
| master | EnrichedLog |
net472 | 150μs | 159ns | 615ns | 0.672 | 0.224 | 0 | 4.39 KB |
| #4927 | EnrichedLog |
net6.0 | 113μs | 130ns | 486ns | 0.0559 | 0 | 0 | 4.21 KB |
| #4927 | EnrichedLog |
netcoreapp3.1 | 118μs | 188ns | 728ns | 0.0584 | 0 | 0 | 4.21 KB |
| #4927 | EnrichedLog |
net472 | 147μs | 74.8ns | 290ns | 0.66 | 0.22 | 0 | 4.39 KB |
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 2.78μs | 0.757ns | 2.73ns | 0.0293 | 0 | 0 | 2.13 KB |
| master | EnrichedLog |
netcoreapp3.1 | 4.19μs | 5.59ns | 21.6ns | 0.0279 | 0 | 0 | 2.13 KB |
| master | EnrichedLog |
net472 | 4.84μs | 3.21ns | 12ns | 0.309 | 0 | 0 | 1.95 KB |
| #4927 | EnrichedLog |
net6.0 | 2.9μs | 1.3ns | 4.87ns | 0.0303 | 0 | 0 | 2.13 KB |
| #4927 | EnrichedLog |
netcoreapp3.1 | 4.3μs | 1.68ns | 6.3ns | 0.0278 | 0 | 0 | 2.13 KB |
| #4927 | EnrichedLog |
net472 | 4.84μs | 2.24ns | 8.67ns | 0.309 | 0 | 0 | 1.95 KB |
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendReceive |
net6.0 | 1.47μs | 1.03ns | 3.97ns | 0.0155 | 0 | 0 | 1.1 KB |
| master | SendReceive |
netcoreapp3.1 | 1.71μs | 3.69ns | 14.3ns | 0.0145 | 0 | 0 | 1.1 KB |
| master | SendReceive |
net472 | 2.13μs | 1.51ns | 5.84ns | 0.177 | 0 | 0 | 1.12 KB |
| #4927 | SendReceive |
net6.0 | 1.37μs | 0.966ns | 3.74ns | 0.0151 | 0 | 0 | 1.1 KB |
| #4927 | SendReceive |
netcoreapp3.1 | 1.86μs | 1.39ns | 5.2ns | 0.0148 | 0 | 0 | 1.1 KB |
| #4927 | SendReceive |
net472 | 2.21μs | 1.24ns | 4.79ns | 0.177 | 0 | 0 | 1.12 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 2.71μs | 0.966ns | 3.74ns | 0.0217 | 0 | 0 | 1.53 KB |
| master | EnrichedLog |
netcoreapp3.1 | 3.99μs | 4.02ns | 15.6ns | 0.0198 | 0 | 0 | 1.58 KB |
| master | EnrichedLog |
net472 | 4.34μs | 1.7ns | 6.59ns | 0.312 | 0 | 0 | 1.97 KB |
| #4927 | EnrichedLog |
net6.0 | 2.67μs | 1.94ns | 7.52ns | 0.0212 | 0 | 0 | 1.53 KB |
| #4927 | EnrichedLog |
netcoreapp3.1 | 3.92μs | 1.03ns | 3.98ns | 0.0196 | 0 | 0 | 1.58 KB |
| #4927 | EnrichedLog |
net472 | 4.31μs | 1.68ns | 6.52ns | 0.311 | 0 | 0 | 1.97 KB |
Benchmarks.Trace.SpanBenchmark - Faster :tada: Same allocations :heavy_check_mark:
Faster :tada: in #4927
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0
1.121
603.40
538.39
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1
1.116
729.72
653.91
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartFinishSpan |
net6.0 | 464ns | 0.316ns | 1.22ns | 0.00741 | 0 | 0 | 536 B |
| master | StartFinishSpan |
netcoreapp3.1 | 730ns | 0.281ns | 1.05ns | 0.00729 | 0 | 0 | 536 B |
| master | StartFinishSpan |
net472 | 739ns | 0.245ns | 0.95ns | 0.0851 | 0 | 0 | 538 B |
| master | StartFinishScope |
net6.0 | 604ns | 0.222ns | 0.86ns | 0.00911 | 0 | 0 | 656 B |
| master | StartFinishScope |
netcoreapp3.1 | 799ns | 0.18ns | 0.65ns | 0.0089 | 0 | 0 | 656 B |
| master | StartFinishScope |
net472 | 974ns | 0.374ns | 1.45ns | 0.0977 | 0 | 0 | 618 B |
| #4927 | StartFinishSpan |
net6.0 | 511ns | 0.107ns | 0.4ns | 0.00745 | 0 | 0 | 536 B |
| #4927 | StartFinishSpan |
netcoreapp3.1 | 655ns | 0.445ns | 1.72ns | 0.00725 | 0 | 0 | 536 B |
| #4927 | StartFinishSpan |
net472 | 795ns | 2.66ns | 10.3ns | 0.0851 | 0 | 0 | 538 B |
| #4927 | StartFinishScope |
net6.0 | 538ns | 0.446ns | 1.73ns | 0.00924 | 0 | 0 | 656 B |
| #4927 | StartFinishScope |
netcoreapp3.1 | 855ns | 0.357ns | 1.29ns | 0.00901 | 0 | 0 | 656 B |
| #4927 | StartFinishScope |
net472 | 926ns | 0.712ns | 2.76ns | 0.0978 | 0 | 0 | 618 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunOnMethodBegin |
net6.0 | 628ns | 0.248ns | 0.961ns | 0.00945 | 0 | 0 | 656 B |
| master | RunOnMethodBegin |
netcoreapp3.1 | 935ns | 0.327ns | 1.23ns | 0.00889 | 0 | 0 | 656 B |
| master | RunOnMethodBegin |
net472 | 1.17μs | 0.704ns | 2.72ns | 0.0979 | 0 | 0 | 618 B |
| #4927 | RunOnMethodBegin |
net6.0 | 657ns | 0.288ns | 1.12ns | 0.00922 | 0 | 0 | 656 B |
| #4927 | RunOnMethodBegin |
netcoreapp3.1 | 961ns | 0.272ns | 1.05ns | 0.00866 | 0 | 0 | 656 B |
| #4927 | RunOnMethodBegin |
net472 | 1.1μs | 0.657ns | 2.55ns | 0.098 | 0 | 0 | 618 B |