[ASM][ApiSecurity] Change sampling algorithm
Open
anna-git
opened this issue 1 year ago
•
6 comments
Summary of changes
Following https://docs.google.com/document/d/1PYoHms9PPXR8V_5_T5-KXAhoFDKQYA8mTnmS12xkGOE/edit#heading=h.rnd972k0hiye
Change sampling algorithm for api security: take into account if:
last run of the waf (once the waf's context has all addresses in)
span priority (if not kept dont do anything)
30 seconds apart for each endpoint
OrderedDictionary for storing endpoints,
Reason for change
Take into account span priority, don't do extra work if priority is low.
Implementation details
Test coverage
Added unit tests to test api security class
Other integration tests should work the same (or contain more addresses as it's the end cycle)
Other details
Datadog Report
Branch report: anna/asm/apisecurity-new-sampling8d4eab8dd-trace-dotnet
:white_check_mark: 0 Failed, 330291 Passed, 1583 Skipped, 44m 14.4s Wall Time
:snowflake: 2 New Flaky
New Flaky Tests (2)
NoExceptions - Datadog.Trace.ClrProfiler.IntegrationTests.SmokeTests.EnumerateAssemblyReferencesTest - Last Failure
Expand for error
xpected exit code: 0, actual exit code: -1073740791.
ObfuscateWithDefaultPattern - Datadog.Trace.Tests.Util.Http.EvidenceRedactorTests - Last Failure
Expand for error
xpected string to be "key1=val1&<redacted>&key2=val2" with a length of 30, but "" has a length of 0, differs near "" (index 0).
Execution-Time Benchmarks Report :stopwatch:
Execution-time results for samples comparing the following branches/commits:
Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red . The following thresholds were used for comparing the execution times:
Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard .
Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).
gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5257) - mean (74ms) : 62, 86
. : milestone, 74,
master - mean (73ms) : 65, 81
. : milestone, 73,
section CallTarget+Inlining+NGEN
This PR (5257) - mean (1,003ms) : 982, 1023
. : milestone, 1003,
master - mean (1,004ms) : 981, 1026
. : milestone, 1004,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5257) - mean (110ms) : 106, 114
. : milestone, 110,
master - mean (111ms) : 108, 115
. : milestone, 111,
section CallTarget+Inlining+NGEN
This PR (5257) - mean (722ms) : 704, 741
. : milestone, 722,
master - mean (724ms) : 700, 749
. : milestone, 724,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5257) - mean (93ms) : 90, 97
. : milestone, 93,
master - mean (94ms) : 91, 97
. : milestone, 94,
section CallTarget+Inlining+NGEN
This PR (5257) - mean (674ms) : 658, 691
. : milestone, 674,
master - mean (679ms) : 656, 702
. : milestone, 679,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5257) - mean (187ms) : 183, 191
. : milestone, 187,
master - mean (188ms) : 183, 193
. : milestone, 188,
section CallTarget+Inlining+NGEN
This PR (5257) - mean (1,070ms) : 1045, 1096
. : milestone, 1070,
master - mean (1,071ms) : 1052, 1090
. : milestone, 1071,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5257) - mean (269ms) : 264, 275
. : milestone, 269,
master - mean (270ms) : 263, 276
. : milestone, 270,
section CallTarget+Inlining+NGEN
This PR (5257) - mean (863ms) : 837, 888
. : milestone, 863,
master - mean (866ms) : 844, 888
. : milestone, 866,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5257) - mean (258ms) : 253, 264
. : milestone, 258,
master - mean (260ms) : 254, 266
. : milestone, 260,
section CallTarget+Inlining+NGEN
This PR (5257) - mean (856ms) : 826, 887
. : milestone, 856,
master - mean (852ms) : 826, 877
. : milestone, 852,
Throughput/Crank Report:zap:
Throughput results for AspNetCoreSimpleController comparing the following branches/commits:
Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red .
Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many , many dashboards!
gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5257) (11.308M) : 0, 11307857
master (11.007M) : 0, 11006674
benchmarks/2.9.0 (11.269M) : 0, 11269489
section Automatic
This PR (5257) (7.873M) : 0, 7872792
master (7.556M) : 0, 7555579
benchmarks/2.9.0 (8.237M) : 0, 8236748
section Trace stats
This PR (5257) (8.232M) : 0, 8232163
master (7.855M) : 0, 7855147
section Manual
This PR (5257) (9.964M) : 0, 9963691
master (9.556M) : 0, 9555551
section Manual + Automatic
This PR (5257) (7.416M) : 0, 7416388
master (7.141M) : 0, 7140945
section Version Conflict
This PR (5257) (6.631M) : 0, 6630911
master (6.452M) : 0, 6451971
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5257) (9.700M) : 0, 9699920
master (9.496M) : 0, 9495623
benchmarks/2.9.0 (9.335M) : 0, 9334561
section Automatic
This PR (5257) (6.382M) : 0, 6382302
master (6.550M) : 0, 6550088
section Trace stats
This PR (5257) (6.845M) : 0, 6844549
master (6.930M) : 0, 6929825
section Manual
This PR (5257) (8.146M) : 0, 8145762
master (8.294M) : 0, 8293659
section Manual + Automatic
This PR (5257) (6.160M) : 0, 6160486
master (6.274M) : 0, 6273873
section Version Conflict
This PR (5257) (5.612M) : 0, 5612223
master (5.748M) : 0, 5748183
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5257) (10.603M) : 0, 10602800
master (10.120M) : 0, 10120352
benchmarks/2.9.0 (10.334M) : 0, 10333846
section Automatic
This PR (5257) (7.350M) : 0, 7349536
master (7.343M) : 0, 7343410
benchmarks/2.9.0 (7.377M) : 0, 7377326
section Trace stats
This PR (5257) (7.625M) : 0, 7624908
master (7.630M) : 0, 7629966
section Manual
This PR (5257) (9.310M) : 0, 9309567
master (8.884M) : 0, 8883719
section Manual + Automatic
This PR (5257) (7.127M) : 0, 7127159
master (6.989M) : 0, 6989119
section Version Conflict
This PR (5257) (6.495M) : 0, 6495100
master (6.322M) : 0, 6322239
gantt
title Throughput Linux x64 (ASM) (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5257) (7.578M) : 0, 7577996
master (7.454M) : 0, 7454237
benchmarks/2.9.0 (7.886M) : 0, 7885996
section No attack
This PR (5257) (1.870M) : 0, 1870454
master (1.841M) : 0, 1841297
benchmarks/2.9.0 (3.256M) : 0, 3255652
section Attack
This PR (5257) (1.455M) : 0, 1454868
master (1.454M) : 0, 1454025
benchmarks/2.9.0 (2.490M) : 0, 2489577
section Blocking
This PR (5257) (3.255M) : 0, 3255451
master (3.195M) : 0, 3194705
section IAST default
This PR (5257) (6.660M) : 0, 6659877
master (6.463M) : 0, 6462928
section IAST full
This PR (5257) (5.682M) : 0, 5682443
master (5.595M) : 0, 5594918
section Base vuln
This PR (5257) (0.942M) : 0, 942280
master (0.946M) : 0, 945501
section IAST vuln
This PR (5257) (0.879M) : 0, 879468
master (0.849M) : 0, 849226
Benchmarks Report :snail:
Benchmarks for #5257 compared to master :
2 benchmarks are slower , with geometric mean 1.180
2 benchmarks have fewer allocations
The following thresholds were used for comparing the benchmark speeds:
Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.
Allocation changes below 0.5% are ignored.
Benchmark details
Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master StartStopWithChildnet6.0
8.83μs
47.6ns
265ns
0.022
0.00881
0
7.49 KB
master StartStopWithChildnetcoreapp3.1
10.9μs
52.4ns
216ns
0.0223
0.0112
0
7.59 KB
master StartStopWithChildnet472
16.8μs
43.2ns
168ns
1.34
0.363
0.11
7.96 KB
#5257
StartStopWithChildnet6.0
9.15μs
46.3ns
202ns
0.0183
0.00916
0
7.49 KB
#5257
StartStopWithChildnetcoreapp3.1
10.7μs
58.3ns
350ns
0.0211
0.0106
0
7.59 KB
#5257
StartStopWithChildnet472
17.1μs
43.2ns
167ns
1.33
0.35
0.111
7.96 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master WriteAndFlushEnrichedTracesnet6.0
448μs
315ns
1.22μs
0
0
0
2.7 KB
master WriteAndFlushEnrichedTracesnetcoreapp3.1
612μs
255ns
989ns
0
0
0
2.7 KB
master WriteAndFlushEnrichedTracesnet472
822μs
559ns
2.16μs
0.411
0
0
3.3 KB
#5257
WriteAndFlushEnrichedTracesnet6.0
437μs
383ns
1.38μs
0
0
0
2.7 KB
#5257
WriteAndFlushEnrichedTracesnetcoreapp3.1
625μs
1.29μs
4.83μs
0
0
0
2.7 KB
#5257
WriteAndFlushEnrichedTracesnet472
819μs
563ns
2.18μs
0.406
0
0
3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower :warning: Same allocations :heavy_check_mark:
Slower :warning: in #5257
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1
1.185
204.58
242.32
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master AllCycleSimpleBodynet6.0
41μs
25.7ns
99.5ns
0.0203
0
0
2.36 KB
master AllCycleSimpleBodynetcoreapp3.1
44μs
61.1ns
237ns
0.0221
0
0
2.34 KB
master AllCycleSimpleBodynet472
46.9μs
17.1ns
66.4ns
0.373
0
0
2.41 KB
master AllCycleMoreComplexBodynet6.0
213μs
73.3ns
274ns
0.108
0
0
9.84 KB
master AllCycleMoreComplexBodynetcoreapp3.1
226μs
179ns
693ns
0.112
0
0
9.73 KB
master AllCycleMoreComplexBodynet472
241μs
84ns
314ns
1.57
0
0
9.91 KB
master ObjectExtractorSimpleBodynet6.0
141ns
0.0601ns
0.217ns
0.00392
0
0
280 B
master ObjectExtractorSimpleBodynetcoreapp3.1
205ns
0.126ns
0.486ns
0.00362
0
0
272 B
master ObjectExtractorSimpleBodynet472
172ns
0.0769ns
0.298ns
0.0446
0
0
281 B
master ObjectExtractorMoreComplexBodynet6.0
3.03μs
4.03ns
14.5ns
0.053
0
0
3.78 KB
master ObjectExtractorMoreComplexBodynetcoreapp3.1
3.93μs
2ns
7.76ns
0.0492
0
0
3.69 KB
master ObjectExtractorMoreComplexBodynet472
3.79μs
2.65ns
10.3ns
0.602
0.00569
0
3.8 KB
#5257
AllCycleSimpleBodynet6.0
40.9μs
37.5ns
145ns
0.0202
0
0
2.36 KB
#5257
AllCycleSimpleBodynetcoreapp3.1
44μs
32.2ns
125ns
0.0219
0
0
2.34 KB
#5257
AllCycleSimpleBodynet472
46.9μs
22.5ns
84.1ns
0.373
0
0
2.41 KB
#5257
AllCycleMoreComplexBodynet6.0
218μs
116ns
401ns
0.109
0
0
9.84 KB
#5257
AllCycleMoreComplexBodynetcoreapp3.1
226μs
110ns
426ns
0.113
0
0
9.73 KB
#5257
AllCycleMoreComplexBodynet472
237μs
119ns
461ns
1.54
0
0
9.91 KB
#5257
ObjectExtractorSimpleBodynet6.0
152ns
0.125ns
0.466ns
0.00397
0
0
280 B
#5257
ObjectExtractorSimpleBodynetcoreapp3.1
243ns
0.159ns
0.594ns
0.00378
0
0
272 B
#5257
ObjectExtractorSimpleBodynet472
164ns
0.0826ns
0.32ns
0.0446
0
0
281 B
#5257
ObjectExtractorMoreComplexBodynet6.0
2.93μs
2.59ns
9.69ns
0.0536
0
0
3.78 KB
#5257
ObjectExtractorMoreComplexBodynetcoreapp3.1
4.03μs
2.39ns
8.63ns
0.0484
0
0
3.69 KB
#5257
ObjectExtractorMoreComplexBodynet472
3.85μs
4.17ns
16.2ns
0.603
0.00576
0
3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master EncodeArgsnet6.0
32μs
21.7ns
84ns
0.397
0
0
27.94 KB
master EncodeArgsnetcoreapp3.1
46.6μs
19.4ns
72.7ns
0.373
0
0
27.94 KB
master EncodeArgsnet472
59.7μs
34.5ns
134ns
4.45
0.0597
0
28.03 KB
master EncodeLegacyArgsnet6.0
97.8μs
197ns
711ns
0.391
0
0
30.34 KB
master EncodeLegacyArgsnetcoreapp3.1
127μs
338ns
1.31μs
0.37
0
0
30.34 KB
master EncodeLegacyArgsnet472
186μs
264ns
1.02μs
4.81
0.363
0
30.44 KB
#5257
EncodeArgsnet6.0
32.9μs
27.1ns
105ns
0.394
0
0
27.94 KB
#5257
EncodeArgsnetcoreapp3.1
45.2μs
26.5ns
103ns
0.384
0
0
27.94 KB
#5257
EncodeArgsnet472
59.9μs
21.1ns
81.9ns
4.43
0.0599
0
28.03 KB
#5257
EncodeLegacyArgsnet6.0
95.6μs
90.2ns
349ns
0.397
0
0
30.34 KB
#5257
EncodeLegacyArgsnetcoreapp3.1
128μs
394ns
1.53μs
0.389
0
0
30.34 KB
#5257
EncodeLegacyArgsnet472
181μs
77.3ns
299ns
4.78
0.368
0
30.44 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master RunWafRealisticBenchmarknet6.0
188μs
184ns
665ns
0.0936
0
0
6.51 KB
master RunWafRealisticBenchmarknetcoreapp3.1
204μs
369ns
1.43μs
0
0
0
6.49 KB
master RunWafRealisticBenchmarknet472
223μs
149ns
578ns
1.01
0
0
6.59 KB
master RunWafRealisticBenchmarkWithAttacknet6.0
125μs
67.9ns
263ns
0.0631
0
0
4.15 KB
master RunWafRealisticBenchmarkWithAttacknetcoreapp3.1
134μs
279ns
1.08μs
0
0
0
4.14 KB
master RunWafRealisticBenchmarkWithAttacknet472
147μs
39.7ns
149ns
0.657
0
0
4.19 KB
#5257
RunWafRealisticBenchmarknet6.0
192μs
491ns
1.9μs
0.0944
0
0
6.51 KB
#5257
RunWafRealisticBenchmarknetcoreapp3.1
205μs
461ns
1.78μs
0
0
0
6.49 KB
#5257
RunWafRealisticBenchmarknet472
224μs
153ns
572ns
1.01
0
0
6.59 KB
#5257
RunWafRealisticBenchmarkWithAttacknet6.0
126μs
78.7ns
305ns
0.0628
0
0
4.15 KB
#5257
RunWafRealisticBenchmarkWithAttacknetcoreapp3.1
135μs
193ns
749ns
0
0
0
4.15 KB
#5257
RunWafRealisticBenchmarkWithAttacknet472
147μs
71.8ns
278ns
0.662
0
0
4.19 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master SendRequestnet6.0
170μs
170ns
635ns
0.255
0
0
18.26 KB
master SendRequestnetcoreapp3.1
189μs
185ns
717ns
0.189
0
0
20.42 KB
master SendRequestnet472
0.000528ns
0.000196ns
0.000735ns
0
0
0
0 b
#5257
SendRequestnet6.0
173μs
127ns
490ns
0.172
0
0
18.26 KB
#5257
SendRequestnetcoreapp3.1
190μs
384ns
1.49μs
0.19
0
0
20.42 KB
#5257
SendRequestnet472
0.000417ns
0.000167ns
0.000623ns
0
0
0
0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master WriteAndFlushEnrichedTracesnet6.0
534μs
201ns
724ns
0.563
0
0
41.42 KB
master WriteAndFlushEnrichedTracesnetcoreapp3.1
653μs
864ns
3.35μs
0.322
0
0
41.92 KB
master WriteAndFlushEnrichedTracesnet472
849μs
3.34μs
12.9μs
8.45
2.53
0.422
53.24 KB
#5257
WriteAndFlushEnrichedTracesnet6.0
531μs
2μs
7.73μs
0.532
0
0
41.46 KB
#5257
WriteAndFlushEnrichedTracesnetcoreapp3.1
668μs
1.21μs
4.7μs
0.332
0
0
41.85 KB
#5257
WriteAndFlushEnrichedTracesnet472
845μs
2.63μs
9.82μs
8.33
2.5
0.417
53.28 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master ExecuteNonQuerynet6.0
1.02μs
0.851ns
3.3ns
0.0107
0
0
776 B
master ExecuteNonQuerynetcoreapp3.1
1.47μs
1.45ns
5.43ns
0.0103
0
0
776 B
master ExecuteNonQuerynet472
1.73μs
0.953ns
3.69ns
0.117
0
0
738 B
#5257
ExecuteNonQuerynet6.0
1.12μs
1.39ns
5.37ns
0.0111
0
0
776 B
#5257
ExecuteNonQuerynetcoreapp3.1
1.49μs
0.733ns
2.84ns
0.0105
0
0
776 B
#5257
ExecuteNonQuerynet472
1.74μs
4.07ns
15.7ns
0.117
0
0
738 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master CallElasticsearchnet6.0
1.14μs
0.398ns
1.49ns
0.0132
0
0
944 B
master CallElasticsearchnetcoreapp3.1
1.46μs
0.551ns
2.13ns
0.013
0
0
944 B
master CallElasticsearchnet472
2.45μs
1.14ns
4.41ns
0.152
0
0
963 B
master CallElasticsearchAsyncnet6.0
1.32μs
0.736ns
2.85ns
0.0132
0
0
920 B
master CallElasticsearchAsyncnetcoreapp3.1
1.61μs
0.524ns
1.96ns
0.013
0
0
992 B
master CallElasticsearchAsyncnet472
2.63μs
0.712ns
2.76ns
0.161
0
0
1.02 KB
#5257
CallElasticsearchnet6.0
1.16μs
0.476ns
1.78ns
0.0133
0
0
944 B
#5257
CallElasticsearchnetcoreapp3.1
1.5μs
0.697ns
2.51ns
0.0124
0
0
944 B
#5257
CallElasticsearchnet472
2.45μs
1.47ns
5.68ns
0.152
0
0
963 B
#5257
CallElasticsearchAsyncnet6.0
1.26μs
0.679ns
2.54ns
0.0132
0
0
920 B
#5257
CallElasticsearchAsyncnetcoreapp3.1
1.65μs
0.835ns
3.12ns
0.0132
0
0
992 B
#5257
CallElasticsearchAsyncnet472
2.55μs
1.52ns
5.48ns
0.162
0
0
1.02 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master ExecuteAsyncnet6.0
1.29μs
2.03ns
7.3ns
0.013
0
0
920 B
master ExecuteAsyncnetcoreapp3.1
1.68μs
3.9ns
14.1ns
0.0123
0
0
920 B
master ExecuteAsyncnet472
1.85μs
0.732ns
2.83ns
0.14
0
0
883 B
#5257
ExecuteAsyncnet6.0
1.21μs
1.05ns
4.05ns
0.0132
0
0
920 B
#5257
ExecuteAsyncnetcoreapp3.1
1.62μs
1.35ns
5.07ns
0.0123
0
0
920 B
#5257
ExecuteAsyncnet472
1.82μs
0.625ns
2.34ns
0.14
0
0
883 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master SendAsyncnet6.0
4.16μs
1.35ns
5.25ns
0.0291
0
0
2.1 KB
master SendAsyncnetcoreapp3.1
5μs
3.26ns
12.6ns
0.035
0
0
2.64 KB
master SendAsyncnet472
7.61μs
3.14ns
12.2ns
0.524
0
0
3.31 KB
#5257
SendAsyncnet6.0
4.06μs
1.03ns
3.85ns
0.0286
0
0
2.1 KB
#5257
SendAsyncnetcoreapp3.1
5.15μs
1.71ns
6.39ns
0.0359
0
0
2.64 KB
#5257
SendAsyncnet472
7.81μs
0.888ns
3.44ns
0.524
0
0
3.31 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:
Fewer allocations :tada: in #5257
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472
62.46 KB
62.11 KB
-352 B
-0.56%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0
215.6 KB
205.04 KB
-10.56 KB
-4.90%
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master StringConcatBenchmarknet6.0
50.8μs
164ns
615ns
0
0
0
43.44 KB
master StringConcatBenchmarknetcoreapp3.1
52μs
183ns
659ns
0
0
0
42.64 KB
master StringConcatBenchmarknet472
37.4μs
87.3ns
315ns
0
0
0
62.46 KB
master StringConcatAspectBenchmarknet6.0
277μs
1.39μs
9.35μs
0
0
0
215.6 KB
master StringConcatAspectBenchmarknetcoreapp3.1
277μs
1.32μs
8.15μs
0
0
0
202.2 KB
master StringConcatAspectBenchmarknet472
233μs
2.59μs
24.7μs
0
0
0
221.18 KB
#5257
StringConcatBenchmarknet6.0
57μs
715ns
7.11μs
0
0
0
43.44 KB
#5257
StringConcatBenchmarknetcoreapp3.1
52.6μs
221ns
796ns
0
0
0
42.64 KB
#5257
StringConcatBenchmarknet472
37.2μs
121ns
453ns
0
0
0
62.11 KB
#5257
StringConcatAspectBenchmarknet6.0
277μs
1.14μs
3.95μs
0
0
0
205.04 KB
#5257
StringConcatAspectBenchmarknetcoreapp3.1
277μs
950ns
5.54μs
0
0
0
203.07 KB
#5257
StringConcatAspectBenchmarknet472
241μs
3.82μs
37.5μs
0
0
0
221.18 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master EnrichedLognet6.0
1.4μs
0.663ns
2.48ns
0.0218
0
0
1.58 KB
master EnrichedLognetcoreapp3.1
2.18μs
0.583ns
2.1ns
0.0217
0
0
1.58 KB
master EnrichedLognet472
2.66μs
1.57ns
5.89ns
0.239
0
0
1.51 KB
#5257
EnrichedLognet6.0
1.5μs
0.97ns
3.5ns
0.0222
0
0
1.58 KB
#5257
EnrichedLognetcoreapp3.1
2.19μs
0.878ns
3.04ns
0.0207
0
0
1.58 KB
#5257
EnrichedLognet472
2.54μs
0.982ns
3.8ns
0.239
0
0
1.51 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master EnrichedLognet6.0
113μs
184ns
713ns
0.0565
0
0
4.22 KB
master EnrichedLognetcoreapp3.1
119μs
152ns
590ns
0.0591
0
0
4.22 KB
master EnrichedLognet472
148μs
199ns
771ns
0.663
0.221
0
4.4 KB
#5257
EnrichedLognet6.0
114μs
123ns
476ns
0.057
0
0
4.22 KB
#5257
EnrichedLognetcoreapp3.1
119μs
126ns
486ns
0
0
0
4.22 KB
#5257
EnrichedLognet472
149μs
124ns
481ns
0.662
0.221
0
4.4 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master EnrichedLognet6.0
2.95μs
8.8ns
32.9ns
0.0292
0
0
2.14 KB
master EnrichedLognetcoreapp3.1
4.05μs
2.42ns
9.06ns
0.0284
0
0
2.14 KB
master EnrichedLognet472
4.72μs
5.48ns
21.2ns
0.308
0
0
1.95 KB
#5257
EnrichedLognet6.0
2.88μs
0.775ns
2.9ns
0.0303
0
0
2.14 KB
#5257
EnrichedLognetcoreapp3.1
4.1μs
1.39ns
5.22ns
0.0289
0
0
2.14 KB
#5257
EnrichedLognet472
4.79μs
1.01ns
3.77ns
0.311
0
0
1.95 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master SendReceivenet6.0
1.38μs
0.856ns
3.31ns
0.0158
0
0
1.11 KB
master SendReceivenetcoreapp3.1
1.75μs
1.06ns
3.96ns
0.0157
0
0
1.11 KB
master SendReceivenet472
1.98μs
0.765ns
2.86ns
0.178
0
0
1.12 KB
#5257
SendReceivenet6.0
1.39μs
0.96ns
3.72ns
0.0153
0
0
1.11 KB
#5257
SendReceivenetcoreapp3.1
1.71μs
1.16ns
4.49ns
0.0154
0
0
1.11 KB
#5257
SendReceivenet472
2.1μs
1.33ns
5.16ns
0.178
0
0
1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master EnrichedLognet6.0
2.9μs
0.839ns
3.25ns
0.0209
0
0
1.54 KB
master EnrichedLognetcoreapp3.1
3.74μs
1.47ns
5.52ns
0.0208
0
0
1.58 KB
master EnrichedLognet472
4.33μs
1.85ns
7.15ns
0.313
0
0
1.97 KB
#5257
EnrichedLognet6.0
2.69μs
0.949ns
3.68ns
0.0213
0
0
1.54 KB
#5257
EnrichedLognetcoreapp3.1
3.92μs
1.41ns
5.48ns
0.0216
0
0
1.58 KB
#5257
EnrichedLognet472
4.21μs
4.1ns
15.9ns
0.312
0
0
1.97 KB
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark:
Slower :warning: in #5257
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1
1.175
772.00
906.84
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master StartFinishSpannet6.0
461ns
0.827ns
3.2ns
0.00759
0
0
544 B
master StartFinishSpannetcoreapp3.1
724ns
1.98ns
7.68ns
0.00747
0
0
544 B
master StartFinishSpannet472
723ns
1.66ns
6.42ns
0.0866
0
0
546 B
master StartFinishScopenet6.0
597ns
0.85ns
3.29ns
0.00927
0
0
664 B
master StartFinishScopenetcoreapp3.1
773ns
1.91ns
7.41ns
0.00881
0
0
664 B
master StartFinishScopenet472
987ns
1.82ns
7.05ns
0.0992
0
0
626 B
#5257
StartFinishSpannet6.0
466ns
0.391ns
1.51ns
0.00773
0
0
544 B
#5257
StartFinishSpannetcoreapp3.1
718ns
0.689ns
2.67ns
0.00739
0
0
544 B
#5257
StartFinishSpannet472
793ns
0.793ns
3.07ns
0.0864
0
0
546 B
#5257
StartFinishScopenet6.0
607ns
0.287ns
1.11ns
0.00934
0
0
664 B
#5257
StartFinishScopenetcoreapp3.1
907ns
0.866ns
3.36ns
0.00894
0
0
664 B
#5257
StartFinishScopenet472
928ns
0.897ns
3.47ns
0.0991
0
0
626 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master RunOnMethodBeginnet6.0
632ns
1.01ns
3.91ns
0.00927
0
0
664 B
master RunOnMethodBeginnetcoreapp3.1
1.03μs
1.38ns
5.34ns
0.00878
0
0
664 B
master RunOnMethodBeginnet472
1.08μs
2.75ns
10.7ns
0.0995
0
0
626 B
#5257
RunOnMethodBeginnet6.0
626ns
0.388ns
1.45ns
0.00929
0
0
664 B
#5257
RunOnMethodBeginnetcoreapp3.1
1μs
0.477ns
1.78ns
0.00894
0
0
664 B
#5257
RunOnMethodBeginnet472
1.1μs
0.53ns
2.05ns
0.0992
0
0
626 B
Snapshots difference summary
The following differences have been observed in committed snapshots. It is meant to help the reviewer.
The diff is simplistic, so please check some files anyway while we improve it.
1 occurrences of :
- _dd.appsec.s.req.params: [{"id":[4]}],
+ _dd.appsec.s.req.params: [{"MS_SubRoutes":[[[{}]],{"len":1}]}],
[...]
+ _dd.appsec.s.res.headers: [{"cache-control":[[[8]],{"len":1}],"content-length":[[[8]],{"len":1}],"content-type":[[[8]],{"len":1}],"expires":[[[8]],{"len":1}],"pragma":[[[8]],{"len":1}],"server":[[[8]],{"len":1}],"x-aspnet-version":[[[8]],{"len":1}]}],
1 occurrences of :
- _dd.appsec.s.req.params: [{"id":[4]}],
+ _dd.appsec.s.req.params: [{"MS_SubRoutes":[[[{}]],{"len":1}]}],
4 occurrences of :
- http.url: http://localhost:00000/home/apisecurity/12,
+ http.url: http://localhost:00000/home/apisecurity/11,
1 occurrences of :
- _dd.appsec.s.req.params: [{"id":[4]}],
+ _dd.appsec.s.req.params: [{"action":[8],"controller":[8],"id":[8]}],
[...]
+ _dd.appsec.s.res.headers: [{"cache-control":[[[8]],{"len":1}],"content-type":[[[8]],{"len":1}],"x-aspnet-version":[[[8]],{"len":1}]}],
1 occurrences of :
- _dd.appsec.s.req.params: [{"id":[4]}],
+ _dd.appsec.s.req.params: [{"action":[8],"controller":[8],"id":[8]}],
Benchmarks Report for tracer :snail:
Benchmarks for #5257 compared to master :
2 benchmarks are faster , with geometric mean 1.222
1 benchmarks are slower , with geometric mean 1.161
All benchmarks have the same allocations
The following thresholds were used for comparing the benchmark speeds:
Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.
Allocation changes below 0.5% are ignored.
Benchmark details
Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master StartStopWithChildnet6.0
8.78μs
49ns
325ns
0.0176
0.0088
0
7.51 KB
master StartStopWithChildnetcoreapp3.1
10.7μs
60.3ns
418ns
0.031
0.0155
0
7.6 KB
master StartStopWithChildnet472
16.8μs
61.5ns
238ns
1.32
0.301
0.0835
8.04 KB
#5257
StartStopWithChildnet6.0
8.57μs
47.2ns
275ns
0.0212
0.00846
0
7.51 KB
#5257
StartStopWithChildnetcoreapp3.1
10.6μs
59ns
405ns
0.0254
0.0102
0
7.6 KB
#5257
StartStopWithChildnet472
16.5μs
53.6ns
208ns
1.34
0.328
0.106
8.07 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master WriteAndFlushEnrichedTracesnet6.0
463μs
196ns
678ns
0
0
0
2.7 KB
master WriteAndFlushEnrichedTracesnetcoreapp3.1
616μs
161ns
625ns
0
0
0
2.7 KB
master WriteAndFlushEnrichedTracesnet472
804μs
274ns
1.06μs
0.401
0
0
3.3 KB
#5257
WriteAndFlushEnrichedTracesnet6.0
453μs
141ns
546ns
0
0
0
2.7 KB
#5257
WriteAndFlushEnrichedTracesnetcoreapp3.1
601μs
173ns
646ns
0
0
0
2.7 KB
#5257
WriteAndFlushEnrichedTracesnet472
820μs
205ns
794ns
0.408
0
0
3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master SendRequestnet6.0
171μs
154ns
597ns
0.184
0
0
18.33 KB
master SendRequestnetcoreapp3.1
193μs
240ns
928ns
0.193
0
0
20.49 KB
master SendRequestnet472
0.000852ns
0.000448ns
0.00161ns
0
0
0
0 b
#5257
SendRequestnet6.0
171μs
167ns
647ns
0.171
0
0
18.33 KB
#5257
SendRequestnetcoreapp3.1
195μs
292ns
1.13μs
0.195
0
0
20.49 KB
#5257
SendRequestnet472
0.000293ns
0.000156ns
0.000606ns
0
0
0
0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master WriteAndFlushEnrichedTracesnet6.0
546μs
983ns
3.81μs
0.561
0
0
41.48 KB
master WriteAndFlushEnrichedTracesnetcoreapp3.1
665μs
1.34μs
5.2μs
0.329
0
0
41.69 KB
master WriteAndFlushEnrichedTracesnet472
870μs
3.69μs
14.3μs
8.13
2.57
0.428
53.25 KB
#5257
WriteAndFlushEnrichedTracesnet6.0
543μs
379ns
1.37μs
0.541
0
0
41.63 KB
#5257
WriteAndFlushEnrichedTracesnetcoreapp3.1
649μs
1.33μs
5.15μs
0.319
0
0
41.73 KB
#5257
WriteAndFlushEnrichedTracesnet472
853μs
4.42μs
21.2μs
8.17
2.45
0.408
53.24 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master ExecuteNonQuerynet6.0
1.17μs
0.623ns
2.41ns
0.0111
0
0
784 B
master ExecuteNonQuerynetcoreapp3.1
1.58μs
5.33ns
20.6ns
0.0103
0
0
784 B
master ExecuteNonQuerynet472
1.89μs
0.577ns
2.16ns
0.118
0
0
746 B
#5257
ExecuteNonQuerynet6.0
1.13μs
0.853ns
3.3ns
0.0108
0
0
784 B
#5257
ExecuteNonQuerynetcoreapp3.1
1.56μs
0.788ns
2.95ns
0.01
0
0
784 B
#5257
ExecuteNonQuerynet472
1.84μs
0.53ns
2.05ns
0.119
0
0
746 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master CallElasticsearchnet6.0
1.2μs
0.599ns
2.24ns
0.0138
0
0
1.01 KB
master CallElasticsearchnetcoreapp3.1
1.62μs
0.769ns
2.98ns
0.0138
0
0
1.01 KB
master CallElasticsearchnet472
2.58μs
2.45ns
9.47ns
0.162
0
0
1.02 KB
master CallElasticsearchAsyncnet6.0
1.47μs
1.39ns
5.19ns
0.0138
0
0
984 B
master CallElasticsearchAsyncnetcoreapp3.1
1.74μs
0.684ns
2.56ns
0.0138
0
0
1.06 KB
master CallElasticsearchAsyncnet472
2.56μs
1.96ns
7.59ns
0.17
0
0
1.08 KB
#5257
CallElasticsearchnet6.0
1.31μs
0.827ns
3.2ns
0.0138
0
0
1.01 KB
#5257
CallElasticsearchnetcoreapp3.1
1.59μs
0.969ns
3.62ns
0.0134
0
0
1.01 KB
#5257
CallElasticsearchnet472
2.47μs
1.08ns
4.04ns
0.161
0
0
1.02 KB
#5257
CallElasticsearchAsyncnet6.0
1.36μs
0.745ns
2.79ns
0.0135
0
0
984 B
#5257
CallElasticsearchAsyncnetcoreapp3.1
1.75μs
6.51ns
25.2ns
0.0139
0
0
1.06 KB
#5257
CallElasticsearchAsyncnet472
2.59μs
2.91ns
11.3ns
0.171
0
0
1.08 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master ExecuteAsyncnet6.0
1.26μs
1.21ns
4.51ns
0.0127
0
0
928 B
master ExecuteAsyncnetcoreapp3.1
1.63μs
0.751ns
2.91ns
0.013
0
0
928 B
master ExecuteAsyncnet472
1.9μs
0.575ns
2.15ns
0.141
0
0
891 B
#5257
ExecuteAsyncnet6.0
1.24μs
0.666ns
2.49ns
0.0129
0
0
928 B
#5257
ExecuteAsyncnetcoreapp3.1
1.62μs
2.14ns
8.28ns
0.0121
0
0
928 B
#5257
ExecuteAsyncnet472
1.99μs
1.64ns
6.37ns
0.141
0.000985
0
891 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master SendAsyncnet6.0
4.24μs
1.91ns
7.14ns
0.0298
0
0
2.16 KB
master SendAsyncnetcoreapp3.1
5.06μs
2.59ns
10ns
0.0352
0
0
2.7 KB
master SendAsyncnet472
7.61μs
4.46ns
16.1ns
0.484
0
0
3.05 KB
#5257
SendAsyncnet6.0
4.12μs
2.1ns
8.14ns
0.0308
0
0
2.16 KB
#5257
SendAsyncnetcoreapp3.1
4.98μs
1.99ns
7.44ns
0.0349
0
0
2.7 KB
#5257
SendAsyncnet472
7.71μs
3.02ns
11.7ns
0.482
0
0
3.05 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master EnrichedLognet6.0
1.46μs
0.675ns
2.62ns
0.0228
0
0
1.65 KB
master EnrichedLognetcoreapp3.1
2.19μs
0.863ns
3.23ns
0.0227
0
0
1.65 KB
master EnrichedLognet472
2.68μs
2.41ns
9.35ns
0.249
0
0
1.57 KB
#5257
EnrichedLognet6.0
1.57μs
0.974ns
3.64ns
0.0234
0
0
1.65 KB
#5257
EnrichedLognetcoreapp3.1
2.27μs
1.82ns
6.56ns
0.0219
0
0
1.65 KB
#5257
EnrichedLognet472
2.55μs
2.3ns
8.9ns
0.25
0
0
1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master EnrichedLognet6.0
111μs
77.8ns
301ns
0.0556
0
0
4.23 KB
master EnrichedLognetcoreapp3.1
119μs
287ns
1.11μs
0.0591
0
0
4.23 KB
master EnrichedLognet472
148μs
151ns
566ns
0.665
0.222
0
4.41 KB
#5257
EnrichedLognet6.0
113μs
115ns
445ns
0.0563
0
0
4.23 KB
#5257
EnrichedLognetcoreapp3.1
117μs
273ns
1.02μs
0.0578
0
0
4.23 KB
#5257
EnrichedLognet472
147μs
195ns
756ns
0.663
0.221
0
4.41 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master EnrichedLognet6.0
3.02μs
1.21ns
4.35ns
0.0316
0
0
2.21 KB
master EnrichedLognetcoreapp3.1
4.15μs
2.14ns
8.27ns
0.0288
0
0
2.21 KB
master EnrichedLognet472
4.82μs
3.46ns
13.4ns
0.321
0
0
2.02 KB
#5257
EnrichedLognet6.0
3.11μs
1.33ns
4.98ns
0.0311
0
0
2.21 KB
#5257
EnrichedLognetcoreapp3.1
4.37μs
2.27ns
8.48ns
0.0286
0
0
2.21 KB
#5257
EnrichedLognet472
4.93μs
2.67ns
9.61ns
0.321
0
0
2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master SendReceivenet6.0
1.45μs
0.621ns
2.24ns
0.0167
0
0
1.18 KB
master SendReceivenetcoreapp3.1
1.79μs
0.434ns
1.56ns
0.016
0
0
1.18 KB
master SendReceivenet472
2.18μs
1.17ns
4.55ns
0.187
0
0
1.18 KB
#5257
SendReceivenet6.0
1.49μs
1.05ns
4.06ns
0.0164
0
0
1.18 KB
#5257
SendReceivenetcoreapp3.1
1.82μs
1.14ns
4.4ns
0.0154
0
0
1.18 KB
#5257
SendReceivenet472
2.31μs
1.92ns
7.42ns
0.186
0
0
1.18 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master EnrichedLognet6.0
2.76μs
0.804ns
2.78ns
0.0221
0
0
1.55 KB
master EnrichedLognetcoreapp3.1
3.86μs
2.09ns
8.1ns
0.0214
0
0
1.6 KB
master EnrichedLognet472
4.51μs
3.37ns
13.1ns
0.315
0
0
1.99 KB
#5257
EnrichedLognet6.0
2.85μs
0.501ns
1.81ns
0.0213
0
0
1.55 KB
#5257
EnrichedLognetcoreapp3.1
3.76μs
1.16ns
4.32ns
0.0207
0
0
1.6 KB
#5257
EnrichedLognet472
4.35μs
1.76ns
6.8ns
0.316
0
0
1.99 KB
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark:
Slower :warning: in #5257
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1
1.161
684.89
795.04
Faster :tada: in #5257
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0
1.223
685.08
559.94
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master StartFinishSpannet6.0
464ns
0.309ns
1.16ns
0.00761
0
0
552 B
master StartFinishSpannetcoreapp3.1
685ns
0.576ns
2.16ns
0.0072
0
0
552 B
master StartFinishSpannet472
803ns
1.01ns
3.64ns
0.0879
0
0
554 B
master StartFinishScopenet6.0
685ns
0.886ns
3.43ns
0.00959
0
0
672 B
master StartFinishScopenetcoreapp3.1
872ns
0.722ns
2.8ns
0.00878
0
0
672 B
master StartFinishScopenet472
977ns
1.28ns
4.95ns
0.1
0
0
634 B
#5257
StartFinishSpannet6.0
505ns
0.548ns
2.12ns
0.0076
0
0
552 B
#5257
StartFinishSpannetcoreapp3.1
796ns
1.04ns
3.88ns
0.00727
0
0
552 B
#5257
StartFinishSpannet472
768ns
1.16ns
4.49ns
0.0878
0
0
554 B
#5257
StartFinishScopenet6.0
560ns
0.556ns
2.15ns
0.00953
0
0
672 B
#5257
StartFinishScopenetcoreapp3.1
904ns
0.901ns
3.49ns
0.00892
0
0
672 B
#5257
StartFinishScopenet472
910ns
1.24ns
4.81ns
0.101
0
0
634 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Faster :tada: Same allocations :heavy_check_mark:
Faster :tada: in #5257
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0
1.220
749.78
614.39
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master RunOnMethodBeginnet6.0
750ns
0.672ns
2.6ns
0.00945
0
0
672 B
master RunOnMethodBeginnetcoreapp3.1
943ns
0.716ns
2.58ns
0.00918
0
0
672 B
master RunOnMethodBeginnet472
1.1μs
1.29ns
4.99ns
0.101
0
0
634 B
#5257
RunOnMethodBeginnet6.0
614ns
1.03ns
3.97ns
0.00947
0
0
672 B
#5257
RunOnMethodBeginnetcoreapp3.1
910ns
1.42ns
5.32ns
0.00891
0
0
672 B
#5257
RunOnMethodBeginnet472
1.1μs
1.64ns
6.35ns
0.1
0
0
634 B
Benchmarks Report for appsec :snail:
Benchmarks for #5257 compared to master :
2 benchmarks are faster , with geometric mean 1.216
7 benchmarks are slower , with geometric mean 22.676
2 benchmarks have fewer allocations
6 benchmarks have more allocations
The following thresholds were used for comparing the benchmark speeds:
Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.
Allocation changes below 0.5% are ignored.
Benchmark details
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower :warning: More allocations :warning:
Slower :warning: in #5257
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net6.0
122.796
578.71
71,063.27
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑netcoreapp3.1
71.572
857.73
61,389.16
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net472
69.744
689.81
48,110.33
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net6.0
21.774
3,509.06
76,407.95
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑netcoreapp3.1
14.598
4,764.69
69,553.29
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net472
12.768
4,328.79
55,268.29
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑net472
1.239
174.60
216.42
Faster :tada: in #5257
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1
1.316
255.94
194.42
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody‑net472
1.124
4,201.90
3,738.25
More allocations :warning: in #5257
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net472
1.1 KB
8.3 KB
7.2 KB
655.51%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑netcoreapp3.1
1.06 KB
6.92 KB
5.85 KB
550.09%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net6.0
1.08 KB
5.97 KB
4.89 KB
453.15%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net472
4.61 KB
11.82 KB
7.2 KB
156.09%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑netcoreapp3.1
4.48 KB
10.33 KB
5.85 KB
130.60%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net6.0
4.58 KB
9.48 KB
4.89 KB
106.72%
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master AllCycleSimpleBodynet6.0
578ns
0.422ns
1.63ns
0.0153
0
0
1.08 KB
master AllCycleSimpleBodynetcoreapp3.1
858ns
0.494ns
1.91ns
0.0146
0
0
1.06 KB
master AllCycleSimpleBodynet472
690ns
0.214ns
0.802ns
0.174
0.000347
0
1.1 KB
master AllCycleMoreComplexBodynet6.0
3.51μs
2.26ns
8.77ns
0.065
0
0
4.58 KB
master AllCycleMoreComplexBodynetcoreapp3.1
4.76μs
2.38ns
9.2ns
0.0597
0
0
4.48 KB
master AllCycleMoreComplexBodynet472
4.33μs
4.21ns
16.3ns
0.732
0.00864
0
4.61 KB
master ObjectExtractorSimpleBodynet6.0
142ns
0.102ns
0.394ns
0.00393
0
0
280 B
master ObjectExtractorSimpleBodynetcoreapp3.1
256ns
0.131ns
0.509ns
0.00371
0
0
272 B
master ObjectExtractorSimpleBodynet472
174ns
0.263ns
1.02ns
0.0446
0
0
281 B
master ObjectExtractorMoreComplexBodynet6.0
3.08μs
1.26ns
4.72ns
0.0526
0
0
3.78 KB
master ObjectExtractorMoreComplexBodynetcoreapp3.1
4.04μs
2.63ns
10.2ns
0.0503
0
0
3.69 KB
master ObjectExtractorMoreComplexBodynet472
4.2μs
2.86ns
10.7ns
0.602
0.00627
0
3.8 KB
#5257
AllCycleSimpleBodynet6.0
71.1μs
114ns
411ns
0.0713
0
0
5.97 KB
#5257
AllCycleSimpleBodynetcoreapp3.1
61.4μs
43.9ns
164ns
0.0916
0
0
6.92 KB
#5257
AllCycleSimpleBodynet472
48.2μs
58ns
225ns
1.3
0
0
8.3 KB
#5257
AllCycleMoreComplexBodynet6.0
76.5μs
124ns
430ns
0.115
0
0
9.48 KB
#5257
AllCycleMoreComplexBodynetcoreapp3.1
69.5μs
77.7ns
301ns
0.139
0
0
10.33 KB
#5257
AllCycleMoreComplexBodynet472
55.3μs
58.2ns
225ns
1.85
0.0276
0
11.82 KB
#5257
ObjectExtractorSimpleBodynet6.0
139ns
0.151ns
0.584ns
0.00393
0
0
280 B
#5257
ObjectExtractorSimpleBodynetcoreapp3.1
194ns
0.0733ns
0.274ns
0.00362
0
0
272 B
#5257
ObjectExtractorSimpleBodynet472
216ns
0.0826ns
0.32ns
0.0445
0
0
281 B
#5257
ObjectExtractorMoreComplexBodynet6.0
3.11μs
1.32ns
4.92ns
0.0531
0
0
3.78 KB
#5257
ObjectExtractorMoreComplexBodynetcoreapp3.1
3.94μs
1.86ns
7.22ns
0.0493
0
0
3.69 KB
#5257
ObjectExtractorMoreComplexBodynet472
3.74μs
2.15ns
8.32ns
0.602
0.00559
0
3.8 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master RunWafRealisticBenchmarknet6.0
187μs
82.7ns
320ns
0
0
0
6.51 KB
master RunWafRealisticBenchmarknetcoreapp3.1
205μs
687ns
2.57μs
0
0
0
6.49 KB
master RunWafRealisticBenchmarknet472
222μs
58.4ns
202ns
1.02
0
0
6.59 KB
master RunWafRealisticBenchmarkWithAttacknet6.0
125μs
331ns
1.28μs
0
0
0
4.15 KB
master RunWafRealisticBenchmarkWithAttacknetcoreapp3.1
133μs
144ns
498ns
0
0
0
4.14 KB
master RunWafRealisticBenchmarkWithAttacknet472
147μs
125ns
468ns
0.656
0
0
4.19 KB
#5257
RunWafRealisticBenchmarknet6.0
189μs
192ns
744ns
0.0948
0
0
6.51 KB
#5257
RunWafRealisticBenchmarknetcoreapp3.1
203μs
271ns
1.05μs
0
0
0
6.49 KB
#5257
RunWafRealisticBenchmarknet472
224μs
155ns
599ns
0.997
0
0
6.59 KB
#5257
RunWafRealisticBenchmarkWithAttacknet6.0
124μs
117ns
406ns
0.0629
0
0
4.15 KB
#5257
RunWafRealisticBenchmarkWithAttacknetcoreapp3.1
133μs
143ns
553ns
0
0
0
4.15 KB
#5257
RunWafRealisticBenchmarkWithAttacknet472
146μs
163ns
630ns
0.654
0
0
4.19 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:
Fewer allocations :tada: in #5257
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472
62.2 KB
61.77 KB
-432 B
-0.69%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0
205.27 KB
202.47 KB
-2.8 KB
-1.36%
Raw results
Branch
Method
Toolchain
Mean
StdError
StdDev
Gen 0
Gen 1
Gen 2
Allocated
master StringConcatBenchmarknet6.0
60.1μs
656ns
6.4μs
0
0
0
43.44 KB
master StringConcatBenchmarknetcoreapp3.1
61.7μs
894ns
8.76μs
0
0
0
42.64 KB
master StringConcatBenchmarknet472
37.6μs
148ns
553ns
0
0
0
62.2 KB
master StringConcatAspectBenchmarknet6.0
298μs
1.65μs
11.5μs
0
0
0
205.27 KB
master StringConcatAspectBenchmarknetcoreapp3.1
307μs
1.64μs
8.38μs
0
0
0
202.09 KB
master StringConcatAspectBenchmarknet472
263μs
4.63μs
44.9μs
0
0
0
221.18 KB
#5257
StringConcatBenchmarknet6.0
52.9μs
258ns
1.1μs
0
0
0
43.44 KB
#5257
StringConcatBenchmarknetcoreapp3.1
54.2μs
245ns
1.8μs
0
0
0
42.64 KB
#5257
StringConcatBenchmarknet472
37.5μs
138ns
498ns
0
0
0
61.77 KB
#5257
StringConcatAspectBenchmarknet6.0
287μs
1.46μs
6.37μs
0
0
0
202.47 KB
#5257
StringConcatAspectBenchmarknetcoreapp3.1
310μs
1.77μs
13.1μs
0
0
0
202.77 KB
#5257
StringConcatAspectBenchmarknet472
261μs
5.15μs
49.4μs
0
0
0
221.18 KB
DataDog
![datadog-ddstaging[bot] avatar](https://avatars.githubusercontent.com/u/365230?v=4 "Published by a pub.dev verified publisher"){kind=small}
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}