dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard
verified publisher icon DataDog

[IAST] Added CallSites scoped to certain methods

Open daniel-romano-DD opened this issue 1 year ago • 5 comments

Summary of changes

Added the hability to filter restrict some callsites to the bodies of the "filtered" methods.

Reason for change

Some special instrumentations required to replace some calls only on certain target functions bodies.

Implementation details

Added a CallSiteFilter property to the AspectClass attribute. The aspects defined under that AspectClass will be applied only to the functiones specified in the CallSiteFilter attribute (array of strings)

Test coverage

Other details

daniel-romano-DD avatar Apr 17 '24 08:04 daniel-romano-DD

Datadog Report

Branch report: dani/iast/callsite_filtering Commit report: 0f251a2 Test service: dd-trace-dotnet

:white_check_mark: 0 Failed, 336742 Passed, 1981 Skipped, 49m 59.32s Wall Time

datadog-ddstaging[bot] avatar Apr 17 '24 08:04 datadog-ddstaging[bot]

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5455) - mean (75ms)  : 62, 88
     .   : milestone, 75,
    master - mean (74ms)  : 66, 81
     .   : milestone, 74,

    section CallTarget+Inlining+NGEN
    This PR (5455) - mean (1,022ms)  : 1004, 1040
     .   : milestone, 1022,
    master - mean (1,016ms)  : 992, 1039
     .   : milestone, 1016,


gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5455) - mean (110ms)  : 106, 114
     .   : milestone, 110,
    master - mean (109ms)  : 106, 112
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (5455) - mean (740ms)  : 720, 760
     .   : milestone, 740,
    master - mean (737ms)  : 716, 759
     .   : milestone, 737,


gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5455) - mean (94ms)  : 91, 96
     .   : milestone, 94,
    master - mean (94ms)  : 91, 98
     .   : milestone, 94,

    section CallTarget+Inlining+NGEN
    This PR (5455) - mean (692ms)  : 669, 714
     .   : milestone, 692,
    master - mean (693ms)  : 675, 711
     .   : milestone, 693,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5455) - mean (191ms)  : 188, 195
     .   : milestone, 191,
    master - mean (192ms)  : 189, 194
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (5455) - mean (1,089ms)  : 1064, 1115
     .   : milestone, 1089,
    master - mean (1,094ms)  : 1072, 1116
     .   : milestone, 1094,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5455) - mean (277ms)  : 273, 281
     .   : milestone, 277,
    master - mean (277ms)  : 272, 281
     .   : milestone, 277,

    section CallTarget+Inlining+NGEN
    This PR (5455) - mean (883ms)  : 864, 903
     .   : milestone, 883,
    master - mean (884ms)  : 859, 909
     .   : milestone, 884,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5455) - mean (265ms)  : 260, 269
     .   : milestone, 265,
    master - mean (265ms)  : 259, 270
     .   : milestone, 265,

    section CallTarget+Inlining+NGEN
    This PR (5455) - mean (868ms)  : 838, 897
     .   : milestone, 868,
    master - mean (865ms)  : 835, 894
     .   : milestone, 865,

andrewlock avatar Apr 17 '24 08:04 andrewlock

Benchmarks Report for tracer :snail:

Benchmarks for #5455 compared to master:

  • 2 benchmarks are faster, with geometric mean 1.144
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.95μs 44.1ns 225ns 0.0284 0.0142 0.00473 7.54 KB
master StartStopWithChild netcoreapp3.1 10.7μs 56.2ns 276ns 0.0253 0.0101 0 7.64 KB
master StartStopWithChild net472 17.1μs 49.2ns 191ns 1.34 0.335 0.103 8.07 KB
#5455 StartStopWithChild net6.0 8.77μs 47.7ns 261ns 0.0179 0.00896 0 7.54 KB
#5455 StartStopWithChild netcoreapp3.1 10.9μs 59.6ns 337ns 0.0279 0.0112 0 7.64 KB
#5455 StartStopWithChild net472 16.8μs 46.4ns 174ns 1.36 0.405 0.118 8.09 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 486μs 325ns 1.26μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 621μs 201ns 777ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 815μs 393ns 1.47μs 0.406 0 0 3.3 KB
#5455 WriteAndFlushEnrichedTraces net6.0 463μs 270ns 1.05μs 0 0 0 2.7 KB
#5455 WriteAndFlushEnrichedTraces netcoreapp3.1 620μs 282ns 1.06μs 0 0 0 2.7 KB
#5455 WriteAndFlushEnrichedTraces net472 831μs 592ns 2.29μs 0.414 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 169μs 157ns 609ns 0.252 0 0 18.38 KB
master SendRequest netcoreapp3.1 191μs 248ns 962ns 0.192 0 0 20.54 KB
master SendRequest net472 0.000134ns 0.000121ns 0.000452ns 0 0 0 0 b
#5455 SendRequest net6.0 167μs 205ns 794ns 0.165 0 0 18.38 KB
#5455 SendRequest netcoreapp3.1 192μs 433ns 1.68μs 0.19 0 0 20.54 KB
#5455 SendRequest net472 0.000887ns 0.000339ns 0.00131ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 533μs 262ns 1.02μs 0.556 0 0 41.53 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 642μs 766ns 2.97μs 0.326 0 0 41.68 KB
master WriteAndFlushEnrichedTraces net472 868μs 2.94μs 11.4μs 8.09 2.25 0.45 53.26 KB
#5455 WriteAndFlushEnrichedTraces net6.0 544μs 708ns 2.74μs 0.558 0 0 41.5 KB
#5455 WriteAndFlushEnrichedTraces netcoreapp3.1 665μs 929ns 3.6μs 0.332 0 0 41.76 KB
#5455 WriteAndFlushEnrichedTraces net472 856μs 2.92μs 11.3μs 8.3 2.62 0.437 53.23 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.17μs 0.429ns 1.66ns 0.0112 0 0 800 B
master ExecuteNonQuery netcoreapp3.1 1.51μs 0.877ns 3.4ns 0.0105 0 0 800 B
master ExecuteNonQuery net472 1.82μs 0.524ns 2.03ns 0.12 0 0 762 B
#5455 ExecuteNonQuery net6.0 1.26μs 0.571ns 2.21ns 0.0114 0 0 800 B
#5455 ExecuteNonQuery netcoreapp3.1 1.5μs 0.79ns 3.06ns 0.0105 0 0 800 B
#5455 ExecuteNonQuery net472 1.81μs 1.42ns 5.11ns 0.12 0 0 762 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.32μs 0.505ns 1.95ns 0.0144 0 0 1.02 KB
master CallElasticsearch netcoreapp3.1 1.59μs 0.873ns 3.27ns 0.0143 0 0 1.02 KB
master CallElasticsearch net472 2.46μs 1.49ns 5.79ns 0.164 0 0 1.04 KB
master CallElasticsearchAsync net6.0 1.29μs 0.989ns 3.7ns 0.0135 0 0 1 KB
master CallElasticsearchAsync netcoreapp3.1 1.77μs 0.811ns 3.03ns 0.0142 0 0 1.07 KB
master CallElasticsearchAsync net472 2.69μs 1.98ns 7.68ns 0.172 0 0 1.09 KB
#5455 CallElasticsearch net6.0 1.24μs 0.565ns 2.11ns 0.0144 0 0 1.02 KB
#5455 CallElasticsearch netcoreapp3.1 1.69μs 1.24ns 4.64ns 0.0137 0 0 1.02 KB
#5455 CallElasticsearch net472 2.48μs 1.86ns 6.95ns 0.164 0 0 1.04 KB
#5455 CallElasticsearchAsync net6.0 1.34μs 0.494ns 1.85ns 0.0141 0 0 1 KB
#5455 CallElasticsearchAsync netcoreapp3.1 1.73μs 0.753ns 2.82ns 0.0138 0 0 1.07 KB
#5455 CallElasticsearchAsync net472 2.71μs 0.878ns 3.4ns 0.172 0 0 1.09 KB
Benchmarks.Trace.GraphQLBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #5455

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0 1.134 1,445.68 1,275.04

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.45μs 0.509ns 1.97ns 0.013 0 0 944 B
master ExecuteAsync netcoreapp3.1 1.61μs 2.43ns 9.42ns 0.0122 0 0 944 B
master ExecuteAsync net472 1.85μs 2.88ns 11.2ns 0.143 0 0 907 B
#5455 ExecuteAsync net6.0 1.28μs 0.613ns 2.29ns 0.0134 0 0 944 B
#5455 ExecuteAsync netcoreapp3.1 1.67μs 0.66ns 2.47ns 0.0126 0 0 944 B
#5455 ExecuteAsync net472 1.77μs 1.21ns 4.7ns 0.144 0 0 907 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.15μs 1.71ns 6.4ns 0.031 0 0 2.18 KB
master SendAsync netcoreapp3.1 5.14μs 12.4ns 46.2ns 0.0358 0 0 2.71 KB
master SendAsync net472 7.71μs 2.44ns 9.46ns 0.484 0 0 3.07 KB
#5455 SendAsync net6.0 4.24μs 2.01ns 7.51ns 0.0312 0 0 2.18 KB
#5455 SendAsync netcoreapp3.1 4.96μs 2.07ns 7.73ns 0.0348 0 0 2.71 KB
#5455 SendAsync net472 7.64μs 2.76ns 10.3ns 0.487 0 0 3.07 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.59μs 0.766ns 2.97ns 0.0238 0 0 1.68 KB
master EnrichedLog netcoreapp3.1 2.4μs 0.911ns 3.28ns 0.0225 0 0 1.68 KB
master EnrichedLog net472 2.81μs 3.79ns 14.7ns 0.254 0 0 1.61 KB
#5455 EnrichedLog net6.0 1.49μs 0.208ns 0.807ns 0.0237 0 0 1.68 KB
#5455 EnrichedLog netcoreapp3.1 2.21μs 1.42ns 5.3ns 0.0224 0 0 1.68 KB
#5455 EnrichedLog net472 2.67μs 2.72ns 9.82ns 0.255 0 0 1.61 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 113μs 149ns 575ns 0.0562 0 0 4.26 KB
master EnrichedLog netcoreapp3.1 118μs 210ns 814ns 0.0592 0 0 4.26 KB
master EnrichedLog net472 149μs 145ns 563ns 0.668 0.223 0 4.45 KB
#5455 EnrichedLog net6.0 114μs 65.6ns 246ns 0.0568 0 0 4.26 KB
#5455 EnrichedLog netcoreapp3.1 121μs 117ns 454ns 0 0 0 4.26 KB
#5455 EnrichedLog net472 149μs 179ns 695ns 0.672 0.224 0 4.45 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.93μs 0.998ns 3.74ns 0.0309 0 0 2.24 KB
master EnrichedLog netcoreapp3.1 4.19μs 1.3ns 5.02ns 0.0294 0 0 2.24 KB
master EnrichedLog net472 4.9μs 1.9ns 7.35ns 0.325 0 0 2.05 KB
#5455 EnrichedLog net6.0 3.01μs 1ns 3.75ns 0.0314 0 0 2.24 KB
#5455 EnrichedLog netcoreapp3.1 4.29μs 2.62ns 9.82ns 0.0299 0 0 2.24 KB
#5455 EnrichedLog net472 4.87μs 1.73ns 6.72ns 0.324 0 0 2.05 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.4μs 1.34ns 5.02ns 0.0168 0 0 1.19 KB
master SendReceive netcoreapp3.1 1.81μs 2.17ns 8.42ns 0.0163 0 0 1.19 KB
master SendReceive net472 2.2μs 1.72ns 6.2ns 0.19 0.00109 0 1.2 KB
#5455 SendReceive net6.0 1.48μs 1.96ns 7.05ns 0.0163 0 0 1.19 KB
#5455 SendReceive netcoreapp3.1 1.85μs 0.517ns 1.86ns 0.0158 0 0 1.19 KB
#5455 SendReceive net472 2.26μs 1.54ns 5.95ns 0.19 0.00112 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.8μs 0.611ns 2.2ns 0.0224 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 3.83μs 2.55ns 9.87ns 0.0212 0 0 1.63 KB
master EnrichedLog net472 4.41μs 2.56ns 9.93ns 0.319 0 0 2.02 KB
#5455 EnrichedLog net6.0 2.77μs 0.84ns 3.03ns 0.022 0 0 1.58 KB
#5455 EnrichedLog netcoreapp3.1 3.89μs 1.69ns 6.54ns 0.0213 0 0 1.63 KB
#5455 EnrichedLog net472 4.42μs 1.84ns 7.13ns 0.32 0 0 2.02 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 544ns 0.236ns 0.884ns 0.00792 0 0 568 B
master StartFinishSpan netcoreapp3.1 752ns 0.662ns 2.57ns 0.00759 0 0 568 B
master StartFinishSpan net472 830ns 0.624ns 2.42ns 0.0905 0 0 570 B
master StartFinishScope net6.0 538ns 0.347ns 1.3ns 0.00967 0 0 688 B
master StartFinishScope netcoreapp3.1 785ns 0.973ns 3.37ns 0.00947 0 0 688 B
master StartFinishScope net472 1.01μs 0.868ns 3.36ns 0.103 0 0 650 B
#5455 StartFinishSpan net6.0 614ns 3.21ns 15.4ns 0.00788 0 0 568 B
#5455 StartFinishSpan netcoreapp3.1 675ns 0.822ns 3.18ns 0.00772 0 0 568 B
#5455 StartFinishSpan net472 750ns 0.907ns 3.51ns 0.0904 0 0 570 B
#5455 StartFinishScope net6.0 565ns 0.438ns 1.64ns 0.00973 0 0 688 B
#5455 StartFinishScope netcoreapp3.1 839ns 1.02ns 3.95ns 0.0093 0 0 688 B
#5455 StartFinishScope net472 1.07μs 1.37ns 5.31ns 0.103 0 0 650 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #5455

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.154 699.96 606.71

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 700ns 0.372ns 1.44ns 0.00975 0 0 688 B
master RunOnMethodBegin netcoreapp3.1 1.02μs 4.38ns 17ns 0.00916 0 0 688 B
master RunOnMethodBegin net472 1.11μs 0.989ns 3.83ns 0.103 0 0 650 B
#5455 RunOnMethodBegin net6.0 607ns 0.541ns 2.09ns 0.00972 0 0 688 B
#5455 RunOnMethodBegin netcoreapp3.1 962ns 3.34ns 12.9ns 0.00914 0 0 688 B
#5455 RunOnMethodBegin net472 1.1μs 1.15ns 4.44ns 0.103 0 0 650 B

andrewlock avatar Apr 17 '24 08:04 andrewlock

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5455) (10.814M)   : 0, 10814010
    master (11.522M)   : 0, 11522381
    benchmarks/2.9.0 (11.170M)   : 0, 11169786

    section Automatic
    This PR (5455) (7.638M)   : 0, 7638149
    master (7.941M)   : 0, 7940524
    benchmarks/2.9.0 (8.435M)   : 0, 8434769

    section Trace stats
    master (8.315M)   : 0, 8314530

    section Manual
    This PR (5455) (9.983M)   : 0, 9983425
    master (9.988M)   : 0, 9988179

    section Manual + Automatic
    This PR (5455) (7.376M)   : 0, 7376384
    master (7.586M)   : 0, 7586252

    section Version Conflict
    master (6.763M)   : 0, 6762646


gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5455) (9.573M)   : 0, 9573486
    master (9.577M)   : 0, 9576962
    benchmarks/2.9.0 (9.658M)   : 0, 9657706

    section Automatic
    This PR (5455) (6.601M)   : 0, 6601089
    master (6.559M)   : 0, 6558544

    section Trace stats
    master (6.899M)   : 0, 6899377

    section Manual
    This PR (5455) (8.434M)   : 0, 8434477
    master (8.285M)   : 0, 8284821

    section Manual + Automatic
    This PR (5455) (6.247M)   : 0, 6247320
    master (6.060M)   : 0, 6060400

    section Version Conflict
    master (5.577M)   : 0, 5576823


gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5455) (9.863M)   : 0, 9863046
    master (9.719M)   : 0, 9718909
    benchmarks/2.9.0 (10.068M)   : 0, 10068415

    section Automatic
    This PR (5455) (6.954M)   : 0, 6954110
    master (7.165M)   : 0, 7165128
    benchmarks/2.9.0 (7.475M)   : 0, 7475050

    section Trace stats
    master (7.388M)   : 0, 7387740

    section Manual
    This PR (5455) (8.816M)   : 0, 8815546
    master (8.762M)   : 0, 8761881

    section Manual + Automatic
    This PR (5455) (6.722M)   : 0, 6722407
    master (6.864M)   : 0, 6863897

    section Version Conflict
    master (6.179M)   : 0, 6179007

andrewlock avatar Apr 17 '24 08:04 andrewlock

Benchmarks Report for appsec :snail:

Benchmarks for #5455 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.172
  • 2 benchmarks are slower, with geometric mean 1.189
  • 2 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #5455

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1 1.195 207.12 247.44

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 72.1μs 96ns 372ns 0.0717 0 0 5.99 KB
master AllCycleSimpleBody netcoreapp3.1 63μs 82.4ns 319ns 0.0944 0 0 6.93 KB
master AllCycleSimpleBody net472 48.7μs 103ns 399ns 1.31 0 0 8.32 KB
master AllCycleMoreComplexBody net6.0 78.1μs 132ns 513ns 0.117 0 0 9.5 KB
master AllCycleMoreComplexBody netcoreapp3.1 69.3μs 125ns 485ns 0.14 0 0 10.35 KB
master AllCycleMoreComplexBody net472 55.4μs 78.1ns 303ns 1.87 0.0275 0 11.83 KB
master ObjectExtractorSimpleBody net6.0 147ns 0.134ns 0.52ns 0.00395 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 207ns 0.0971ns 0.35ns 0.00375 0 0 272 B
master ObjectExtractorSimpleBody net472 175ns 0.0957ns 0.371ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.05μs 2.68ns 10.4ns 0.0532 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.91μs 2.57ns 9.96ns 0.0507 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 3.81μs 2.18ns 8.15ns 0.603 0.00574 0 3.8 KB
#5455 AllCycleSimpleBody net6.0 72.4μs 71ns 266ns 0.0722 0 0 5.99 KB
#5455 AllCycleSimpleBody netcoreapp3.1 61.6μs 40.6ns 146ns 0.0929 0 0 6.93 KB
#5455 AllCycleSimpleBody net472 47.7μs 55.3ns 214ns 1.31 0 0 8.32 KB
#5455 AllCycleMoreComplexBody net6.0 78.4μs 73.6ns 275ns 0.118 0 0 9.5 KB
#5455 AllCycleMoreComplexBody netcoreapp3.1 69.2μs 70ns 271ns 0.138 0 0 10.35 KB
#5455 AllCycleMoreComplexBody net472 55.1μs 73.4ns 284ns 1.87 0.0274 0 11.83 KB
#5455 ObjectExtractorSimpleBody net6.0 146ns 0.101ns 0.39ns 0.004 0 0 280 B
#5455 ObjectExtractorSimpleBody netcoreapp3.1 247ns 0.174ns 0.673ns 0.00361 0 0 272 B
#5455 ObjectExtractorSimpleBody net472 174ns 0.124ns 0.481ns 0.0446 0 0 281 B
#5455 ObjectExtractorMoreComplexBody net6.0 2.96μs 1.89ns 6.82ns 0.0536 0 0 3.78 KB
#5455 ObjectExtractorMoreComplexBody netcoreapp3.1 3.99μs 2.08ns 8.04ns 0.0498 0 0 3.69 KB
#5455 ObjectExtractorMoreComplexBody net472 3.86μs 3.02ns 11.7ns 0.601 0.00576 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 40.7μs 25ns 96.7ns 0.451 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 56.3μs 20.9ns 81.1ns 0.448 0 0 32.4 KB
master EncodeArgs net472 68.9μs 30ns 112ns 5.16 0.0688 0 32.5 KB
master EncodeLegacyArgs net6.0 74.9μs 50.4ns 174ns 0 0 0 2.14 KB
master EncodeLegacyArgs netcoreapp3.1 104μs 149ns 577ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 153μs 109ns 420ns 0.304 0 0 2.15 KB
#5455 EncodeArgs net6.0 39.7μs 17.5ns 67.9ns 0.459 0 0 32.4 KB
#5455 EncodeArgs netcoreapp3.1 55.8μs 28.1ns 105ns 0.416 0 0 32.4 KB
#5455 EncodeArgs net472 69.1μs 27.1ns 105ns 5.16 0.0688 0 32.5 KB
#5455 EncodeLegacyArgs net6.0 71.1μs 40.4ns 157ns 0 0 0 2.14 KB
#5455 EncodeLegacyArgs netcoreapp3.1 106μs 209ns 755ns 0 0 0 2.14 KB
#5455 EncodeLegacyArgs net472 153μs 89.3ns 346ns 0.306 0 0 2.15 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 180μs 277ns 1.07μs 0 0 0 2.13 KB
master RunWafRealisticBenchmark netcoreapp3.1 194μs 150ns 580ns 0 0 0 2.1 KB
master RunWafRealisticBenchmark net472 210μs 113ns 438ns 0.316 0 0 2.17 KB
master RunWafRealisticBenchmarkWithAttack net6.0 120μs 173ns 669ns 0 0 0 1.37 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 127μs 85.9ns 321ns 0 0 0 1.36 KB
master RunWafRealisticBenchmarkWithAttack net472 139μs 32.9ns 123ns 0.207 0 0 1.39 KB
#5455 RunWafRealisticBenchmark net6.0 181μs 115ns 446ns 0 0 0 2.13 KB
#5455 RunWafRealisticBenchmark netcoreapp3.1 195μs 151ns 522ns 0 0 0 2.1 KB
#5455 RunWafRealisticBenchmark net472 212μs 114ns 442ns 0.318 0 0 2.17 KB
#5455 RunWafRealisticBenchmarkWithAttack net6.0 120μs 63.6ns 238ns 0 0 0 1.37 KB
#5455 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 129μs 98.4ns 381ns 0 0 0 1.36 KB
#5455 RunWafRealisticBenchmarkWithAttack net472 139μs 64.7ns 251ns 0.208 0 0 1.39 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower :warning: Fewer allocations :tada:

Slower :warning: in #5455

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 1.184 52,800.00 62,500.00 bimodal

Faster :tada: in #5455

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 1.172 61,250.00 52,250.00 bimodal

Fewer allocations :tada: in #5455

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 204.16 KB 202.51 KB -1.65 KB -0.81%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 203.71 KB 201.61 KB -2.1 KB -1.03%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 60.4μs 726ns 6.89μs 0 0 0 43.44 KB
master StringConcatBenchmark netcoreapp3.1 52.7μs 255ns 987ns 0 0 0 42.64 KB
master StringConcatBenchmark net472 37.9μs 69.9ns 261ns 0 0 0 59.26 KB
master StringConcatAspectBenchmark net6.0 295μs 1.6μs 8.46μs 0 0 0 204.16 KB
master StringConcatAspectBenchmark netcoreapp3.1 311μs 1.31μs 8.36μs 0 0 0 203.71 KB
master StringConcatAspectBenchmark net472 269μs 5.87μs 57.2μs 0 0 0 221.18 KB
#5455 StringConcatBenchmark net6.0 52.3μs 228ns 854ns 0 0 0 43.44 KB
#5455 StringConcatBenchmark netcoreapp3.1 63.2μs 838ns 8.3μs 0 0 0 42.64 KB
#5455 StringConcatBenchmark net472 37.8μs 69.3ns 259ns 0 0 0 59.22 KB
#5455 StringConcatAspectBenchmark net6.0 280μs 1.51μs 8.42μs 0 0 0 202.51 KB
#5455 StringConcatAspectBenchmark netcoreapp3.1 304μs 1.74μs 13.8μs 0 0 0 201.61 KB
#5455 StringConcatAspectBenchmark net472 251μs 3.57μs 34.3μs 0 0 0 221.18 KB

andrewlock avatar Apr 17 '24 09:04 andrewlock