[test PR:do not review]: test high RPS on baseline branch

[faydef]

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

[datadog-datadog-prod-us1[bot]]

Datadog Report

Branch report: fayssal/test-master Commit report: c58ad6c Test service: dd-trace-dotnet

:x: 1 Failed (0 Known Flaky), 252760 Passed, 2369 Skipped, 21h 11m 57.04s Total Time

:x: Failed Tests (1)

• TestIastXpathInjectionRequest - Datadog.Trace.Security.IntegrationTests.Iast.AspNetCore5IastTestsFullSamplingIastEnabled - Details

  Expand for error

  esults do not match.
  ifferences:
  eceived: Iast.XpathInjection.AspNetCore5.IastEnabled.received.txt
  erified: Iast.XpathInjection.AspNetCore5.IastEnabled.verified.txt
  eceived Content:
  ]
  erified Content:
  
   {
     TraceId: Id_1,
  ..
  

[andrewlock]

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

• This PR (6816)
• master

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

• Welch test with statistical test for significance of 5%
• Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6816) - mean (69ms)  : 67, 71
     .   : milestone, 69,
    master - mean (69ms)  : 67, 71
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6816) - mean (1,008ms)  : 986, 1031
     .   : milestone, 1008,
    master - mean (1,005ms)  : 983, 1027
     .   : milestone, 1005,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6816) - mean (103ms)  : 101, 105
     .   : milestone, 103,
    master - mean (102ms)  : 100, 104
     .   : milestone, 102,

    section CallTarget+Inlining+NGEN
    This PR (6816) - mean (691ms)  : 670, 711
     .   : milestone, 691,
    master - mean (688ms)  : 670, 706
     .   : milestone, 688,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6816) - mean (89ms)  : 87, 92
     .   : milestone, 89,
    master - mean (89ms)  : 87, 92
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (6816) - mean (638ms)  : 621, 655
     .   : milestone, 638,
    master - mean (644ms)  : 626, 661
     .   : milestone, 644,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6816) - mean (190ms)  : 187, 193
     .   : milestone, 190,
    master - mean (189ms)  : 186, 193
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (6816) - mean (1,104ms)  : 1081, 1127
     .   : milestone, 1104,
    master - mean (1,104ms)  : 1077, 1130
     .   : milestone, 1104,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6816) - mean (269ms)  : 265, 273
     .   : milestone, 269,
    master - mean (268ms)  : 263, 272
     .   : milestone, 268,

    section CallTarget+Inlining+NGEN
    This PR (6816) - mean (874ms)  : 833, 915
     .   : milestone, 874,
    master - mean (873ms)  : 838, 908
     .   : milestone, 873,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6816) - mean (261ms)  : 258, 265
     .   : milestone, 261,
    master - mean (261ms)  : 258, 265
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (6816) - mean (856ms)  : 822, 891
     .   : milestone, 856,
    master - mean (860ms)  : 826, 894
     .   : milestone, 860,

[andrewlock]

Benchmarks Report for tracer :snail:

Benchmarks for #6816 compared to master:

• 1 benchmarks are faster, with geometric mean 1.153
• 1 benchmarks are slower, with geometric mean 1.213
• All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

• Mann–Whitney U test with statistical test for significance of 5%
• Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	StartStopWithChild	net6.0	7.99μs	45.8ns	330ns	0.0197	0.00789	0	5.61 KB
master	StartStopWithChild	netcoreapp3.1	10.2μs	57.3ns	367ns	0.0249	0.00996	0	5.8 KB
master	StartStopWithChild	net472	15.9μs	30.6ns	110ns	1.05	0.33	0.102	6.21 KB
#6816	StartStopWithChild	net6.0	8.03μs	45.7ns	333ns	0.0158	0.00791	0	5.6 KB
#6816	StartStopWithChild	netcoreapp3.1	10.3μs	58.3ns	391ns	0.0153	0.00511	0	5.81 KB
#6816	StartStopWithChild	net472	16.3μs	36.3ns	141ns	1.04	0.3	0.0891	6.21 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	WriteAndFlushEnrichedTraces	net6.0	502μs	350ns	1.36μs	0	0	0	2.7 KB
master	WriteAndFlushEnrichedTraces	netcoreapp3.1	651μs	421ns	1.57μs	0	0	0	2.7 KB
master	WriteAndFlushEnrichedTraces	net472	860μs	538ns	1.94μs	0.428	0	0	3.3 KB
#6816	WriteAndFlushEnrichedTraces	net6.0	505μs	364ns	1.41μs	0	0	0	2.7 KB
#6816	WriteAndFlushEnrichedTraces	netcoreapp3.1	664μs	673ns	2.61μs	0	0	0	2.7 KB
#6816	WriteAndFlushEnrichedTraces	net472	848μs	421ns	1.52μs	0.422	0	0	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	SendRequest	net6.0	129μs	390ns	1.51μs	0.193	0	0	14.47 KB
master	SendRequest	netcoreapp3.1	148μs	344ns	1.33μs	0.218	0	0	17.27 KB
master	SendRequest	net472	0.000248ns	0.000156ns	0.000603ns	0	0	0	0 b
#6816	SendRequest	net6.0	127μs	326ns	1.26μs	0.186	0	0	14.47 KB
#6816	SendRequest	netcoreapp3.1	148μs	162ns	626ns	0.22	0	0	17.27 KB
#6816	SendRequest	net472	0.000558ns	0.000298ns	0.00111ns	0	0	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	WriteAndFlushEnrichedTraces	net6.0	611μs	3.55μs	32μs	0.303	0	0	41.6 KB
master	WriteAndFlushEnrichedTraces	netcoreapp3.1	707μs	4.1μs	35.5μs	0.351	0	0	41.79 KB
master	WriteAndFlushEnrichedTraces	net472	870μs	4.28μs	18.7μs	8.19	2.59	0.431	53.28 KB
#6816	WriteAndFlushEnrichedTraces	net6.0	577μs	3.39μs	32.7μs	0.576	0	0	41.57 KB
#6816	WriteAndFlushEnrichedTraces	netcoreapp3.1	720μs	3.8μs	20.8μs	0.336	0	0	41.84 KB
#6816	WriteAndFlushEnrichedTraces	net472	866μs	4.33μs	19.4μs	8.3	2.62	0.437	53.38 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	ExecuteNonQuery	net6.0	1.34μs	1.48ns	5.74ns	0.0141	0	0	1.02 KB
master	ExecuteNonQuery	netcoreapp3.1	1.79μs	1.49ns	5.79ns	0.0143	0	0	1.02 KB
master	ExecuteNonQuery	net472	2.03μs	2.51ns	9.37ns	0.157	0.00103	0	987 B
#6816	ExecuteNonQuery	net6.0	1.25μs	1.27ns	4.94ns	0.0144	0	0	1.02 KB
#6816	ExecuteNonQuery	netcoreapp3.1	1.82μs	1.9ns	7.09ns	0.0136	0	0	1.02 KB
#6816	ExecuteNonQuery	net472	2.11μs	2.68ns	10.4ns	0.157	0.00106	0	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	CallElasticsearch	net6.0	1.2μs	0.901ns	3.49ns	0.014	0	0	976 B
master	CallElasticsearch	netcoreapp3.1	1.59μs	0.867ns	3.36ns	0.0129	0	0	976 B
master	CallElasticsearch	net472	2.64μs	1.08ns	3.9ns	0.158	0	0	995 B
master	CallElasticsearchAsync	net6.0	1.26μs	0.914ns	3.42ns	0.0131	0	0	952 B
master	CallElasticsearchAsync	netcoreapp3.1	1.67μs	1.41ns	5.27ns	0.0142	0	0	1.02 KB
master	CallElasticsearchAsync	net472	2.59μs	1.24ns	4.8ns	0.166	0	0	1.05 KB
#6816	CallElasticsearch	net6.0	1.22μs	0.563ns	2.18ns	0.0141	0	0	976 B
#6816	CallElasticsearch	netcoreapp3.1	1.47μs	0.839ns	2.91ns	0.0133	0	0	976 B
#6816	CallElasticsearch	net472	2.56μs	0.951ns	3.68ns	0.158	0	0	995 B
#6816	CallElasticsearchAsync	net6.0	1.37μs	0.776ns	2.69ns	0.0131	0	0	952 B
#6816	CallElasticsearchAsync	netcoreapp3.1	1.65μs	0.582ns	2.25ns	0.0139	0	0	1.02 KB
#6816	CallElasticsearchAsync	net472	2.68μs	1.28ns	4.96ns	0.166	0	0	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	ExecuteAsync	net6.0	1.38μs	0.616ns	2.3ns	0.0131	0	0	952 B
master	ExecuteAsync	netcoreapp3.1	1.71μs	3.02ns	11.7ns	0.0128	0	0	952 B
master	ExecuteAsync	net472	1.79μs	0.34ns	1.23ns	0.145	0	0	915 B
#6816	ExecuteAsync	net6.0	1.29μs	0.461ns	1.79ns	0.0136	0	0	952 B
#6816	ExecuteAsync	netcoreapp3.1	1.63μs	1.97ns	7.37ns	0.0127	0	0	952 B
#6816	ExecuteAsync	net472	1.82μs	0.375ns	1.4ns	0.145	0	0	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	SendAsync	net6.0	4.38μs	1.42ns	5.51ns	0.0329	0	0	2.31 KB
master	SendAsync	netcoreapp3.1	5.3μs	3.79ns	14.7ns	0.0371	0	0	2.85 KB
master	SendAsync	net472	7.52μs	1.55ns	5.99ns	0.494	0	0	3.12 KB
#6816	SendAsync	net6.0	4.35μs	1.96ns	7.07ns	0.0307	0	0	2.31 KB
#6816	SendAsync	netcoreapp3.1	5.34μs	2.03ns	7.58ns	0.0373	0	0	2.85 KB
#6816	SendAsync	net472	7.33μs	1.42ns	5.5ns	0.496	0	0	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #6816

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1	1.153	2,451.49	2,125.68

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	EnrichedLog	net6.0	1.51μs	0.751ns	2.81ns	0.0234	0	0	1.69 KB
master	EnrichedLog	netcoreapp3.1	2.45μs	1.25ns	4.68ns	0.0222	0	0	1.69 KB
master	EnrichedLog	net472	2.6μs	1.11ns	4.15ns	0.258	0	0	1.62 KB
#6816	EnrichedLog	net6.0	1.64μs	0.51ns	1.84ns	0.0237	0	0	1.69 KB
#6816	EnrichedLog	netcoreapp3.1	2.13μs	1.84ns	6.88ns	0.0222	0	0	1.69 KB
#6816	EnrichedLog	net472	2.6μs	0.923ns	3.45ns	0.257	0	0	1.62 KB

Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	EnrichedLog	net6.0	112μs	105ns	406ns	0.0557	0	0	4.3 KB
master	EnrichedLog	netcoreapp3.1	117μs	180ns	698ns	0.0582	0	0	4.3 KB
master	EnrichedLog	net472	150μs	172ns	665ns	0.677	0.226	0	4.49 KB
#6816	EnrichedLog	net6.0	115μs	53ns	205ns	0.0569	0	0	4.3 KB
#6816	EnrichedLog	netcoreapp3.1	118μs	263ns	1.02μs	0.0584	0	0	4.3 KB
#6816	EnrichedLog	net472	150μs	88.7ns	320ns	0.671	0.224	0	4.49 KB

Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	EnrichedLog	net6.0	3.21μs	1.63ns	6.11ns	0.0305	0	0	2.25 KB
master	EnrichedLog	netcoreapp3.1	4.2μs	2.91ns	11.3ns	0.0293	0	0	2.25 KB
master	EnrichedLog	net472	4.93μs	0.945ns	3.66ns	0.329	0	0	2.07 KB
#6816	EnrichedLog	net6.0	3.1μs	1.3ns	4.49ns	0.031	0	0	2.25 KB
#6816	EnrichedLog	netcoreapp3.1	4.13μs	2.24ns	8.66ns	0.0311	0	0	2.25 KB
#6816	EnrichedLog	net472	4.88μs	1.67ns	6.24ns	0.328	0	0	2.07 KB

Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	SendReceive	net6.0	1.33μs	0.491ns	1.9ns	0.016	0	0	1.14 KB
master	SendReceive	netcoreapp3.1	1.79μs	1.43ns	5.54ns	0.0152	0	0	1.14 KB
master	SendReceive	net472	2.1μs	1.05ns	3.79ns	0.183	0	0	1.16 KB
#6816	SendReceive	net6.0	1.4μs	0.651ns	2.52ns	0.0161	0	0	1.14 KB
#6816	SendReceive	netcoreapp3.1	1.73μs	0.75ns	2.9ns	0.0155	0	0	1.14 KB
#6816	SendReceive	net472	2.14μs	1.19ns	4.46ns	0.183	0	0	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	EnrichedLog	net6.0	2.87μs	0.819ns	3.17ns	0.023	0	0	1.62 KB
master	EnrichedLog	netcoreapp3.1	4.08μs	2.28ns	8.52ns	0.0224	0	0	1.67 KB
master	EnrichedLog	net472	4.37μs	2.84ns	10.2ns	0.327	0	0	2.07 KB
#6816	EnrichedLog	net6.0	2.75μs	0.884ns	3.42ns	0.022	0	0	1.62 KB
#6816	EnrichedLog	netcoreapp3.1	4.02μs	1.41ns	5.46ns	0.0219	0	0	1.67 KB
#6816	EnrichedLog	net472	4.38μs	1.38ns	5.36ns	0.328	0	0	2.07 KB

Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #6816

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1	1.213	671.62	814.43

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	StartFinishSpan	net6.0	404ns	0.237ns	0.918ns	0.00811	0	0	576 B
master	StartFinishSpan	netcoreapp3.1	560ns	0.223ns	0.836ns	0.00785	0	0	576 B
master	StartFinishSpan	net472	695ns	0.715ns	2.77ns	0.0917	0	0	578 B
master	StartFinishScope	net6.0	473ns	0.152ns	0.568ns	0.00973	0	0	696 B
master	StartFinishScope	netcoreapp3.1	671ns	0.286ns	1.07ns	0.00924	0	0	696 B
master	StartFinishScope	net472	803ns	0.783ns	3.03ns	0.104	0	0	658 B
#6816	StartFinishSpan	net6.0	410ns	0.114ns	0.443ns	0.00804	0	0	576 B
#6816	StartFinishSpan	netcoreapp3.1	562ns	0.325ns	1.26ns	0.00793	0	0	576 B
#6816	StartFinishSpan	net472	631ns	0.143ns	0.555ns	0.0918	0	0	578 B
#6816	StartFinishScope	net6.0	503ns	0.128ns	0.494ns	0.00984	0	0	696 B
#6816	StartFinishScope	netcoreapp3.1	814ns	0.404ns	1.56ns	0.00933	0	0	696 B
#6816	StartFinishScope	net472	803ns	0.797ns	3.09ns	0.104	0	0	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	RunOnMethodBegin	net6.0	667ns	0.543ns	2.1ns	0.00965	0	0	696 B
master	RunOnMethodBegin	netcoreapp3.1	959ns	0.52ns	1.95ns	0.00926	0	0	696 B
master	RunOnMethodBegin	net472	1.14μs	0.332ns	1.28ns	0.104	0	0	658 B
#6816	RunOnMethodBegin	net6.0	685ns	0.226ns	0.876ns	0.00983	0	0	696 B
#6816	RunOnMethodBegin	netcoreapp3.1	947ns	0.537ns	2.08ns	0.00949	0	0	696 B
#6816	RunOnMethodBegin	net472	1.13μs	0.261ns	1.01ns	0.104	0	0	658 B

[bouwkast]

Closing as stale