dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard

Published 20 hours ago • verified publisher icon DataDog

Adding Throughput Test for Kafka & DSM

Open robcarlan-datadog opened this issue 5 months ago • 2 comments

Summary of changes

We want to enable Data Streams Monitoring by default. To ensure that the DSM code path has a minimal impact on customers, we've added throughput testing to ensure that the overhead of DSM is acceptable in various scenarios.

This PR adds throughput testing for publishing and consuming N Kafka messages. We compare the runtime of DSM with APM and APM without DSM enabled to ensure we have an acceptable performance overhead.

Reason for change

Implementation details

Test coverage

Other details

robcarlan-datadog avatar Jun 23 '25 18:06 robcarlan-datadog

Datadog Report

Branch report: rob.carlan/dsm-throughput-testing Commit report: a26a646 Test service: dd-trace-dotnet

:white_check_mark: 0 Failed, 241734 Passed, 2270 Skipped, 14h 52m 8.59s Total Time

datadog-datadog-prod-us1[bot] avatar Jun 23 '25 19:06 datadog-datadog-prod-us1[bot]

Benchmarks

Benchmarks Report for benchmark platform :snail:

Benchmarks for #7133 compared to master:

  • 2 benchmarks are slower, with geometric mean 1.195
  • 41 benchmarks have fewer allocations
  • 7 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net6.0 5.58 KB 5.5 KB -81 B -1.45%
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1 5.75 KB 5.66 KB -95 B -1.65%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 11.2μs 59.7ns 322ns 0 0 0 5.58 KB
master StartStopWithChild netcoreapp3.1 14.1μs 67.5ns 286ns 0 0 0 5.75 KB
master StartStopWithChild net472 22.1μs 119ns 651ns 1.03 0.411 0.103 6.09 KB
#7133 StartStopWithChild net6.0 10.4μs 58.4ns 404ns 0 0 0 5.5 KB
#7133 StartStopWithChild netcoreapp3.1 14μs 69.9ns 297ns 0 0 0 5.66 KB
#7133 StartStopWithChild net472 22.1μs 98.5ns 369ns 0.888 0.222 0 6.1 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 3.33 KB 3.31 KB -23 B -0.69%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 927μs 26.6ns 103ns 0 0 0 2.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.04ms 404ns 1.56μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.25ms 357ns 1.38μs 0 0 0 3.33 KB
#7133 WriteAndFlushEnrichedTraces net6.0 941μs 71.5ns 277ns 0 0 0 2.71 KB
#7133 WriteAndFlushEnrichedTraces netcoreapp3.1 1.02ms 152ns 546ns 0 0 0 2.7 KB
#7133 WriteAndFlushEnrichedTraces net472 1.23ms 97.2ns 376ns 0 0 0 3.31 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed :heavy_check_mark: More allocations :warning:

More allocations :warning: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net472 236.35 KB 239.66 KB 3.3 KB 1.40%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net472 239.87 KB 243.17 KB 3.3 KB 1.38%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 331μs 1.76μs 8.42μs 0 0 0 197.06 KB
master AllCycleSimpleBody netcoreapp3.1 510μs 1.45μs 5.61μs 0 0 0 204.77 KB
master AllCycleSimpleBody net472 436μs 119ns 460ns 36.6 2.16 0 236.35 KB
master AllCycleMoreComplexBody net6.0 338μs 1.76μs 8.8μs 0 0 0 200.56 KB
master AllCycleMoreComplexBody netcoreapp3.1 495μs 987ns 3.56μs 0 0 0 208.18 KB
master AllCycleMoreComplexBody net472 446μs 106ns 412ns 36.6 2.16 0 239.87 KB
master ObjectExtractorSimpleBody net6.0 311ns 1.77ns 12.3ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 409ns 1.93ns 8.2ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 303ns 0.175ns 0.676ns 0.0442 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.52μs 29.5ns 110ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.76μs 36.2ns 140ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.66μs 0.89ns 3.33ns 0.599 0 0 3.8 KB
#7133 AllCycleSimpleBody net6.0 335μs 520ns 1.95μs 0 0 0 197.61 KB
#7133 AllCycleSimpleBody netcoreapp3.1 513μs 283ns 1.09μs 0 0 0 205.37 KB
#7133 AllCycleSimpleBody net472 441μs 101ns 392ns 36.6 2.16 0 239.66 KB
#7133 AllCycleMoreComplexBody net6.0 340μs 805ns 3.12μs 0 0 0 201.11 KB
#7133 AllCycleMoreComplexBody netcoreapp3.1 503μs 1.16μs 4.36μs 0 0 0 208.78 KB
#7133 AllCycleMoreComplexBody net472 452μs 215ns 806ns 37.9 2.23 0 243.17 KB
#7133 ObjectExtractorSimpleBody net6.0 318ns 1.62ns 7.25ns 0 0 0 280 B
#7133 ObjectExtractorSimpleBody netcoreapp3.1 410ns 2.09ns 10ns 0 0 0 272 B
#7133 ObjectExtractorSimpleBody net472 309ns 1.6ns 7.65ns 0.0443 0 0 281 B
#7133 ObjectExtractorMoreComplexBody net6.0 6.37μs 12.2ns 45.5ns 0 0 0 3.78 KB
#7133 ObjectExtractorMoreComplexBody netcoreapp3.1 7.9μs 27.6ns 107ns 0 0 0 3.69 KB
#7133 ObjectExtractorMoreComplexBody net472 6.9μs 6.05ns 22.6ns 0.585 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs‑net6.0 2.16 KB 2.15 KB -11 B -0.51%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 73.3μs 263ns 984ns 0 0 0 32.41 KB
master EncodeArgs netcoreapp3.1 95.7μs 36.2ns 130ns 0 0 0 32.4 KB
master EncodeArgs net472 107μs 17.6ns 65.8ns 4.82 0 0 32.51 KB
master EncodeLegacyArgs net6.0 143μs 122ns 472ns 0 0 0 2.16 KB
master EncodeLegacyArgs netcoreapp3.1 197μs 42.5ns 147ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 261μs 66.8ns 250ns 0 0 0 2.16 KB
#7133 EncodeArgs net6.0 76.5μs 133ns 497ns 0 0 0 32.4 KB
#7133 EncodeArgs netcoreapp3.1 97.6μs 255ns 987ns 0 0 0 32.4 KB
#7133 EncodeArgs net472 109μs 13.1ns 50.7ns 4.93 0 0 32.5 KB
#7133 EncodeLegacyArgs net6.0 144μs 34.2ns 132ns 0 0 0 2.15 KB
#7133 EncodeLegacyArgs netcoreapp3.1 200μs 69.8ns 261ns 0 0 0 2.14 KB
#7133 EncodeLegacyArgs net472 264μs 46.7ns 181ns 0 0 0 2.16 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 271μs 133ns 499ns 0 0 0 4.55 KB
master RunWafRealisticBenchmark netcoreapp3.1 294μs 264ns 989ns 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 307μs 38.1ns 147ns 0 0 0 4.66 KB
master RunWafRealisticBenchmarkWithAttack net6.0 181μs 80.1ns 300ns 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 198μs 104ns 404ns 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 207μs 45.3ns 169ns 0 0 0 2.28 KB
#7133 RunWafRealisticBenchmark net6.0 272μs 39.2ns 152ns 0 0 0 4.55 KB
#7133 RunWafRealisticBenchmark netcoreapp3.1 298μs 754ns 2.72μs 0 0 0 4.48 KB
#7133 RunWafRealisticBenchmark net472 310μs 43ns 167ns 0 0 0 4.66 KB
#7133 RunWafRealisticBenchmarkWithAttack net6.0 181μs 18.3ns 68.4ns 0 0 0 2.24 KB
#7133 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 195μs 301ns 1.09μs 0 0 0 2.22 KB
#7133 RunWafRealisticBenchmarkWithAttack net472 207μs 30.5ns 114ns 0 0 0 2.28 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 61.1μs 29ns 104ns 0 0 0 14.53 KB
master SendRequest netcoreapp3.1 69.9μs 118ns 440ns 0 0 0 17.42 KB
master SendRequest net472 0.0171ns 0.00147ns 0.00551ns 0 0 0 0 b
#7133 SendRequest net6.0 60.9μs 12.7ns 47.4ns 0 0 0 14.52 KB
#7133 SendRequest netcoreapp3.1 71.7μs 138ns 498ns 0 0 0 17.42 KB
#7133 SendRequest net472 0.00988ns 0.0023ns 0.00891ns 0 0 0 0 b
Benchmarks.Trace.CharSliceBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
#7133 OriginalCharSlice net6.0 1.96ms 365ns 1.32μs 0 0 0 640 KB
#7133 OriginalCharSlice netcoreapp3.1 2.12ms 6.23μs 23.3μs 0 0 0 640 KB
#7133 OriginalCharSlice net472 2.69ms 492ns 1.84μs 100 0 0 641.95 KB
#7133 OptimizedCharSlice net6.0 1.41ms 209ns 810ns 0 0 0 4 B
#7133 OptimizedCharSlice netcoreapp3.1 1.79ms 368ns 1.43μs 0 0 0 1 B
#7133 OptimizedCharSlice net472 1.99ms 314ns 1.18μs 0 0 0 73 B
#7133 OptimizedCharSliceWithPool net6.0 846μs 78.5ns 294ns 0 0 0 3 B
#7133 OptimizedCharSliceWithPool netcoreapp3.1 827μs 50.2ns 181ns 0 0 0 0 b
#7133 OptimizedCharSliceWithPool net472 1.14ms 33.7ns 131ns 0 0 0 47 B
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Slower :warning: More allocations :warning:

Slower :warning: in #7133

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 1.131 631,437.50 714,230.42

More allocations :warning: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 55.75 KB 56.42 KB 669 B 1.20%
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.89 KB 42.3 KB 410 B 0.98%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 645μs 512ns 1.98μs 0 0 0 41.73 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 631μs 1.67μs 6.67μs 0 0 0 41.89 KB
master WriteAndFlushEnrichedTraces net472 923μs 1.93μs 7.2μs 4.46 0 0 55.75 KB
#7133 WriteAndFlushEnrichedTraces net6.0 614μs 432ns 1.88μs 0 0 0 41.73 KB
#7133 WriteAndFlushEnrichedTraces netcoreapp3.1 714μs 2.12μs 8.2μs 0 0 0 42.3 KB
#7133 WriteAndFlushEnrichedTraces net472 982μs 4.24μs 15.9μs 4.81 0 0 56.42 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net6.0 1.03 KB 1.02 KB -8 B -0.78%
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑netcoreapp3.1 1.02 KB 1.02 KB -8 B -0.78%
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net472 995 B 987 B -8 B -0.80%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.99μs 10.2ns 46.8ns 0 0 0 1.03 KB
master ExecuteNonQuery netcoreapp3.1 2.53μs 3.43ns 13.3ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.7μs 2.41ns 9.34ns 0.147 0.0134 0 995 B
#7133 ExecuteNonQuery net6.0 1.93μs 1.04ns 4.03ns 0 0 0 1.02 KB
#7133 ExecuteNonQuery netcoreapp3.1 2.66μs 13.3ns 56.5ns 0 0 0 1.02 KB
#7133 ExecuteNonQuery net472 2.74μs 3.1ns 12ns 0.149 0.0136 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net472 1.11 KB 1.1 KB -8 B -0.72%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑netcoreapp3.1 1.09 KB 1.08 KB -8 B -0.74%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net472 1.05 KB 1.04 KB -8 B -0.76%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0 1.04 KB 1.03 KB -8 B -0.77%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑netcoreapp3.1 1.04 KB 1.03 KB -8 B -0.77%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net6.0 1.02 KB 1.01 KB -8 B -0.79%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.83μs 5.85ns 22.7ns 0 0 0 1.04 KB
master CallElasticsearch netcoreapp3.1 2.3μs 11.8ns 56.6ns 0 0 0 1.04 KB
master CallElasticsearch net472 3.55μs 3.05ns 11.8ns 0.159 0 0 1.05 KB
master CallElasticsearchAsync net6.0 1.81μs 3.89ns 14.5ns 0 0 0 1.02 KB
master CallElasticsearchAsync netcoreapp3.1 2.35μs 7.26ns 28.1ns 0 0 0 1.09 KB
master CallElasticsearchAsync net472 3.84μs 3.19ns 12.4ns 0.169 0 0 1.11 KB
#7133 CallElasticsearch net6.0 1.79μs 8.08ns 30.2ns 0 0 0 1.03 KB
#7133 CallElasticsearch netcoreapp3.1 2.31μs 9.85ns 36.9ns 0 0 0 1.03 KB
#7133 CallElasticsearch net472 3.67μs 3.2ns 11.6ns 0.165 0 0 1.04 KB
#7133 CallElasticsearchAsync net6.0 1.82μs 2.71ns 10.5ns 0 0 0 1.01 KB
#7133 CallElasticsearchAsync netcoreapp3.1 2.42μs 8.89ns 34.4ns 0 0 0 1.08 KB
#7133 CallElasticsearchAsync net472 3.94μs 3.46ns 13.4ns 0.157 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0 960 B 952 B -8 B -0.83%
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑netcoreapp3.1 960 B 952 B -8 B -0.83%
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net472 923 B 915 B -8 B -0.87%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.78μs 9.27ns 43.5ns 0 0 0 960 B
master ExecuteAsync netcoreapp3.1 2.29μs 7.45ns 25.8ns 0 0 0 960 B
master ExecuteAsync net472 2.58μs 1.78ns 6.88ns 0.143 0 0 923 B
#7133 ExecuteAsync net6.0 1.87μs 3.73ns 14.4ns 0 0 0 952 B
#7133 ExecuteAsync netcoreapp3.1 2.24μs 0.526ns 1.9ns 0 0 0 952 B
#7133 ExecuteAsync net472 2.55μs 1.33ns 5.13ns 0.14 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 7.1μs 10.4ns 39ns 0 0 0 2.37 KB
master SendAsync netcoreapp3.1 8.69μs 14.1ns 54.6ns 0 0 0 2.9 KB
master SendAsync net472 12.5μs 10.3ns 38.5ns 0.498 0 0 3.19 KB
#7133 SendAsync net6.0 7.06μs 6.87ns 26.6ns 0 0 0 2.36 KB
#7133 SendAsync netcoreapp3.1 8.71μs 6.06ns 22.7ns 0 0 0 2.9 KB
#7133 SendAsync net472 12.2μs 9.5ns 34.3ns 0.484 0 0 3.18 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower :warning: More allocations :warning:

Slower :warning: in #7133

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 1.262 413,700.00 522,250.00

More allocations :warning: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 57.34 KB 65.54 KB 8.19 KB 14.29%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 259.96 KB 274.69 KB 14.73 KB 5.67%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 43.83 KB 44.14 KB 304 B 0.69%

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 286.72 KB 278.46 KB -8.26 KB -2.88%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 44.6μs 212ns 1.6μs 0 0 0 43.83 KB
master StringConcatBenchmark netcoreapp3.1 47.2μs 224ns 838ns 0 0 0 42.64 KB
master StringConcatBenchmark net472 56.8μs 259ns 968ns 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 458μs 1.08μs 3.89μs 0 0 0 259.96 KB
master StringConcatAspectBenchmark netcoreapp3.1 447μs 6.44μs 63.7μs 0 0 0 274.93 KB
master StringConcatAspectBenchmark net472 410μs 2.07μs 9.27μs 0 0 0 286.72 KB
#7133 StringConcatBenchmark net6.0 45.3μs 265ns 2.16μs 0 0 0 44.14 KB
#7133 StringConcatBenchmark netcoreapp3.1 48.3μs 248ns 1.11μs 0 0 0 42.68 KB
#7133 StringConcatBenchmark net472 58.2μs 189ns 733ns 0 0 0 65.54 KB
#7133 StringConcatAspectBenchmark net6.0 473μs 2.03μs 7.04μs 0 0 0 274.69 KB
#7133 StringConcatAspectBenchmark netcoreapp3.1 521μs 1.66μs 6.23μs 0 0 0 273.96 KB
#7133 StringConcatAspectBenchmark net472 406μs 2.23μs 13.2μs 0 0 0 278.46 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net6.0 1.76 KB 1.7 KB -56 B -3.18%
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1 1.76 KB 1.7 KB -56 B -3.18%
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net472 1.69 KB 1.64 KB -56 B -3.31%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.63μs 2.6ns 10.1ns 0 0 0 1.76 KB
master EnrichedLog netcoreapp3.1 3.44μs 4.38ns 17ns 0 0 0 1.76 KB
master EnrichedLog net472 4.05μs 4.83ns 18.7ns 0.265 0 0 1.69 KB
#7133 EnrichedLog net6.0 2.61μs 9.28ns 35.9ns 0 0 0 1.7 KB
#7133 EnrichedLog netcoreapp3.1 3.56μs 6.52ns 25.2ns 0 0 0 1.7 KB
#7133 EnrichedLog net472 3.93μs 2.93ns 11ns 0.255 0 0 1.64 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑net472 4.57 KB 4.52 KB -55 B -1.20%
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑netcoreapp3.1 4.37 KB 4.31 KB -56 B -1.28%
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑net6.0 4.37 KB 4.31 KB -58 B -1.33%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 123μs 145ns 563ns 0 0 0 4.37 KB
master EnrichedLog netcoreapp3.1 126μs 330ns 1.24μs 0 0 0 4.37 KB
master EnrichedLog net472 167μs 161ns 603ns 0 0 0 4.57 KB
#7133 EnrichedLog net6.0 123μs 76.9ns 277ns 0 0 0 4.31 KB
#7133 EnrichedLog netcoreapp3.1 127μs 279ns 1.04μs 0 0 0 4.31 KB
#7133 EnrichedLog net472 167μs 86.1ns 334ns 0 0 0 4.52 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑net6.0 2.32 KB 2.26 KB -56 B -2.41%
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑netcoreapp3.1 2.32 KB 2.26 KB -56 B -2.41%
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑net472 2.14 KB 2.08 KB -56 B -2.62%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.94μs 19.3ns 74.6ns 0 0 0 2.32 KB
master EnrichedLog netcoreapp3.1 6.77μs 22.2ns 79.9ns 0 0 0 2.32 KB
master EnrichedLog net472 7.45μs 8.04ns 31.1ns 0.335 0 0 2.14 KB
#7133 EnrichedLog net6.0 5.1μs 24.9ns 103ns 0 0 0 2.26 KB
#7133 EnrichedLog netcoreapp3.1 6.78μs 26ns 101ns 0 0 0 2.26 KB
#7133 EnrichedLog net472 7.52μs 4.43ns 17.1ns 0.3 0 0 2.08 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.RedisBenchmark.SendReceive‑net472 1.21 KB 1.2 KB -8 B -0.66%
Benchmarks.Trace.RedisBenchmark.SendReceive‑net6.0 1.21 KB 1.2 KB -8 B -0.66%
Benchmarks.Trace.RedisBenchmark.SendReceive‑netcoreapp3.1 1.21 KB 1.2 KB -8 B -0.66%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2.04μs 10.5ns 49.4ns 0 0 0 1.21 KB
master SendReceive netcoreapp3.1 2.53μs 11.8ns 47.4ns 0 0 0 1.21 KB
master SendReceive net472 3.28μs 2.71ns 10.5ns 0.178 0 0 1.21 KB
#7133 SendReceive net6.0 1.96μs 8.84ns 34.2ns 0 0 0 1.2 KB
#7133 SendReceive netcoreapp3.1 2.65μs 13.1ns 58.7ns 0 0 0 1.2 KB
#7133 SendReceive net472 3.15μs 3.87ns 15ns 0.188 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑net472 2.08 KB 2.03 KB -56 B -2.69%
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑netcoreapp3.1 1.69 KB 1.63 KB -56 B -3.32%
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑net6.0 1.64 KB 1.58 KB -56 B -3.41%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.18μs 0.92ns 3.44ns 0 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 5.64μs 23ns 89.3ns 0 0 0 1.69 KB
master EnrichedLog net472 6.67μs 7.28ns 27.3ns 0.298 0 0 2.08 KB
#7133 EnrichedLog net6.0 4.21μs 2.56ns 9.92ns 0 0 0 1.58 KB
#7133 EnrichedLog netcoreapp3.1 5.62μs 11ns 42.8ns 0 0 0 1.63 KB
#7133 EnrichedLog net472 7.17μs 7.11ns 27.5ns 0.287 0 0 2.03 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 704 B 696 B -8 B -1.14%
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 704 B 696 B -8 B -1.14%
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 666 B 658 B -8 B -1.20%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472 586 B 578 B -8 B -1.37%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 584 B 576 B -8 B -1.37%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 584 B 576 B -8 B -1.37%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 751ns 3.97ns 22.4ns 0 0 0 584 B
master StartFinishSpan netcoreapp3.1 953ns 4.42ns 17.7ns 0 0 0 584 B
master StartFinishSpan net472 914ns 0.809ns 3.13ns 0.0912 0 0 586 B
master StartFinishScope net6.0 918ns 0.479ns 1.73ns 0 0 0 704 B
master StartFinishScope netcoreapp3.1 1.15μs 6.24ns 34.2ns 0 0 0 704 B
master StartFinishScope net472 1.09μs 0.174ns 0.652ns 0.104 0 0 666 B
#7133 StartFinishSpan net6.0 750ns 3.85ns 19.2ns 0 0 0 576 B
#7133 StartFinishSpan netcoreapp3.1 936ns 4.96ns 24.8ns 0 0 0 576 B
#7133 StartFinishSpan net472 959ns 0.254ns 0.915ns 0.0915 0 0 578 B
#7133 StartFinishScope net6.0 882ns 3.87ns 14.5ns 0 0 0 696 B
#7133 StartFinishScope netcoreapp3.1 1.17μs 6.18ns 31.5ns 0 0 0 696 B
#7133 StartFinishScope net472 1.12μs 0.367ns 1.42ns 0.101 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #7133

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 704 B 696 B -8 B -1.14%
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1 704 B 696 B -8 B -1.14%
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472 666 B 658 B -8 B -1.20%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.02μs 5.36ns 26.8ns 0 0 0 704 B
master RunOnMethodBegin netcoreapp3.1 1.38μs 2.28ns 8.82ns 0 0 0 704 B
master RunOnMethodBegin net472 1.36μs 0.141ns 0.544ns 0.102 0 0 666 B
#7133 RunOnMethodBegin net6.0 1.06μs 3.78ns 14.6ns 0 0 0 696 B
#7133 RunOnMethodBegin netcoreapp3.1 1.41μs 5.87ns 22.7ns 0 0 0 696 B
#7133 RunOnMethodBegin net472 1.46μs 1.66ns 6.42ns 0.104 0 0 658 B

pr-commenter[bot] avatar Jun 23 '25 19:06 pr-commenter[bot]

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7133) - mean (77ms)  : 75, 79
     .   : milestone, 77,
    master - mean (76ms)  : 74, 78
     .   : milestone, 76,

    section Baseline
    This PR (7133) - mean (72ms)  : 69, 76
     .   : milestone, 72,
    master - mean (72ms)  : 69, 75
     .   : milestone, 72,

    section CallTarget+Inlining+NGEN
    This PR (7133) - mean (1,047ms)  : 1019, 1074
     .   : milestone, 1047,
    master - mean (1,039ms)  : 1014, 1063
     .   : milestone, 1039,


gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7133) - mean (111ms)  : 109, 113
     .   : milestone, 111,
    master - mean (110ms)  : 108, 112
     .   : milestone, 110,

    section Baseline
    This PR (7133) - mean (110ms)  : 107, 113
     .   : milestone, 110,
    master - mean (109ms)  : 106, 112
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (7133) - mean (726ms)  : 708, 744
     .   : milestone, 726,
    master - mean (725ms)  : 701, 750
     .   : milestone, 725,


gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7133) - mean (98ms)  : 96, 100
     .   : milestone, 98,
    master - mean (97ms)  : 95, 99
     .   : milestone, 97,

    section Baseline
    This PR (7133) - mean (97ms)  : 95, 100
     .   : milestone, 97,
    master - mean (97ms)  : 95, 99
     .   : milestone, 97,

    section CallTarget+Inlining+NGEN
    This PR (7133) - mean (685ms)  : 666, 703
     .   : milestone, 685,
    master - mean (684ms)  : 667, 701
     .   : milestone, 684,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7133) - mean (205ms)  : 197, 214
     .   : milestone, 205,
    master - mean (195ms)  : 189, 202
     .   : milestone, 195,

    section Baseline
    This PR (7133) - mean (201ms)  : 192, 209
     .   : milestone, 201,
    master - mean (194ms)  : 187, 201
     .   : milestone, 194,

    section CallTarget+Inlining+NGEN
    This PR (7133) - mean (1,168ms)  : 1138, 1198
     .   : milestone, 1168,
    master - mean (1,151ms)  : 1093, 1209
     .   : milestone, 1151,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7133) - mean (278ms)  : 272, 285
     .   : milestone, 278,
    master - mean (276ms)  : 272, 281
     .   : milestone, 276,

    section Baseline
    This PR (7133) - mean (278ms)  : 271, 284
     .   : milestone, 278,
    master - mean (278ms)  : 270, 286
     .   : milestone, 278,

    section CallTarget+Inlining+NGEN
    This PR (7133) - mean (918ms)  : 890, 945
     .   : milestone, 918,
    master - mean (916ms)  : 888, 944
     .   : milestone, 916,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7133) - mean (279ms)  : 264, 294
     .   : milestone, 279,
    master - mean (271ms)  : 266, 277
     .   : milestone, 271,

    section Baseline
    This PR (7133) - mean (273ms)  : 262, 284
     .   : milestone, 273,
    master - mean (272ms)  : 266, 277
     .   : milestone, 272,

    section CallTarget+Inlining+NGEN
    This PR (7133) - mean (911ms)  : 864, 957
     .   : milestone, 911,
    master - mean (924ms)  : 867, 982
     .   : milestone, 924,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7133) - mean (284ms)  : 263, 305
     .   : milestone, 284,
    master - mean (272ms)  : 266, 278
     .   : milestone, 272,

    section Baseline
    This PR (7133) - mean (280ms)  : 265, 296
     .   : milestone, 280,
    master - mean (273ms)  : 260, 287
     .   : milestone, 273,

    section CallTarget+Inlining+NGEN
    This PR (7133) - mean (814ms)  : 779, 849
     .   : milestone, 814,
    master - mean (810ms)  : 786, 834
     .   : milestone, 810,

dd-trace-dotnet-ci-bot[bot] avatar Jul 02 '25 21:07 dd-trace-dotnet-ci-bot[bot]

/merge

robcarlan-datadog avatar Jul 10 '25 18:07 robcarlan-datadog

View all feedbacks in Devflow UI.

2025-07-10 18:18:21 UTC :information_source: Start processing command /merge

2025-07-10 18:18:29 UTC :information_source: MergeQueue: queue is disabled

Added to the queue but the mergequeue is not enabled for now.

2025-07-10 18:18:40 UTC :information_source: MergeQueue: This merge request was already merged

This pull request was merged directly.

dd-devflow-routing-codex[bot] avatar Jul 10 '25 18:07 dd-devflow-routing-codex[bot]