dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard

[Test Optimization] Override trace enabled env-var on CI Visibility mode.

Open tonyredondo opened this issue 6 months ago • 4 comments

Summary of changes

This PR overrides the trace enabled settings when we are in CI Visibility mode.

Reason for change

DD_TRACE_ENABLED=false shouldn't disable CI Visibility.

Implementation details

Test coverage

Other details

tonyredondo avatar May 09 '25 21:05 tonyredondo

Benchmarks

Benchmarks Report for benchmark platform :snail:

Benchmarks for #6951 compared to master:

  • All benchmarks have the same speed
  • 4 benchmarks have fewer allocations
  • 4 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 10.7μs 57.6ns 305ns 0 0 0 5.53 KB
master StartStopWithChild netcoreapp3.1 13.7μs 69ns 301ns 0 0 0 5.76 KB
master StartStopWithChild net472 21.8μs 125ns 1μs 0.983 0.328 0.109 6.12 KB
#6951 StartStopWithChild net6.0 10.3μs 57ns 342ns 0 0 0 5.55 KB
#6951 StartStopWithChild netcoreapp3.1 14.3μs 69.7ns 279ns 0 0 0 5.74 KB
#6951 StartStopWithChild net472 21.9μs 112ns 524ns 0.992 0.331 0.11 6.13 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada:

Fewer allocations :tada: in #6951

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 3.35 KB 3.31 KB -46 B -1.37%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 943μs 50.7ns 196ns 0 0 0 2.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.07ms 228ns 882ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.24ms 544ns 2.11μs 0 0 0 3.35 KB
#6951 WriteAndFlushEnrichedTraces net6.0 935μs 65.9ns 238ns 0 0 0 2.7 KB
#6951 WriteAndFlushEnrichedTraces netcoreapp3.1 1.02ms 421ns 1.63μs 0 0 0 2.7 KB
#6951 WriteAndFlushEnrichedTraces net472 1.24ms 117ns 453ns 0 0 0 3.31 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 329μs 1.61μs 6.42μs 0 0 0 197.06 KB
master AllCycleSimpleBody netcoreapp3.1 493μs 1.45μs 5.63μs 0 0 0 204.77 KB
master AllCycleSimpleBody net472 436μs 83.6ns 313ns 36.6 2.16 0 236.35 KB
master AllCycleMoreComplexBody net6.0 339μs 1.4μs 5.42μs 0 0 0 200.56 KB
master AllCycleMoreComplexBody netcoreapp3.1 477μs 2.05μs 7.94μs 0 0 0 208.18 KB
master AllCycleMoreComplexBody net472 448μs 281ns 1.09μs 37.9 2.23 0 239.88 KB
master ObjectExtractorSimpleBody net6.0 311ns 1.71ns 10.1ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 397ns 2.01ns 9.44ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 301ns 0.0308ns 0.111ns 0.0441 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.3μs 30.4ns 118ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.62μs 38.6ns 168ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.72μs 2ns 7.75ns 0.57 0 0 3.8 KB
#6951 AllCycleSimpleBody net6.0 330μs 1.6μs 6.98μs 0 0 0 197.41 KB
#6951 AllCycleSimpleBody netcoreapp3.1 495μs 1.69μs 6.54μs 0 0 0 205.17 KB
#6951 AllCycleSimpleBody net472 443μs 126ns 488ns 36.6 2.16 0 236.95 KB
#6951 AllCycleMoreComplexBody net6.0 342μs 1.81μs 9.74μs 0 0 0 200.92 KB
#6951 AllCycleMoreComplexBody netcoreapp3.1 516μs 2.22μs 8.62μs 0 0 0 208.58 KB
#6951 AllCycleMoreComplexBody net472 446μs 168ns 628ns 37.9 2.23 0 240.46 KB
#6951 ObjectExtractorSimpleBody net6.0 320ns 0.188ns 0.73ns 0 0 0 280 B
#6951 ObjectExtractorSimpleBody netcoreapp3.1 403ns 1.25ns 4.85ns 0 0 0 272 B
#6951 ObjectExtractorSimpleBody net472 305ns 0.0348ns 0.135ns 0.0435 0 0 281 B
#6951 ObjectExtractorMoreComplexBody net6.0 6.32μs 2.32ns 8.97ns 0 0 0 3.78 KB
#6951 ObjectExtractorMoreComplexBody netcoreapp3.1 7.7μs 28.6ns 111ns 0 0 0 3.69 KB
#6951 ObjectExtractorMoreComplexBody net472 6.7μs 4.67ns 17.5ns 0.572 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
master EncodeArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master EncodeArgs net472 N/A N/A N/A NaN NaN NaN 0 b
master EncodeLegacyArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
master EncodeLegacyArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master EncodeLegacyArgs net472 N/A N/A N/A NaN NaN NaN 0 b
#6951 EncodeArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
#6951 EncodeArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#6951 EncodeArgs net472 N/A N/A N/A NaN NaN NaN 0 b
#6951 EncodeLegacyArgs net6.0 N/A N/A N/A NaN NaN NaN 0 b
#6951 EncodeLegacyArgs netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#6951 EncodeLegacyArgs net472 N/A N/A N/A NaN NaN NaN 0 b
Benchmarks.Trace.Asm.AppSecWafBenchmark - Unknown :shrug: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmark netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmark net472 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmarkWithAttack net6.0 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master RunWafRealisticBenchmarkWithAttack net472 N/A N/A N/A NaN NaN NaN 0 b
#6951 RunWafRealisticBenchmark net6.0 N/A N/A N/A NaN NaN NaN 0 b
#6951 RunWafRealisticBenchmark netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#6951 RunWafRealisticBenchmark net472 N/A N/A N/A NaN NaN NaN 0 b
#6951 RunWafRealisticBenchmarkWithAttack net6.0 N/A N/A N/A NaN NaN NaN 0 b
#6951 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#6951 RunWafRealisticBenchmarkWithAttack net472 N/A N/A N/A NaN NaN NaN 0 b
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 61.7μs 45.7ns 158ns 0 0 0 14.53 KB
master SendRequest netcoreapp3.1 71.7μs 34.9ns 131ns 0 0 0 17.42 KB
master SendRequest net472 0.026ns 0.002ns 0.00775ns 0 0 0 0 b
#6951 SendRequest net6.0 59.9μs 50.6ns 182ns 0 0 0 14.53 KB
#6951 SendRequest netcoreapp3.1 70.4μs 83.1ns 311ns 0 0 0 17.42 KB
#6951 SendRequest net472 0.0185ns 0.00187ns 0.00723ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: More allocations :warning:

More allocations :warning: in #6951

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.87 KB 42.17 KB 295 B 0.70%

Fewer allocations :tada: in #6951

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 56.53 KB 56.01 KB -512 B -0.91%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 636μs 868ns 3.36μs 0 0 0 41.72 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 677μs 3.73μs 23.3μs 0 0 0 41.87 KB
master WriteAndFlushEnrichedTraces net472 889μs 4.46μs 19.5μs 8.33 0 0 56.53 KB
#6951 WriteAndFlushEnrichedTraces net6.0 685μs 1.48μs 5.73μs 0 0 0 41.71 KB
#6951 WriteAndFlushEnrichedTraces netcoreapp3.1 681μs 923ns 3.45μs 0 0 0 42.17 KB
#6951 WriteAndFlushEnrichedTraces net472 949μs 3.79μs 15.2μs 8.33 0 0 56.01 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.95μs 7.3ns 28.3ns 0 0 0 1.03 KB
master ExecuteNonQuery netcoreapp3.1 2.46μs 8.67ns 33.6ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.9μs 3.09ns 12ns 0.158 0.0143 0 995 B
#6951 ExecuteNonQuery net6.0 1.95μs 10.1ns 46.1ns 0 0 0 1.03 KB
#6951 ExecuteNonQuery netcoreapp3.1 2.59μs 6.38ns 24.7ns 0 0 0 1.02 KB
#6951 ExecuteNonQuery net472 2.83μs 4.98ns 19.3ns 0.156 0.0142 0 995 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.75μs 8.81ns 34.1ns 0 0 0 1.04 KB
master CallElasticsearch netcoreapp3.1 2.25μs 11.2ns 48.9ns 0 0 0 1.04 KB
master CallElasticsearch net472 3.45μs 4.1ns 15.9ns 0.157 0 0 1.05 KB
master CallElasticsearchAsync net6.0 1.82μs 7.68ns 28.7ns 0 0 0 1.02 KB
master CallElasticsearchAsync netcoreapp3.1 2.43μs 11.1ns 44.5ns 0 0 0 1.09 KB
master CallElasticsearchAsync net472 3.75μs 3.03ns 11.7ns 0.168 0 0 1.11 KB
#6951 CallElasticsearch net6.0 1.74μs 9.7ns 50.4ns 0 0 0 1.04 KB
#6951 CallElasticsearch netcoreapp3.1 2.23μs 12.2ns 67.6ns 0 0 0 1.04 KB
#6951 CallElasticsearch net472 3.48μs 3.73ns 14.5ns 0.156 0 0 1.05 KB
#6951 CallElasticsearchAsync net6.0 1.9μs 0.431ns 1.61ns 0 0 0 1.02 KB
#6951 CallElasticsearchAsync netcoreapp3.1 2.33μs 10.7ns 38.6ns 0 0 0 1.09 KB
#6951 CallElasticsearchAsync net472 3.55μs 2.73ns 9.84ns 0.16 0 0 1.11 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.81μs 2.09ns 8.1ns 0 0 0 960 B
master ExecuteAsync netcoreapp3.1 2.28μs 1.73ns 6ns 0 0 0 960 B
master ExecuteAsync net472 2.68μs 5.65ns 21.9ns 0.147 0 0 923 B
#6951 ExecuteAsync net6.0 1.88μs 9.12ns 37.6ns 0 0 0 960 B
#6951 ExecuteAsync netcoreapp3.1 2.3μs 8.15ns 31.5ns 0 0 0 960 B
#6951 ExecuteAsync net472 2.53μs 1.86ns 6.71ns 0.139 0 0 923 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 7.08μs 19.9ns 77.2ns 0 0 0 2.37 KB
master SendAsync netcoreapp3.1 8.6μs 9.04ns 35ns 0 0 0 2.9 KB
master SendAsync net472 12.3μs 9.97ns 38.6ns 0.491 0 0 3.19 KB
#6951 SendAsync net6.0 6.97μs 9.43ns 35.3ns 0 0 0 2.37 KB
#6951 SendAsync netcoreapp3.1 8.48μs 27.2ns 105ns 0 0 0 2.9 KB
#6951 SendAsync net472 12.5μs 7.39ns 28.6ns 0.501 0 0 3.19 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed :heavy_check_mark: More allocations :warning:

More allocations :warning: in #6951

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 260.98 KB 277.58 KB 16.6 KB 6.36%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 42.64 KB 43.62 KB 976 B 2.29%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 43.82 KB 44.37 KB 552 B 1.26%

Fewer allocations :tada: in #6951

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 286.72 KB 278.53 KB -8.19 KB -2.86%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 274.12 KB 255.98 KB -18.14 KB -6.62%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 41.9μs 223ns 1.2μs 0 0 0 43.82 KB
master StringConcatBenchmark netcoreapp3.1 49.7μs 290ns 2.34μs 0 0 0 42.64 KB
master StringConcatBenchmark net472 56.9μs 154ns 555ns 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 482μs 906ns 3.14μs 0 0 0 260.98 KB
master StringConcatAspectBenchmark netcoreapp3.1 505μs 2.28μs 8.53μs 0 0 0 274.12 KB
master StringConcatAspectBenchmark net472 409μs 1.37μs 4.95μs 0 0 0 286.72 KB
#6951 StringConcatBenchmark net6.0 43μs 213ns 879ns 0 0 0 44.37 KB
#6951 StringConcatBenchmark netcoreapp3.1 46.7μs 204ns 1.12μs 0 0 0 43.62 KB
#6951 StringConcatBenchmark net472 56.6μs 148ns 535ns 0 0 0 57.34 KB
#6951 StringConcatAspectBenchmark net6.0 497μs 2.23μs 7.74μs 0 0 0 277.58 KB
#6951 StringConcatAspectBenchmark netcoreapp3.1 500μs 2.26μs 8.46μs 0 0 0 255.98 KB
#6951 StringConcatAspectBenchmark net472 414μs 2.28μs 13.9μs 0 0 0 278.53 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.67μs 7.54ns 29.2ns 0 0 0 1.76 KB
master EnrichedLog netcoreapp3.1 3.5μs 7.98ns 30.9ns 0 0 0 1.76 KB
master EnrichedLog net472 3.91μs 2.47ns 9.58ns 0.254 0 0 1.69 KB
#6951 EnrichedLog net6.0 2.62μs 4.82ns 18.7ns 0 0 0 1.76 KB
#6951 EnrichedLog netcoreapp3.1 3.44μs 11.6ns 45ns 0 0 0 1.76 KB
#6951 EnrichedLog net472 3.98μs 3.24ns 12.5ns 0.259 0 0 1.69 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 124μs 106ns 412ns 0 0 0 4.37 KB
master EnrichedLog netcoreapp3.1 128μs 54.6ns 197ns 0 0 0 4.37 KB
master EnrichedLog net472 166μs 112ns 434ns 0 0 0 4.57 KB
#6951 EnrichedLog net6.0 123μs 150ns 562ns 0 0 0 4.37 KB
#6951 EnrichedLog netcoreapp3.1 128μs 249ns 933ns 0 0 0 4.37 KB
#6951 EnrichedLog net472 167μs 132ns 512ns 0 0 0 4.57 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.99μs 6.45ns 25ns 0 0 0 2.32 KB
master EnrichedLog netcoreapp3.1 6.87μs 19.5ns 75.6ns 0 0 0 2.32 KB
master EnrichedLog net472 7.36μs 10.4ns 40.5ns 0.327 0 0 2.14 KB
#6951 EnrichedLog net6.0 5.04μs 3.98ns 15.4ns 0 0 0 2.32 KB
#6951 EnrichedLog netcoreapp3.1 6.78μs 15.3ns 59.1ns 0 0 0 2.32 KB
#6951 EnrichedLog net472 7.37μs 8.41ns 32.6ns 0.334 0 0 2.14 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2μs 9.17ns 36.7ns 0 0 0 1.21 KB
master SendReceive netcoreapp3.1 2.57μs 10.1ns 39.2ns 0 0 0 1.21 KB
master SendReceive net472 3.03μs 3.28ns 12.7ns 0.18 0 0 1.21 KB
#6951 SendReceive net6.0 2.01μs 9.26ns 37ns 0 0 0 1.21 KB
#6951 SendReceive netcoreapp3.1 2.64μs 2.97ns 11.5ns 0 0 0 1.21 KB
#6951 SendReceive net472 3.25μs 4.09ns 15.9ns 0.179 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.23μs 8.89ns 33.2ns 0 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 5.55μs 20.7ns 80ns 0 0 0 1.69 KB
master EnrichedLog net472 6.52μs 5.26ns 19.7ns 0.325 0 0 2.08 KB
#6951 EnrichedLog net6.0 4.25μs 18.5ns 71.8ns 0 0 0 1.64 KB
#6951 EnrichedLog netcoreapp3.1 5.67μs 10.8ns 42ns 0 0 0 1.69 KB
#6951 EnrichedLog net472 6.78μs 13.5ns 50.6ns 0.304 0 0 2.08 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 752ns 4.07ns 21.1ns 0 0 0 584 B
master StartFinishSpan netcoreapp3.1 938ns 4.37ns 16.9ns 0 0 0 584 B
master StartFinishSpan net472 911ns 0.311ns 1.2ns 0.0923 0 0 586 B
master StartFinishScope net6.0 898ns 4.19ns 16.2ns 0 0 0 704 B
master StartFinishScope netcoreapp3.1 1.18μs 5.49ns 21.3ns 0 0 0 704 B
master StartFinishScope net472 1.13μs 0.236ns 0.915ns 0.102 0 0 666 B
#6951 StartFinishSpan net6.0 743ns 4.06ns 20.3ns 0 0 0 584 B
#6951 StartFinishSpan netcoreapp3.1 948ns 5.03ns 25.6ns 0 0 0 584 B
#6951 StartFinishSpan net472 905ns 0.256ns 0.991ns 0.0911 0 0 586 B
#6951 StartFinishScope net6.0 898ns 0.399ns 1.55ns 0 0 0 704 B
#6951 StartFinishScope netcoreapp3.1 1.15μs 5.74ns 25ns 0 0 0 704 B
#6951 StartFinishScope net472 1.13μs 0.169ns 0.631ns 0.102 0 0 666 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.04μs 0.517ns 2ns 0 0 0 704 B
master RunOnMethodBegin netcoreapp3.1 1.35μs 6.15ns 23.8ns 0 0 0 704 B
master RunOnMethodBegin net472 1.35μs 0.798ns 2.99ns 0.102 0 0 666 B
#6951 RunOnMethodBegin net6.0 998ns 5.34ns 26.7ns 0 0 0 704 B
#6951 RunOnMethodBegin netcoreapp3.1 1.32μs 6.17ns 23.9ns 0 0 0 704 B
#6951 RunOnMethodBegin net472 1.38μs 0.728ns 2.82ns 0.103 0 0 666 B

pr-commenter[bot] avatar May 09 '25 22:05 pr-commenter[bot]

Datadog Report

All test runs d2f83c3 :link:

:x: 2 Total Test Services: 1 Failed, 1 Passed

Test Services
Service Name Failed Known Flaky New Flaky Passed Skipped Total Time Test Service View
dd-trace-dotnet 1 0 0 244474 2255 17h 46m 13.27s Link
exploration_tests 0 0 0 22085 3 2m 11.19s Link

:x: Failed Tests (1)

  • InjectsLogsWhenEnabled - Datadog.Trace.ClrProfiler.IntegrationTests.SerilogTests - Details

    Expand for error
    xpected exit code: 0, actual exit code: 139.
    

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6951) - mean (68ms)  : 65, 70
     .   : milestone, 68,
    master - mean (68ms)  : 65, 70
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (6951) - mean (1,004ms)  : 980, 1028
     .   : milestone, 1004,
    master - mean (1,000ms)  : 980, 1020
     .   : milestone, 1000,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6951) - mean (102ms)  : 100, 104
     .   : milestone, 102,
    master - mean (102ms)  : 100, 105
     .   : milestone, 102,

    section CallTarget+Inlining+NGEN
    This PR (6951) - mean (696ms)  : 680, 712
     .   : milestone, 696,
    master - mean (695ms)  : 672, 717
     .   : milestone, 695,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6951) - mean (89ms)  : 87, 92
     .   : milestone, 89,
    master - mean (89ms)  : 87, 91
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (6951) - mean (657ms)  : 637, 676
     .   : milestone, 657,
    master - mean (651ms)  : 633, 668
     .   : milestone, 651,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6951) - mean (191ms)  : 185, 197
     .   : milestone, 191,
    master - mean (190ms)  : 185, 194
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6951) - mean (1,112ms)  : 1084, 1139
     .   : milestone, 1112,
    master - mean (1,107ms)  : 1084, 1130
     .   : milestone, 1107,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6951) - mean (269ms)  : 265, 273
     .   : milestone, 269,
    master - mean (268ms)  : 262, 274
     .   : milestone, 268,

    section CallTarget+Inlining+NGEN
    This PR (6951) - mean (879ms)  : 847, 912
     .   : milestone, 879,
    master - mean (879ms)  : 844, 913
     .   : milestone, 879,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6951) - mean (261ms)  : 256, 266
     .   : milestone, 261,
    master - mean (260ms)  : 256, 264
     .   : milestone, 260,

    section CallTarget+Inlining+NGEN
    This PR (6951) - mean (868ms)  : 834, 902
     .   : milestone, 868,
    master - mean (869ms)  : 836, 902
     .   : milestone, 869,

andrewlock avatar May 09 '25 22:05 andrewlock

Datadog Summary

✅ Code Quality    ✅ Code Security    ✅ Dependencies


Was this helpful? Give us feedback!