dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard

Add helper for taking memory dump in CI on mutex timeout

Open andrewlock opened this issue 2 weeks ago • 5 comments

Summary of changes

Adds a helper, WaitOrDump which waits for a mutex, and takes a memory dump if it fails

Reason for change

We've been seeing unexpected timeouts of mutexes in CI, and we want to get to the bottom of them

Implementation details

Use the existing MemoryDumpHelper.CaptureMemoryDump() to take a memory dump of the test process. I kept the signature essentially the same as Wait for ease of replacing calls. We can pass in Output in the future for more details, just doing the minimum right now

Test coverage

This is the test - if we don't see any failures, I'll retry until we do!

Other details

andrewlock avatar Dec 09 '25 11:12 andrewlock

⚠️ Tests

Fix all issues with Cursor

⚠️ Warnings

🧪 8 Tests failed

CallFlushAutomatically from Datadog.Trace.Tests.Agent.StatsAggregatorTests (Datadog) (Fix with Cursor)
Expected success to be True, but found False.
CallFlushAutomatically from Datadog.Trace.Tests.Agent.StatsAggregatorTests (Datadog) (Fix with Cursor)
Expected success to be True, but found False.
CallFlushAutomatically from Datadog.Trace.Tests.Agent.StatsAggregatorTests (Datadog) (Fix with Cursor)
Expected success to be True, but found False.
View all

ℹ️ Info

❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: c82d7ef | Docs | Datadog PR Page | Was this helpful? Give us feedback!

Benchmarks

Benchmarks Report for benchmark platform :snail:

Benchmarks for #7926 compared to master:

  • 2 benchmarks are slower, with geometric mean 1.448
  • 1 benchmarks have fewer allocations
  • 2 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 10.6μs 60.9ns 447ns 0 0 0 5.51 KB
master StartStopWithChild netcoreapp3.1 14μs 74.1ns 363ns 0 0 0 5.71 KB
master StartStopWithChild net472 22.2μs 115ns 540ns 0.944 0.21 0 5.99 KB
#7926 StartStopWithChild net6.0 11.1μs 58.3ns 286ns 0 0 0 5.51 KB
#7926 StartStopWithChild netcoreapp3.1 15.2μs 48.1ns 186ns 0 0 0 5.69 KB
#7926 StartStopWithChild net472 21.8μs 117ns 639ns 1.01 0.335 0.112 5.97 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 1.27ms 481ns 1.86μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.37ms 250ns 902ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.7ms 3.13μs 12.1μs 0 0 0 3.26 KB
#7926 WriteAndFlushEnrichedTraces net6.0 1.27ms 190ns 736ns 0 0 0 2.7 KB
#7926 WriteAndFlushEnrichedTraces netcoreapp3.1 1.4ms 104ns 402ns 0 0 0 2.7 KB
#7926 WriteAndFlushEnrichedTraces net472 1.7ms 320ns 1.11μs 0 0 0 3.26 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 1.11μs 0.365ns 1.41ns 0 0 0 1.22 KB
master AllCycleSimpleBody netcoreapp3.1 1.37μs 7.82ns 58ns 0 0 0 1.2 KB
master AllCycleSimpleBody net472 1.01μs 0.974ns 3.77ns 0.192 0 0 1.23 KB
master AllCycleMoreComplexBody net6.0 7.51μs 39.2ns 204ns 0 0 0 4.72 KB
master AllCycleMoreComplexBody netcoreapp3.1 9μs 43.8ns 191ns 0 0 0 4.62 KB
master AllCycleMoreComplexBody net472 7.62μs 2.48ns 8.95ns 0.724 0 0 4.74 KB
master ObjectExtractorSimpleBody net6.0 353ns 1.98ns 12.5ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 392ns 2.13ns 12.2ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 308ns 0.00934ns 0.0324ns 0.0434 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.41μs 34.4ns 188ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.85μs 37.9ns 161ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.7μs 6.73ns 26.1ns 0.573 0 0 3.8 KB
#7926 AllCycleSimpleBody net6.0 1.09μs 6.29ns 41.7ns 0 0 0 1.22 KB
#7926 AllCycleSimpleBody netcoreapp3.1 1.38μs 7.74ns 53.6ns 0 0 0 1.2 KB
#7926 AllCycleSimpleBody net472 1.02μs 1.34ns 5.18ns 0.193 0 0 1.23 KB
#7926 AllCycleMoreComplexBody net6.0 7.34μs 36.5ns 159ns 0 0 0 4.72 KB
#7926 AllCycleMoreComplexBody netcoreapp3.1 8.94μs 39ns 151ns 0 0 0 4.62 KB
#7926 AllCycleMoreComplexBody net472 7.69μs 9.54ns 37ns 0.73 0 0 4.74 KB
#7926 ObjectExtractorSimpleBody net6.0 333ns 1.81ns 7.67ns 0 0 0 280 B
#7926 ObjectExtractorSimpleBody netcoreapp3.1 395ns 2.11ns 11.6ns 0 0 0 272 B
#7926 ObjectExtractorSimpleBody net472 300ns 0.0158ns 0.0612ns 0.0439 0 0 281 B
#7926 ObjectExtractorMoreComplexBody net6.0 6.47μs 31.6ns 141ns 0 0 0 3.78 KB
#7926 ObjectExtractorMoreComplexBody netcoreapp3.1 7.67μs 39.7ns 186ns 0 0 0 3.69 KB
#7926 ObjectExtractorMoreComplexBody net472 6.74μs 4.99ns 18.7ns 0.573 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 75.6μs 20.9ns 81.1ns 0 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 97.6μs 275ns 1.03μs 0 0 0 32.4 KB
master EncodeArgs net472 108μs 12.3ns 47.5ns 4.88 0 0 32.5 KB
master EncodeLegacyArgs net6.0 144μs 326ns 1.26μs 0 0 0 2.14 KB
master EncodeLegacyArgs netcoreapp3.1 199μs 137ns 529ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 264μs 37.1ns 144ns 0 0 0 2.14 KB
#7926 EncodeArgs net6.0 76.7μs 294ns 1.1μs 0 0 0 32.4 KB
#7926 EncodeArgs netcoreapp3.1 97μs 352ns 1.36μs 0 0 0 32.4 KB
#7926 EncodeArgs net472 108μs 36.7ns 132ns 4.87 0 0 32.5 KB
#7926 EncodeLegacyArgs net6.0 143μs 360ns 1.39μs 0 0 0 2.14 KB
#7926 EncodeLegacyArgs netcoreapp3.1 198μs 77.3ns 299ns 0 0 0 2.14 KB
#7926 EncodeLegacyArgs net472 264μs 20ns 74.8ns 0 0 0 2.14 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 421μs 1.21μs 4.2μs 0 0 0 4.55 KB
master RunWafRealisticBenchmark netcoreapp3.1 438μs 1.31μs 4.54μs 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 486μs 378ns 1.36μs 0 0 0 0 b
master RunWafRealisticBenchmarkWithAttack net6.0 302μs 871ns 3.14μs 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 340μs 3.31μs 31.4μs 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 363μs 195ns 704ns 0 0 0 0 b
#7926 RunWafRealisticBenchmark net6.0 418μs 2.16μs 10.8μs 0 0 0 4.55 KB
#7926 RunWafRealisticBenchmark netcoreapp3.1 474μs 4.83μs 44μs 0 0 0 4.48 KB
#7926 RunWafRealisticBenchmark net472 485μs 279ns 1.01μs 0 0 0 0 b
#7926 RunWafRealisticBenchmarkWithAttack net6.0 303μs 664ns 2.4μs 0 0 0 2.24 KB
#7926 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 345μs 3.22μs 31.3μs 0 0 0 2.22 KB
#7926 RunWafRealisticBenchmarkWithAttack net472 363μs 447ns 1.61μs 0 0 0 0 b
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 85μs 263ns 1.02μs 0 0 0 18.97 KB
master SendRequest netcoreapp3.1 96.8μs 502ns 2.97μs 0 0 0 21.18 KB
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#7926 SendRequest net6.0 85.1μs 398ns 1.54μs 0 0 0 18.98 KB
#7926 SendRequest netcoreapp3.1 99.2μs 464ns 1.8μs 0 0 0 21.18 KB
#7926 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
Benchmarks.Trace.CharSliceBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #7926

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice‑netcoreapp3.1 1.828 2,163,000.00 3,953,100.00

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master OriginalCharSlice net6.0 1.9ms 767ns 2.87μs 0 0 0 640 KB
master OriginalCharSlice netcoreapp3.1 2.18ms 6.86μs 43.4μs 0 0 0 640 KB
master OriginalCharSlice net472 2.61ms 503ns 1.88μs 0 0 0 638.98 KB
master OptimizedCharSlice net6.0 1.41ms 514ns 1.92μs 0 0 0 0 b
master OptimizedCharSlice netcoreapp3.1 2.73ms 1.09μs 4.06μs 0 0 0 0 b
master OptimizedCharSlice net472 1.92ms 873ns 3.38μs 0 0 0 0 b
master OptimizedCharSliceWithPool net6.0 1.06ms 481ns 1.73μs 0 0 0 0 b
master OptimizedCharSliceWithPool netcoreapp3.1 1.96ms 1.82μs 6.83μs 0 0 0 0 b
master OptimizedCharSliceWithPool net472 1.14ms 823ns 3.19μs 0 0 0 0 b
#7926 OriginalCharSlice net6.0 1.94ms 1.87μs 7.26μs 0 0 0 640 KB
#7926 OriginalCharSlice netcoreapp3.1 3.95ms 1.35μs 4.88μs 0 0 0 640.05 KB
#7926 OriginalCharSlice net472 2.61ms 815ns 3.16μs 0 0 0 638.98 KB
#7926 OptimizedCharSlice net6.0 1.43ms 920ns 3.32μs 0 0 0 0 b
#7926 OptimizedCharSlice netcoreapp3.1 2.76ms 1.38μs 5.15μs 0 0 0 0 b
#7926 OptimizedCharSlice net472 1.91ms 633ns 2.28μs 0 0 0 0 b
#7926 OptimizedCharSliceWithPool net6.0 1.01ms 418ns 1.45μs 0 0 0 0 b
#7926 OptimizedCharSliceWithPool netcoreapp3.1 1.94ms 1.12μs 4.19μs 0 0 0 0 b
#7926 OptimizedCharSliceWithPool net472 1.14ms 453ns 1.7μs 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 630μs 1.19μs 4.63μs 0 0 0 41.67 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 685μs 3.98μs 33.5μs 0 0 0 41.92 KB
master WriteAndFlushEnrichedTraces net472 919μs 3.53μs 13.2μs 8.33 4.17 0 56.05 KB
#7926 WriteAndFlushEnrichedTraces net6.0 648μs 983ns 3.68μs 0 0 0 41.65 KB
#7926 WriteAndFlushEnrichedTraces netcoreapp3.1 718μs 4.09μs 30.3μs 0 0 0 41.84 KB
#7926 WriteAndFlushEnrichedTraces net472 826μs 2.62μs 10.2μs 8.33 4.17 0 56.08 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.9μs 0.352ns 1.32ns 0 0 0 968 B
master ExecuteNonQuery netcoreapp3.1 2.59μs 8.08ns 31.3ns 0 0 0 960 B
master ExecuteNonQuery net472 2.75μs 4.48ns 16.8ns 0.137 0 0 931 B
#7926 ExecuteNonQuery net6.0 1.94μs 1.45ns 5.43ns 0 0 0 968 B
#7926 ExecuteNonQuery netcoreapp3.1 2.61μs 11.7ns 45.3ns 0 0 0 960 B
#7926 ExecuteNonQuery net472 2.73μs 2.44ns 9.45ns 0.137 0 0 931 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.68μs 6.24ns 24.2ns 0 0 0 952 B
master CallElasticsearch netcoreapp3.1 2.21μs 10.9ns 44.8ns 0 0 0 968 B
master CallElasticsearch net472 3.58μs 4.73ns 18.3ns 0.139 0 0 955 B
master CallElasticsearchAsync net6.0 1.72μs 8.32ns 35.3ns 0 0 0 928 B
master CallElasticsearchAsync netcoreapp3.1 2.32μs 8.82ns 34.2ns 0 0 0 1.02 KB
master CallElasticsearchAsync net472 3.42μs 2.28ns 8.23ns 0.154 0 0 1.01 KB
#7926 CallElasticsearch net6.0 1.68μs 8.44ns 37.7ns 0 0 0 952 B
#7926 CallElasticsearch netcoreapp3.1 2.16μs 10.7ns 46.7ns 0 0 0 968 B
#7926 CallElasticsearch net472 3.51μs 4ns 15.5ns 0.138 0 0 955 B
#7926 CallElasticsearchAsync net6.0 1.72μs 7.59ns 28.4ns 0 0 0 928 B
#7926 CallElasticsearchAsync netcoreapp3.1 2.29μs 6.53ns 25.3ns 0 0 0 1.02 KB
#7926 CallElasticsearchAsync net472 3.5μs 4.74ns 18.4ns 0.157 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.8μs 7.17ns 26.8ns 0 0 0 896 B
master ExecuteAsync netcoreapp3.1 2.34μs 11ns 41.3ns 0 0 0 896 B
master ExecuteAsync net472 2.52μs 1.31ns 5.08ns 0.126 0 0 858 B
#7926 ExecuteAsync net6.0 1.8μs 3.15ns 11.8ns 0 0 0 896 B
#7926 ExecuteAsync netcoreapp3.1 2.44μs 3.85ns 14.4ns 0 0 0 896 B
#7926 ExecuteAsync net472 2.56μs 2.04ns 7.89ns 0.127 0 0 858 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 6.54μs 18.5ns 69.3ns 0 0 0 2.29 KB
master SendAsync netcoreapp3.1 8.49μs 26.8ns 104ns 0 0 0 2.83 KB
master SendAsync net472 12μs 9.41ns 36.4ns 0.484 0 0 3.08 KB
#7926 SendAsync net6.0 6.69μs 21.7ns 84ns 0 0 0 2.29 KB
#7926 SendAsync netcoreapp3.1 8.47μs 9.96ns 38.6ns 0 0 0 2.83 KB
#7926 SendAsync net472 11.9μs 9.18ns 35.6ns 0.474 0 0 3.08 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower :warning: More allocations :warning:

Slower :warning: in #7926

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 1.148 437,150.00 501,750.00

More allocations :warning: in #7926

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 246.35 KB 260.96 KB 14.61 KB 5.93%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 270.34 KB 272.44 KB 2.1 KB 0.78%

Fewer allocations :tada: in #7926

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 272.16 KB 257.89 KB -14.27 KB -5.24%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 44.6μs 239ns 1.72μs 0 0 0 42.51 KB
master StringConcatBenchmark netcoreapp3.1 48.2μs 259ns 1.32μs 0 0 0 42.54 KB
master StringConcatBenchmark net472 56.7μs 282ns 1.29μs 0 0 0 49.15 KB
master StringConcatAspectBenchmark net6.0 439μs 1.91μs 6.88μs 0 0 0 246.35 KB
master StringConcatAspectBenchmark netcoreapp3.1 518μs 1.88μs 7.51μs 0 0 0 272.16 KB
master StringConcatAspectBenchmark net472 395μs 2.03μs 9.74μs 0 0 0 270.34 KB
#7926 StringConcatBenchmark net6.0 44.7μs 248ns 1.61μs 0 0 0 42.51 KB
#7926 StringConcatBenchmark netcoreapp3.1 48.5μs 287ns 2.63μs 0 0 0 42.54 KB
#7926 StringConcatBenchmark net472 57.9μs 235ns 848ns 0 0 0 49.15 KB
#7926 StringConcatAspectBenchmark net6.0 502μs 2.27μs 8.48μs 0 0 0 260.96 KB
#7926 StringConcatAspectBenchmark netcoreapp3.1 507μs 2.71μs 15.1μs 0 0 0 257.89 KB
#7926 StringConcatAspectBenchmark net472 401μs 2.13μs 10.6μs 0 0 0 272.44 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.68μs 14ns 72.8ns 0 0 0 1.69 KB
master EnrichedLog netcoreapp3.1 3.51μs 17.3ns 71.5ns 0 0 0 1.7 KB
master EnrichedLog net472 3.93μs 3.67ns 14.2ns 0.237 0 0 1.6 KB
#7926 EnrichedLog net6.0 2.66μs 1.21ns 4.67ns 0 0 0 1.69 KB
#7926 EnrichedLog netcoreapp3.1 3.57μs 16.8ns 65.1ns 0 0 0 1.7 KB
#7926 EnrichedLog net472 3.84μs 2.15ns 8.33ns 0.248 0 0 1.6 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 123μs 47.8ns 172ns 0 0 0 4.31 KB
master EnrichedLog netcoreapp3.1 128μs 25.6ns 99.1ns 0 0 0 4.31 KB
master EnrichedLog net472 168μs 29.6ns 111ns 0 0 0 4.51 KB
#7926 EnrichedLog net6.0 123μs 67.7ns 253ns 0 0 0 4.31 KB
#7926 EnrichedLog netcoreapp3.1 127μs 127ns 477ns 0 0 0 4.31 KB
#7926 EnrichedLog net472 167μs 22.1ns 85.5ns 0 0 0 4.51 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.95μs 18ns 67.2ns 0 0 0 2.24 KB
master EnrichedLog netcoreapp3.1 6.79μs 20.8ns 80.4ns 0 0 0 2.26 KB
master EnrichedLog net472 7.61μs 6.45ns 25ns 0.304 0 0 2.05 KB
#7926 EnrichedLog net6.0 5.09μs 6.72ns 26ns 0 0 0 2.24 KB
#7926 EnrichedLog netcoreapp3.1 6.92μs 28.9ns 112ns 0 0 0 2.26 KB
#7926 EnrichedLog net472 7.61μs 8.1ns 31.4ns 0.304 0 0 2.05 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.95μs 9.74ns 43.6ns 0 0 0 1.12 KB
master SendReceive netcoreapp3.1 2.45μs 11.2ns 43.4ns 0 0 0 1.14 KB
master SendReceive net472 3.04μs 5.5ns 21.3ns 0.165 0 0 1.12 KB
#7926 SendReceive net6.0 1.99μs 0.799ns 2.99ns 0 0 0 1.12 KB
#7926 SendReceive netcoreapp3.1 2.48μs 11.9ns 46.3ns 0 0 0 1.14 KB
#7926 SendReceive net472 2.95μs 3.19ns 11.9ns 0.177 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.41μs 1.11ns 4.29ns 0 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 5.59μs 16.4ns 61.4ns 0 0 0 1.63 KB
master EnrichedLog net472 6.72μs 6.45ns 25ns 0.302 0 0 2.03 KB
#7926 EnrichedLog net6.0 4.53μs 10.2ns 38.3ns 0 0 0 1.58 KB
#7926 EnrichedLog netcoreapp3.1 5.77μs 11.9ns 45.9ns 0 0 0 1.63 KB
#7926 EnrichedLog net472 6.98μs 4.95ns 19.2ns 0.314 0 0 2.03 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 778ns 4.23ns 22.4ns 0 0 0 520 B
master StartFinishSpan netcoreapp3.1 936ns 4.32ns 16.1ns 0 0 0 520 B
master StartFinishSpan net472 883ns 0.151ns 0.584ns 0.0797 0 0 522 B
master StartFinishScope net6.0 906ns 0.526ns 1.97ns 0 0 0 640 B
master StartFinishScope netcoreapp3.1 1.1μs 5.99ns 33.9ns 0 0 0 640 B
master StartFinishScope net472 1.07μs 0.193ns 0.747ns 0.0911 0 0 602 B
master StartFinishTwoScopes net6.0 1.78μs 0.596ns 2.31ns 0 0 0 1.19 KB
master StartFinishTwoScopes netcoreapp3.1 2.19μs 11.8ns 52.8ns 0 0 0 1.19 KB
master StartFinishTwoScopes net472 2.11μs 1.68ns 6.49ns 0.17 0 0 1.08 KB
#7926 StartFinishSpan net6.0 757ns 3.48ns 13.5ns 0 0 0 520 B
#7926 StartFinishSpan netcoreapp3.1 940ns 4.74ns 20.1ns 0 0 0 520 B
#7926 StartFinishSpan net472 890ns 0.0943ns 0.365ns 0.0802 0 0 522 B
#7926 StartFinishScope net6.0 901ns 4.5ns 19.1ns 0 0 0 640 B
#7926 StartFinishScope netcoreapp3.1 1.12μs 5.86ns 29.3ns 0 0 0 640 B
#7926 StartFinishScope net472 1.13μs 1.96ns 7.6ns 0.0901 0 0 602 B
#7926 StartFinishTwoScopes net6.0 1.75μs 8.49ns 36ns 0 0 0 1.19 KB
#7926 StartFinishTwoScopes netcoreapp3.1 2.22μs 11.5ns 54.2ns 0 0 0 1.19 KB
#7926 StartFinishTwoScopes net472 2.14μs 1.37ns 5.32ns 0.171 0 0 1.08 KB
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.05μs 5.17ns 22.5ns 0 0 0 640 B
master RunOnMethodBegin netcoreapp3.1 1.33μs 4.1ns 15.9ns 0 0 0 640 B
master RunOnMethodBegin net472 1.42μs 0.633ns 2.45ns 0.0923 0 0 602 B
#7926 RunOnMethodBegin net6.0 1.03μs 5.49ns 29ns 0 0 0 640 B
#7926 RunOnMethodBegin netcoreapp3.1 1.33μs 2.17ns 8.41ns 0 0 0 640 B
#7926 RunOnMethodBegin net472 1.44μs 1.33ns 5.14ns 0.0937 0 0 602 B

pr-commenter[bot] avatar Dec 09 '25 12:12 pr-commenter[bot]

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing This PR (7926) and master.

✅ No regressions detected - check the details below

Full Metrics Comparison

FakeDbCommand

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration75.12 ± (75.07 - 75.83) ms75.51 ± (75.34 - 76.02) ms+0.5%✅⬆️
.NET Framework 4.8 - Bailout
duration83.45 ± (83.02 - 84.16) ms79.72 ± (79.46 - 80.18) ms-4.5%
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1070.34 ± (1070.32 - 1077.70) ms1069.93 ± (1068.52 - 1076.21) ms-0.0%
.NET Core 3.1 - Baseline
process.internal_duration_ms23.03 ± (22.95 - 23.10) ms22.76 ± (22.69 - 22.83) ms-1.1%
process.time_to_main_ms87.56 ± (87.20 - 87.92) ms85.93 ± (85.63 - 86.24) ms-1.9%
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%
runtime.dotnet.mem.committed10.94 ± (10.93 - 10.94) MB10.91 ± (10.91 - 10.91) MB-0.3%
runtime.dotnet.threads.count12 ± (12 - 12)12 ± (12 - 12)+0.0%
.NET Core 3.1 - Bailout
process.internal_duration_ms22.88 ± (22.81 - 22.95) ms22.88 ± (22.80 - 22.95) ms-0.0%
process.time_to_main_ms89.08 ± (88.69 - 89.46) ms89.00 ± (88.62 - 89.39) ms-0.1%
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%
runtime.dotnet.mem.committed10.99 ± (10.99 - 11.00) MB10.95 ± (10.95 - 10.96) MB-0.4%
runtime.dotnet.threads.count13 ± (13 - 13)13 ± (13 - 13)+0.0%
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms221.17 ± (219.20 - 223.15) ms221.58 ± (220.02 - 223.13) ms+0.2%✅⬆️
process.time_to_main_ms506.14 ± (504.75 - 507.53) ms504.42 ± (503.25 - 505.59) ms-0.3%
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%
runtime.dotnet.mem.committed48.09 ± (48.07 - 48.11) MB48.12 ± (48.10 - 48.14) MB+0.1%✅⬆️
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)+0.6%✅⬆️
.NET 6 - Baseline
process.internal_duration_ms21.63 ± (21.57 - 21.69) ms21.54 ± (21.48 - 21.60) ms-0.4%
process.time_to_main_ms75.31 ± (74.98 - 75.63) ms75.61 ± (75.25 - 75.98) ms+0.4%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%
runtime.dotnet.mem.committed10.63 ± (10.63 - 10.64) MB10.63 ± (10.63 - 10.63) MB-0.0%
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%
.NET 6 - Bailout
process.internal_duration_ms21.57 ± (21.50 - 21.63) ms21.45 ± (21.40 - 21.51) ms-0.5%
process.time_to_main_ms76.87 ± (76.55 - 77.20) ms76.42 ± (76.11 - 76.72) ms-0.6%
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%
runtime.dotnet.mem.committed10.69 ± (10.69 - 10.70) MB10.67 ± (10.66 - 10.67) MB-0.2%
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms211.08 ± (209.75 - 212.41) ms209.97 ± (208.84 - 211.09) ms-0.5%
process.time_to_main_ms469.81 ± (468.79 - 470.83) ms467.44 ± (466.48 - 468.39) ms-0.5%
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%
runtime.dotnet.mem.committed48.43 ± (48.41 - 48.46) MB48.49 ± (48.47 - 48.51) MB+0.1%✅⬆️
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)-0.1%
.NET 8 - Baseline
process.internal_duration_ms19.86 ± (19.78 - 19.94) ms19.74 ± (19.68 - 19.80) ms-0.6%
process.time_to_main_ms75.26 ± (74.87 - 75.66) ms75.11 ± (74.78 - 75.44) ms-0.2%
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%
runtime.dotnet.mem.committed7.70 ± (7.69 - 7.70) MB7.67 ± (7.66 - 7.67) MB-0.4%
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%
.NET 8 - Bailout
process.internal_duration_ms19.64 ± (19.59 - 19.70) ms19.67 ± (19.61 - 19.73) ms+0.1%✅⬆️
process.time_to_main_ms75.42 ± (75.10 - 75.74) ms76.13 ± (75.83 - 76.42) ms+0.9%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%
runtime.dotnet.mem.committed7.74 ± (7.73 - 7.74) MB7.71 ± (7.70 - 7.72) MB-0.3%
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms193.09 ± (192.19 - 193.98) ms192.51 ± (191.51 - 193.51) ms-0.3%
process.time_to_main_ms452.19 ± (451.23 - 453.15) ms453.91 ± (452.88 - 454.94) ms+0.4%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%
runtime.dotnet.mem.committed36.54 ± (36.50 - 36.59) MB36.48 ± (36.45 - 36.52) MB-0.2%
runtime.dotnet.threads.count27 ± (27 - 27)27 ± (27 - 27)+0.1%✅⬆️

HttpMessageHandler

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration192.50 ± (192.51 - 193.28) ms193.01 ± (192.95 - 193.71) ms+0.3%✅⬆️
.NET Framework 4.8 - Bailout
duration196.45 ± (196.28 - 196.81) ms195.89 ± (195.87 - 196.36) ms-0.3%
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1122.36 ± (1126.64 - 1137.44) ms1106.69 ± (1110.04 - 1118.16) ms-1.4%
.NET Core 3.1 - Baseline
process.internal_duration_ms187.75 ± (187.39 - 188.11) ms187.29 ± (187.01 - 187.57) ms-0.2%
process.time_to_main_ms80.62 ± (80.40 - 80.83) ms80.58 ± (80.36 - 80.80) ms-0.0%
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%
runtime.dotnet.mem.committed16.10 ± (16.07 - 16.13) MB16.09 ± (16.06 - 16.12) MB-0.0%
runtime.dotnet.threads.count20 ± (19 - 20)20 ± (20 - 20)+0.6%✅⬆️
.NET Core 3.1 - Bailout
process.internal_duration_ms187.51 ± (187.21 - 187.80) ms187.38 ± (187.08 - 187.69) ms-0.1%
process.time_to_main_ms82.05 ± (81.93 - 82.17) ms82.09 ± (81.93 - 82.24) ms+0.0%✅⬆️
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%
runtime.dotnet.mem.committed16.15 ± (16.12 - 16.17) MB16.10 ± (16.07 - 16.13) MB-0.3%
runtime.dotnet.threads.count21 ± (20 - 21)21 ± (21 - 21)+0.3%✅⬆️
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms401.24 ± (398.16 - 404.32) ms396.34 ± (393.86 - 398.82) ms-1.2%
process.time_to_main_ms474.56 ± (473.90 - 475.22) ms474.04 ± (473.43 - 474.64) ms-0.1%
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%
runtime.dotnet.mem.committed58.28 ± (58.13 - 58.43) MB58.68 ± (58.53 - 58.83) MB+0.7%✅⬆️
runtime.dotnet.threads.count29 ± (29 - 29)29 ± (29 - 29)+0.1%✅⬆️
.NET 6 - Baseline
process.internal_duration_ms191.96 ± (191.68 - 192.23) ms193.59 ± (193.11 - 194.07) ms+0.9%✅⬆️
process.time_to_main_ms69.95 ± (69.78 - 70.12) ms70.33 ± (70.16 - 70.50) ms+0.5%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%
runtime.dotnet.mem.committed16.06 ± (15.97 - 16.15) MB16.27 ± (16.17 - 16.38) MB+1.3%✅⬆️
runtime.dotnet.threads.count18 ± (18 - 18)19 ± (18 - 19)+2.2%✅⬆️
.NET 6 - Bailout
process.internal_duration_ms190.86 ± (190.59 - 191.14) ms190.93 ± (190.67 - 191.19) ms+0.0%✅⬆️
process.time_to_main_ms70.82 ± (70.73 - 70.90) ms70.79 ± (70.67 - 70.91) ms-0.0%
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%
runtime.dotnet.mem.committed16.17 ± (16.04 - 16.31) MB16.18 ± (16.04 - 16.32) MB+0.1%✅⬆️
runtime.dotnet.threads.count20 ± (20 - 20)19 ± (19 - 20)-2.0%
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms408.41 ± (405.74 - 411.08) ms408.48 ± (406.20 - 410.76) ms+0.0%✅⬆️
process.time_to_main_ms443.94 ± (443.34 - 444.54) ms445.30 ± (444.67 - 445.93) ms+0.3%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%
runtime.dotnet.mem.committed58.72 ± (58.56 - 58.87) MB59.16 ± (59.03 - 59.29) MB+0.8%✅⬆️
runtime.dotnet.threads.count30 ± (30 - 30)30 ± (30 - 30)-0.1%
.NET 8 - Baseline
process.internal_duration_ms189.41 ± (189.06 - 189.76) ms189.55 ± (189.22 - 189.88) ms+0.1%✅⬆️
process.time_to_main_ms69.46 ± (69.26 - 69.65) ms69.18 ± (69.02 - 69.35) ms-0.4%
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%
runtime.dotnet.mem.committed11.72 ± (11.69 - 11.75) MB11.81 ± (11.78 - 11.84) MB+0.7%✅⬆️
runtime.dotnet.threads.count18 ± (18 - 18)18 ± (18 - 18)+0.7%✅⬆️
.NET 8 - Bailout
process.internal_duration_ms188.61 ± (188.35 - 188.87) ms188.70 ± (188.50 - 188.90) ms+0.0%✅⬆️
process.time_to_main_ms70.29 ± (70.19 - 70.38) ms70.15 ± (70.04 - 70.26) ms-0.2%
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%
runtime.dotnet.mem.committed11.85 ± (11.81 - 11.88) MB11.85 ± (11.81 - 11.90) MB+0.1%✅⬆️
runtime.dotnet.threads.count19 ± (19 - 19)19 ± (19 - 19)-0.2%
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms365.46 ± (364.01 - 366.91) ms364.08 ± (362.33 - 365.83) ms-0.4%
process.time_to_main_ms427.67 ± (427.15 - 428.19) ms428.27 ± (427.66 - 428.89) ms+0.1%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%
runtime.dotnet.mem.committed47.91 ± (47.88 - 47.95) MB47.88 ± (47.84 - 47.92) MB-0.1%
runtime.dotnet.threads.count29 ± (29 - 29)29 ± (29 - 29)-0.6%
Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts
FakeDbCommand (.NET Framework 4.8)
gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7926) - mean (76ms)  : 71, 81
    master - mean (75ms)  : 70, 81

    section Bailout
    This PR (7926) - mean (80ms)  : 75, 85
    master - mean (84ms)  : 75, 92

    section CallTarget+Inlining+NGEN
    This PR (7926) - mean (1,072ms)  : 1018, 1127
    master - mean (1,074ms)  : 1022, 1126

FakeDbCommand (.NET Core 3.1)
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7926) - mean (116ms)  : 110, 122
    master - mean (118ms)  : 112, 125

    section Bailout
    This PR (7926) - mean (119ms)  : 113, 126
    master - mean (120ms)  : 113, 127

    section CallTarget+Inlining+NGEN
    This PR (7926) - mean (765ms)  : 722, 807
    master - mean (771ms)  : 729, 812

FakeDbCommand (.NET 6)
gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7926) - mean (104ms)  : 98, 110
    master - mean (104ms)  : 96, 111

    section Bailout
    This PR (7926) - mean (105ms)  : 98, 111
    master - mean (106ms)  : 99, 112

    section CallTarget+Inlining+NGEN
    This PR (7926) - mean (704ms)  : 677, 732
    master - mean (717ms)  : 664, 769

FakeDbCommand (.NET 8)
gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7926) - mean (103ms)  : 97, 109
    master - mean (104ms)  : 96, 112

    section Bailout
    This PR (7926) - mean (104ms)  : 99, 109
    master - mean (104ms)  : 97, 110

    section CallTarget+Inlining+NGEN
    This PR (7926) - mean (680ms)  : 642, 718
    master - mean (686ms)  : 658, 714

HttpMessageHandler (.NET Framework 4.8)
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7926) - mean (193ms)  : 189, 197
    master - mean (193ms)  : 189, 197

    section Bailout
    This PR (7926) - mean (196ms)  : 194, 198
    master - mean (197ms)  : 194, 199

    section CallTarget+Inlining+NGEN
    This PR (7926) - mean (1,114ms)  : 1056, 1172
    master - mean (1,132ms)  : 1050, 1214

HttpMessageHandler (.NET Core 3.1)
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7926) - mean (276ms)  : 272, 281
    master - mean (277ms)  : 271, 282

    section Bailout
    This PR (7926) - mean (278ms)  : 274, 281
    master - mean (278ms)  : 274, 282

    section CallTarget+Inlining+NGEN
    This PR (7926) - mean (902ms)  : 854, 950
    master - mean (910ms)  : 866, 954

HttpMessageHandler (.NET 6)
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7926) - mean (273ms)  : 266, 279
    master - mean (270ms)  : 263, 277

    section Bailout
    This PR (7926) - mean (270ms)  : 265, 275
    master - mean (269ms)  : 266, 273

    section CallTarget+Inlining+NGEN
    This PR (7926) - mean (887ms)  : 844, 931
    master - mean (888ms)  : 842, 933

HttpMessageHandler (.NET 8)
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7926) - mean (268ms)  : 264, 273
    master - mean (269ms)  : 263, 275

    section Bailout
    This PR (7926) - mean (268ms)  : 266, 271
    master - mean (268ms)  : 264, 272

    section CallTarget+Inlining+NGEN
    This PR (7926) - mean (825ms)  : 805, 845
    master - mean (824ms)  : 805, 844

Just noticed this in the CI. We should review those.

##[error]Target "RunManagedUnitTests" has thrown an exception 12:38:08 [ERR] Target RunManagedUnitTests has thrown an exception System.IO.IOException: There is not enough space on the disk. : 'D:\a_work\1\s\artifacts\build_data\dumps\testhost.exe_251209_123600.dmp'. at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite) at Nuke.Common.IO.FileSystemTasks.CopyFile(String source, String target, FileExistsPolicy policy, Boolean

NachoEchevarria avatar Dec 10 '25 10:12 NachoEchevarria

Just noticed this in the CI. We should review those.

Yeah, to be honest, we may not need to merge this PR at all. We're looking at those dumps and tl;dr; there's thread exhaustion as best as I can tell. The real question is why 😬

andrewlock avatar Dec 10 '25 11:12 andrewlock