dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard
verified publisher icon DataDog

[Crashtracking] Collect PDB info

Open kevingosse opened this issue 1 year ago • 4 comments

Summary of changes

On Windows, when generating a crash report, collect the PDB info (signature, age).

Reason for change

Needed to resolve the symbols on server side.

Implementation details

In theory, we could have used SymGetModuleInfoW64 from dbghelp to retrieve that information, but the returned IMAGEHLP_MODULEW64 structure has evolved over time so we can't rely on the version of the DLL bundled with Windows. We could ship the dbghelp from the Windows SDK but that's a lot of extra space for an information that we can retrieve by other means.

So instead, this PR parses the PE header of the loaded modules to retrieve the PDB info. One source of complexity is that we're processing a remote process, so we have to use ReadProcessMemory first to fetch the memory we need. I introduced an abstraction to be able to test it locally in unit tests.

Test coverage

  • Added unit tests to validate that we correctly parse a PE32 and a PE64 image (I had to embed DLLs into the test project for that)
  • Added assertions to the integration tests to make sure we properly extract the PDB information

kevingosse avatar Oct 21 '24 17:10 kevingosse

Datadog Report

Branch report: kevin/pdb Commit report: 24a2f6e Test service: dd-trace-dotnet

:white_check_mark: 0 Failed, 365469 Passed, 2422 Skipped, 16h 48m 0.35s Total Time

datadog-ddstaging[bot] avatar Oct 21 '24 18:10 datadog-ddstaging[bot]

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6176) - mean (71ms)  : 68, 74
     .   : milestone, 71,
    master - mean (71ms)  : 68, 75
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (6176) - mean (1,115ms)  : 1094, 1136
     .   : milestone, 1115,
    master - mean (1,120ms)  : 1092, 1149
     .   : milestone, 1120,


gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6176) - mean (109ms)  : 106, 112
     .   : milestone, 109,
    master - mean (109ms)  : 107, 112
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (6176) - mean (776ms)  : 759, 793
     .   : milestone, 776,
    master - mean (776ms)  : 757, 795
     .   : milestone, 776,


gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6176) - mean (93ms)  : 89, 97
     .   : milestone, 93,
    master - mean (93ms)  : 89, 97
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (6176) - mean (725ms)  : 709, 742
     .   : milestone, 725,
    master - mean (728ms)  : 710, 746
     .   : milestone, 728,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6176) - mean (191ms)  : 187, 194
     .   : milestone, 191,
    master - mean (190ms)  : 187, 193
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6176) - mean (1,205ms)  : 1176, 1235
     .   : milestone, 1205,
    master - mean (1,193ms)  : 1168, 1218
     .   : milestone, 1193,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6176) - mean (276ms)  : 272, 279
     .   : milestone, 276,
    master - mean (275ms)  : 270, 279
     .   : milestone, 275,

    section CallTarget+Inlining+NGEN
    This PR (6176) - mean (942ms)  : 922, 961
     .   : milestone, 942,
    master - mean (943ms)  : 924, 961
     .   : milestone, 943,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6176) - mean (264ms)  : 259, 269
     .   : milestone, 264,
    master - mean (264ms)  : 260, 268
     .   : milestone, 264,

    section CallTarget+Inlining+NGEN
    This PR (6176) - mean (924ms)  : 904, 943
     .   : milestone, 924,
    master - mean (923ms)  : 907, 938
     .   : milestone, 923,

andrewlock avatar Oct 21 '24 18:10 andrewlock

Benchmarks Report for tracer :snail:

Benchmarks for #6176 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.123
  • 3 benchmarks are slower, with geometric mean 1.140
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 7.73μs 43.6ns 296ns 0.016 0.00799 0 5.43 KB
master StartStopWithChild netcoreapp3.1 9.86μs 48.1ns 204ns 0.0192 0.00959 0 5.62 KB
master StartStopWithChild net472 16.2μs 74.6ns 299ns 1.02 0.309 0.0975 6.06 KB
#6176 StartStopWithChild net6.0 7.82μs 43.1ns 280ns 0.0116 0.00772 0 5.43 KB
#6176 StartStopWithChild netcoreapp3.1 9.97μs 46.8ns 175ns 0.0247 0.00987 0 5.62 KB
#6176 StartStopWithChild net472 16.3μs 41.9ns 157ns 1.03 0.333 0.0914 6.07 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 517μs 218ns 817ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 643μs 343ns 1.33μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 840μs 533ns 2.06μs 0.419 0 0 3.3 KB
#6176 WriteAndFlushEnrichedTraces net6.0 474μs 309ns 1.2μs 0 0 0 2.7 KB
#6176 WriteAndFlushEnrichedTraces netcoreapp3.1 656μs 298ns 1.07μs 0 0 0 2.7 KB
#6176 WriteAndFlushEnrichedTraces net472 844μs 625ns 2.34μs 0.419 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 201μs 1.14μs 9.58μs 0.231 0 0 18.45 KB
master SendRequest netcoreapp3.1 223μs 1.3μs 11.7μs 0.216 0 0 20.61 KB
master SendRequest net472 0.000962ns 0.000263ns 0.00095ns 0 0 0 0 b
#6176 SendRequest net6.0 197μs 1.12μs 8.29μs 0.2 0 0 18.45 KB
#6176 SendRequest netcoreapp3.1 233μs 1.55μs 15.4μs 0.238 0 0 20.61 KB
#6176 SendRequest net472 0.00316ns 0.000837ns 0.00324ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 550μs 1.47μs 5.11μs 0.566 0 0 41.65 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 676μs 2.66μs 9.94μs 0.343 0 0 41.93 KB
master WriteAndFlushEnrichedTraces net472 862μs 3.49μs 13.5μs 8.45 2.53 0.422 53.31 KB
#6176 WriteAndFlushEnrichedTraces net6.0 610μs 3.14μs 14.7μs 0.573 0 0 41.81 KB
#6176 WriteAndFlushEnrichedTraces netcoreapp3.1 704μs 3.94μs 26.4μs 0.353 0 0 41.78 KB
#6176 WriteAndFlushEnrichedTraces net472 867μs 4μs 15.5μs 8.45 2.53 0.422 53.29 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.38μs 1.32ns 5.11ns 0.0144 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.74μs 0.944ns 3.66ns 0.0138 0 0 1.02 KB
master ExecuteNonQuery net472 2.1μs 2.18ns 8.42ns 0.157 0 0 987 B
#6176 ExecuteNonQuery net6.0 1.25μs 1.37ns 5.31ns 0.0149 0 0 1.02 KB
#6176 ExecuteNonQuery netcoreapp3.1 1.74μs 1.13ns 4.24ns 0.0141 0 0 1.02 KB
#6176 ExecuteNonQuery net472 2.17μs 2.34ns 9.06ns 0.157 0 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #6176

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net6.0 1.123 1,451.73 1,292.94

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.16μs 0.43ns 1.61ns 0.0134 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.53μs 0.598ns 2.07ns 0.0137 0 0 976 B
master CallElasticsearch net472 2.56μs 2.04ns 7.89ns 0.158 0 0 995 B
master CallElasticsearchAsync net6.0 1.45μs 0.481ns 1.8ns 0.0131 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.71μs 0.783ns 2.93ns 0.0138 0 0 1.02 KB
master CallElasticsearchAsync net472 2.64μs 0.885ns 3.31ns 0.166 0 0 1.05 KB
#6176 CallElasticsearch net6.0 1.1μs 0.423ns 1.58ns 0.0133 0 0 976 B
#6176 CallElasticsearch netcoreapp3.1 1.51μs 0.623ns 2.33ns 0.0136 0 0 976 B
#6176 CallElasticsearch net472 2.51μs 4.3ns 16.6ns 0.158 0.00125 0 995 B
#6176 CallElasticsearchAsync net6.0 1.29μs 0.561ns 2.17ns 0.0136 0 0 952 B
#6176 CallElasticsearchAsync netcoreapp3.1 1.56μs 0.679ns 2.54ns 0.0141 0 0 1.02 KB
#6176 CallElasticsearchAsync net472 2.56μs 11.4ns 44.2ns 0.166 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.26μs 0.698ns 2.7ns 0.0133 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.63μs 0.913ns 3.54ns 0.0123 0 0 952 B
master ExecuteAsync net472 1.78μs 0.679ns 2.63ns 0.145 0 0 915 B
#6176 ExecuteAsync net6.0 1.23μs 0.39ns 1.46ns 0.0129 0 0 952 B
#6176 ExecuteAsync netcoreapp3.1 1.64μs 0.768ns 2.97ns 0.0124 0 0 952 B
#6176 ExecuteAsync net472 1.74μs 0.562ns 2.1ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.25μs 1.25ns 4.83ns 0.0297 0 0 2.22 KB
master SendAsync netcoreapp3.1 5.15μs 3.64ns 14.1ns 0.0361 0 0 2.76 KB
master SendAsync net472 7.82μs 1.25ns 4.68ns 0.497 0 0 3.15 KB
#6176 SendAsync net6.0 4.15μs 2.48ns 8.95ns 0.0313 0 0 2.22 KB
#6176 SendAsync netcoreapp3.1 5.13μs 4.15ns 15ns 0.0359 0 0 2.76 KB
#6176 SendAsync net472 7.66μs 3.08ns 11.5ns 0.498 0 0 3.15 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.65μs 1.65ns 6.4ns 0.0227 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.11μs 0.716ns 2.68ns 0.0221 0 0 1.64 KB
master EnrichedLog net472 2.68μs 1.33ns 5.17ns 0.249 0 0 1.57 KB
#6176 EnrichedLog net6.0 1.53μs 1.66ns 6.43ns 0.0228 0 0 1.64 KB
#6176 EnrichedLog netcoreapp3.1 2.28μs 1.07ns 4.02ns 0.0216 0 0 1.64 KB
#6176 EnrichedLog net472 2.69μs 1.71ns 6.63ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 116μs 153ns 594ns 0.0579 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 122μs 191ns 741ns 0 0 0 4.28 KB
master EnrichedLog net472 149μs 120ns 466ns 0.674 0.225 0 4.46 KB
#6176 EnrichedLog net6.0 116μs 91.6ns 355ns 0.0574 0 0 4.28 KB
#6176 EnrichedLog netcoreapp3.1 121μs 328ns 1.27μs 0 0 0 4.28 KB
#6176 EnrichedLog net472 152μs 177ns 685ns 0.684 0.228 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.08μs 0.781ns 3.03ns 0.031 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.16μs 1.1ns 4.25ns 0.0291 0 0 2.2 KB
master EnrichedLog net472 4.75μs 2.05ns 7.95ns 0.321 0 0 2.02 KB
#6176 EnrichedLog net6.0 3.03μs 0.758ns 2.84ns 0.0304 0 0 2.2 KB
#6176 EnrichedLog netcoreapp3.1 4.05μs 1.7ns 6.35ns 0.0304 0 0 2.2 KB
#6176 EnrichedLog net472 4.72μs 1.28ns 4.98ns 0.32 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.34μs 1.24ns 4.82ns 0.0161 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.7μs 0.574ns 2.15ns 0.0146 0 0 1.14 KB
master SendReceive net472 2.29μs 0.971ns 3.63ns 0.183 0.00115 0 1.16 KB
#6176 SendReceive net6.0 1.35μs 0.664ns 2.57ns 0.0162 0 0 1.14 KB
#6176 SendReceive netcoreapp3.1 1.81μs 1.55ns 6.01ns 0.0154 0 0 1.14 KB
#6176 SendReceive net472 2.18μs 3.07ns 11.9ns 0.183 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.77μs 1ns 3.88ns 0.0223 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 3.99μs 1.67ns 6.47ns 0.0219 0 0 1.65 KB
master EnrichedLog net472 4.47μs 1.07ns 4.14ns 0.322 0 0 2.04 KB
#6176 EnrichedLog net6.0 2.78μs 0.966ns 3.74ns 0.0223 0 0 1.6 KB
#6176 EnrichedLog netcoreapp3.1 4.05μs 3.8ns 14.7ns 0.0221 0 0 1.65 KB
#6176 EnrichedLog net472 4.4μs 3.09ns 12ns 0.322 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #6176

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 1.123 700.91 787.42

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 420ns 0.237ns 0.917ns 0.00806 0 0 576 B
master StartFinishSpan netcoreapp3.1 612ns 0.556ns 2.08ns 0.00786 0 0 576 B
master StartFinishSpan net472 706ns 0.537ns 2.08ns 0.0918 0 0 578 B
master StartFinishScope net6.0 476ns 0.391ns 1.52ns 0.00976 0 0 696 B
master StartFinishScope netcoreapp3.1 700ns 0.817ns 3.16ns 0.00963 0 0 696 B
master StartFinishScope net472 846ns 0.938ns 3.63ns 0.104 0 0 658 B
#6176 StartFinishSpan net6.0 406ns 0.83ns 3.22ns 0.00798 0 0 576 B
#6176 StartFinishSpan netcoreapp3.1 639ns 0.359ns 1.34ns 0.00772 0 0 576 B
#6176 StartFinishSpan net472 705ns 0.414ns 1.6ns 0.0915 0 0 578 B
#6176 StartFinishScope net6.0 516ns 0.184ns 0.712ns 0.00974 0 0 696 B
#6176 StartFinishScope netcoreapp3.1 787ns 0.343ns 1.33ns 0.00942 0 0 696 B
#6176 StartFinishScope net472 893ns 2.42ns 9.38ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #6176

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.154 586.87 677.24
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1 1.143 880.22 1,006.09

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 587ns 0.62ns 2.4ns 0.00967 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 880ns 0.765ns 2.86ns 0.0092 0 0 696 B
master RunOnMethodBegin net472 1.17μs 1.09ns 4.21ns 0.105 0 0 658 B
#6176 RunOnMethodBegin net6.0 678ns 0.808ns 3.13ns 0.00993 0 0 696 B
#6176 RunOnMethodBegin netcoreapp3.1 1.01μs 0.483ns 1.87ns 0.00938 0 0 696 B
#6176 RunOnMethodBegin net472 1.1μs 0.279ns 1.04ns 0.104 0 0 658 B

andrewlock avatar Oct 22 '24 01:10 andrewlock

Throughput/Crank Report :zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6176) (11.051M)   : 0, 11050520
    master (11.141M)   : 0, 11140719
    benchmarks/2.9.0 (11.081M)   : 0, 11080577

    section Automatic
    This PR (6176) (7.404M)   : 0, 7403841
    master (7.280M)   : 0, 7279870
    benchmarks/2.9.0 (7.732M)   : 0, 7732233

    section Trace stats
    master (7.669M)   : 0, 7668885

    section Manual
    master (11.075M)   : 0, 11074558

    section Manual + Automatic
    This PR (6176) (6.797M)   : 0, 6797281
    master (6.727M)   : 0, 6727438

    section DD_TRACE_ENABLED=0
    master (10.143M)   : 0, 10143410


gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6176) (9.557M)   : 0, 9556581
    master (9.575M)   : 0, 9575114
    benchmarks/2.9.0 (9.798M)   : 0, 9798067

    section Automatic
    This PR (6176) (6.389M)   : 0, 6389244
    master (6.573M)   : 0, 6573227

    section Trace stats
    master (6.889M)   : 0, 6888898

    section Manual
    master (9.554M)   : 0, 9554430

    section Manual + Automatic
    This PR (6176) (6.000M)   : 0, 6000377
    master (6.223M)   : 0, 6222792

    section DD_TRACE_ENABLED=0
    master (8.834M)   : 0, 8833554


gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6176) (9.742M)   : 0, 9742172
    master (10.041M)   : 0, 10040592
    benchmarks/2.9.0 (10.067M)   : 0, 10067315

    section Automatic
    This PR (6176) (6.566M)   : 0, 6566168
    master (6.448M)   : 0, 6447976
    benchmarks/2.9.0 (7.552M)   : 0, 7552193

    section Trace stats
    master (7.067M)   : 0, 7067450

    section Manual
    master (9.677M)   : 0, 9676828

    section Manual + Automatic
    This PR (6176) (5.841M)   : 0, 5840518
    master (6.140M)   : 0, 6140399

    section DD_TRACE_ENABLED=0
    master (9.137M)   : 0, 9137058

andrewlock avatar Oct 22 '24 10:10 andrewlock