dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard
verified publisher icon DataDog

[Installer] Remove the CLR Profiler settings from W3SVC

Open zacharycmontoya opened this issue 2 years ago • 6 comments

Summary of changes

Removes the installer directive that adds the CLR Profiler settings to the W3SVC service

Reason for change

Previously, we were setting the CLR Profiler settings in both the WAS and the W3SVC. We had to set the W3SVC service to cover IIS versions 6 and below, since the WAS was introduced in IIS version 7, but this is no longer needed. Now that the minimum version of .NET Framework that we support is .NET Framework 4.6.1, and the corresponding minimum OS version is Windows Server 2008 R2 SP1, we will only encounter IIS 7+.

Resources:

  • https://techcommunity.microsoft.com/t5/iis-support-blog/iis-services-http-sys-w3svc-was-w3wp-oh-my/ba-p/287856
  • https://learn.microsoft.com/en-us/iis/manage/provisioning-and-managing-iis/features-of-the-windows-process-activation-service-was
  • https://learn.microsoft.com/en-us/lifecycle/products/internet-information-services-iis
  • https://learn.microsoft.com/en-us/dotnet/framework/migration-guide/versions-and-dependencies#net-framework-461

Implementation details

Removes the lines.

Test coverage

Existing integration tests and smoke tests should pass

Other details

N/A

zacharycmontoya avatar Apr 18 '23 18:04 zacharycmontoya

Datadog Report

Branch report: zach/iis-environment-variables Commit report: 2dcb6cb

:snowflake: dd-trace-dotnet: 0 Failed, 2 New Flaky, 242949 Passed, 915 Skipped, 35m 24.54s Wall Time

New Flaky Tests (2)

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.AgentMalfunctionTests - Last Failure

    Expand for error 
    xpected exit code: 0, actual exit code: -1073741819.

  • NoExceptions - Datadog.Trace.ClrProfiler.IntegrationTests.SmokeTests.HttpHandlerStackOverflowExceptionSmokeTest - Last Failure

    Expand for error 
    he smoke test is running for too long or was lost.

datadog-ddstaging[bot] avatar Apr 18 '23 19:04 datadog-ddstaging[bot]

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4056) - mean (3,014ms)  : 2914, 3114
     .   : milestone, 3014,
    master - mean (3,047ms)  : 2954, 3141
     .   : milestone, 3047,

    section CallTarget+Inlining+NGEN
    This PR (4056) - mean (3,681ms)  : 3601, 3762
     .   : milestone, 3681,
    master - mean (3,664ms)  : 3625, 3703
     .   : milestone, 3664,


gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4056) - mean (3,132ms)  : 3015, 3250
     .   : milestone, 3132,
    master - mean (3,102ms)  : 3010, 3195
     .   : milestone, 3102,

    section CallTarget+Inlining+NGEN
    This PR (4056) - mean (3,543ms)  : 3466, 3621
     .   : milestone, 3543,
    master - mean (3,570ms)  : 3443, 3696
     .   : milestone, 3570,


gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4056) - mean (3,080ms)  : 2944, 3215
     .   : milestone, 3080,
    master - mean (3,094ms)  : 2979, 3209
     .   : milestone, 3094,

    section CallTarget+Inlining+NGEN
    This PR (4056) - mean (3,517ms)  : 3423, 3611
     .   : milestone, 3517,
    master - mean (3,505ms)  : 3423, 3588
     .   : milestone, 3505,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4056) - mean (187ms)  : 183, 190
     .   : milestone, 187,
    master - mean (187ms)  : 183, 191
     .   : milestone, 187,

    section CallTarget+Inlining+NGEN
    This PR (4056) - mean (941ms)  : 908, 975
     .   : milestone, 941,
    master - mean (943ms)  : 904, 981
     .   : milestone, 943,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4056) - mean (367ms)  : 361, 372
     .   : milestone, 367,
    master - mean (365ms)  : 360, 371
     .   : milestone, 365,

    section CallTarget+Inlining+NGEN
    This PR (4056) - mean (1,084ms)  : 1052, 1115
     .   : milestone, 1084,
    master - mean (1,083ms)  : 1059, 1108
     .   : milestone, 1083,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4056) - mean (355ms)  : 349, 361
     .   : milestone, 355,
    master - mean (355ms)  : 350, 360
     .   : milestone, 355,

    section CallTarget+Inlining+NGEN
    This PR (4056) - mean (1,033ms)  : 1006, 1060
     .   : milestone, 1033,
    master - mean (1,031ms)  : 1001, 1060
     .   : milestone, 1031,

andrewlock avatar Apr 18 '23 19:04 andrewlock

Benchmarks Report :snail:

Benchmarks for #4056 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.153
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces netcoreapp3.1 600μs 143ns 534ns 0 0 0 2.63 KB
master WriteAndFlushEnrichedTraces net472 824μs 249ns 965ns 0.403 0 0 3.22 KB
#4056 WriteAndFlushEnrichedTraces netcoreapp3.1 609μs 127ns 475ns 0 0 0 2.63 KB
#4056 WriteAndFlushEnrichedTraces net472 785μs 173ns 622ns 0.391 0 0 3.22 KB
Benchmarks.Trace.AppSecBodyBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #4056

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AppSecBodyBenchmark.AllCycleSimpleBody‑net472 1.153 105,949.11 91,872.13

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody netcoreapp3.1 97.3μs 548ns 3.99μs 0 0 0 1.66 KB
master AllCycleSimpleBody net472 106μs 152ns 569ns 0.245 0 0 1.71 KB
master AllCycleMoreComplexBody netcoreapp3.1 273μs 1.32μs 5.27μs 0 0 0 9.14 KB
master AllCycleMoreComplexBody net472 267μs 357ns 1.24μs 1.47 0 0 9.31 KB
master ObjectExtractorSimpleBody netcoreapp3.1 170ns 0.161ns 0.622ns 0.00376 0 0 272 B
master ObjectExtractorSimpleBody net472 148ns 0.104ns 0.403ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.05μs 1.4ns 5.04ns 0.0506 0 0 3.78 KB
master ObjectExtractorMoreComplexBody net472 4.26μs 2.91ns 10.9ns 0.618 0.00648 0 3.89 KB
#4056 AllCycleSimpleBody netcoreapp3.1 100μs 547ns 3.19μs 0 0 0 1.66 KB
#4056 AllCycleSimpleBody net472 91.8μs 148ns 572ns 0.235 0 0 1.71 KB
#4056 AllCycleMoreComplexBody netcoreapp3.1 270μs 459ns 1.78μs 0 0 0 9.14 KB
#4056 AllCycleMoreComplexBody net472 265μs 316ns 1.22μs 1.45 0 0 9.31 KB
#4056 ObjectExtractorSimpleBody netcoreapp3.1 162ns 0.216ns 0.838ns 0.00371 0 0 272 B
#4056 ObjectExtractorSimpleBody net472 147ns 0.25ns 0.936ns 0.0446 0 0 281 B
#4056 ObjectExtractorMoreComplexBody netcoreapp3.1 4.1μs 3.91ns 15.1ns 0.0514 0 0 3.78 KB
#4056 ObjectExtractorMoreComplexBody net472 4.13μs 6.18ns 23.9ns 0.617 0.00617 0 3.89 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest netcoreapp3.1 183μs 384ns 1.49μs 0.182 0 0 20.36 KB
master SendRequest net472 0.000739ns 0.00034ns 0.00123ns 0 0 0 0 b
#4056 SendRequest netcoreapp3.1 183μs 256ns 993ns 0.183 0 0 20.36 KB
#4056 SendRequest net472 8.07E‑05ns 4.67E‑05ns 0.000169ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: More allocations :warning:

More allocations :warning: in #4056

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.59 KB 41.98 KB 395 B 0.95%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces netcoreapp3.1 598μs 1.42μs 5.11μs 0.305 0 0 41.59 KB
master WriteAndFlushEnrichedTraces net472 777μs 3.34μs 12.9μs 8.38 2.67 0.381 53.23 KB
#4056 WriteAndFlushEnrichedTraces netcoreapp3.1 608μs 2.12μs 8.23μs 0.311 0 0 41.98 KB
#4056 WriteAndFlushEnrichedTraces net472 798μs 2.58μs 10μs 8.2 2.34 0.391 53.23 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery netcoreapp3.1 1.29μs 0.929ns 3.6ns 0.0122 0 0 904 B
master ExecuteNonQuery net472 1.72μs 2.37ns 9.18ns 0.144 0.000857 0 907 B
#4056 ExecuteNonQuery netcoreapp3.1 1.34μs 0.83ns 2.88ns 0.012 0 0 904 B
#4056 ExecuteNonQuery net472 1.64μs 2.37ns 9.16ns 0.144 0 0 907 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch netcoreapp3.1 1.27μs 0.606ns 2.27ns 0.0146 0 0 1.06 KB
master CallElasticsearch net472 2.12μs 1.34ns 5.19ns 0.175 0.00106 0 1.11 KB
master CallElasticsearchAsync netcoreapp3.1 1.37μs 0.505ns 1.82ns 0.0163 0 0 1.18 KB
master CallElasticsearchAsync net472 2.31μs 1.1ns 4.27ns 0.197 0.00115 0 1.24 KB
#4056 CallElasticsearch netcoreapp3.1 1.33μs 0.707ns 2.74ns 0.014 0 0 1.06 KB
#4056 CallElasticsearch net472 2.29μs 0.632ns 2.45ns 0.176 0.00114 0 1.11 KB
#4056 CallElasticsearchAsync netcoreapp3.1 1.34μs 0.431ns 1.61ns 0.0161 0 0 1.18 KB
#4056 CallElasticsearchAsync net472 2.3μs 0.972ns 3.76ns 0.197 0 0 1.24 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync netcoreapp3.1 1.53μs 0.786ns 2.72ns 0.0173 0 0 1.28 KB
master ExecuteAsync net472 1.84μs 0.541ns 2.09ns 0.207 0.000918 0 1.3 KB
#4056 ExecuteAsync netcoreapp3.1 1.47μs 0.434ns 1.56ns 0.017 0 0 1.28 KB
#4056 ExecuteAsync net472 1.87μs 0.658ns 2.55ns 0.206 0 0 1.3 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync netcoreapp3.1 4.23μs 0.929ns 3.35ns 0.0362 0 0 2.66 KB
master SendAsync net472 6.73μs 1.75ns 6.77ns 0.479 0 0 3.03 KB
#4056 SendAsync netcoreapp3.1 4.46μs 1.34ns 5.01ns 0.0355 0 0 2.66 KB
#4056 SendAsync net472 6.9μs 2.02ns 7.81ns 0.479 0 0 3.03 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog netcoreapp3.1 1.87μs 0.932ns 3.49ns 0.0244 0 0 1.83 KB
master EnrichedLog net472 2.51μs 1.68ns 6.27ns 0.283 0 0 1.79 KB
#4056 EnrichedLog netcoreapp3.1 1.95μs 1.21ns 4.21ns 0.0252 0 0 1.83 KB
#4056 EnrichedLog net472 2.59μs 1.83ns 6.84ns 0.284 0 0 1.79 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog netcoreapp3.1 119μs 126ns 488ns 0.0593 0 0 4.42 KB
master EnrichedLog net472 147μs 278ns 1.08μs 0.734 0.22 0 4.63 KB
#4056 EnrichedLog netcoreapp3.1 119μs 104ns 391ns 0 0 0 4.42 KB
#4056 EnrichedLog net472 148μs 71.2ns 266ns 0.667 0.222 0 4.63 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog netcoreapp3.1 3.83μs 1.98ns 7.66ns 0.0536 0 0 3.9 KB
master EnrichedLog net472 5.17μs 0.891ns 3.45ns 0.565 0.00258 0 3.56 KB
#4056 EnrichedLog netcoreapp3.1 3.89μs 2.34ns 8.45ns 0.0543 0 0 3.9 KB
#4056 EnrichedLog net472 5.22μs 1.94ns 7.52ns 0.565 0.0026 0 3.56 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive netcoreapp3.1 1.56μs 0.537ns 1.94ns 0.0174 0 0 1.3 KB
master SendReceive net472 1.91μs 0.46ns 1.66ns 0.212 0 0 1.34 KB
#4056 SendReceive netcoreapp3.1 1.56μs 0.76ns 2.84ns 0.0171 0 0 1.3 KB
#4056 SendReceive net472 1.97μs 1.97ns 7.64ns 0.213 0 0 1.34 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog netcoreapp3.1 3.75μs 1.67ns 6.48ns 0.0244 0 0 1.78 KB
master EnrichedLog net472 4.43μs 1.8ns 6.98ns 0.35 0 0 2.21 KB
#4056 EnrichedLog netcoreapp3.1 3.66μs 1.63ns 6.11ns 0.0238 0 0 1.78 KB
#4056 EnrichedLog net472 4.37μs 1.26ns 4.72ns 0.351 0 0 2.21 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan netcoreapp3.1 720ns 0.319ns 1.19ns 0.00975 0 0 720 B
master StartFinishSpan net472 830ns 0.219ns 0.818ns 0.121 0 0 762 B
master StartFinishScope netcoreapp3.1 856ns 0.556ns 2.08ns 0.0116 0 0 840 B
master StartFinishScope net472 1.15μs 0.363ns 1.36ns 0.134 0 0 842 B
#4056 StartFinishSpan netcoreapp3.1 699ns 0.599ns 2.32ns 0.00965 0 0 720 B
#4056 StartFinishSpan net472 912ns 0.354ns 1.32ns 0.121 0 0 762 B
#4056 StartFinishScope netcoreapp3.1 821ns 0.265ns 1.03ns 0.0111 0 0 840 B
#4056 StartFinishScope net472 1.13μs 0.949ns 3.42ns 0.134 0 0 842 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin netcoreapp3.1 939ns 0.175ns 0.631ns 0.0113 0 0 840 B
master RunOnMethodBegin net472 1.16μs 0.315ns 1.18ns 0.133 0 0 842 B
#4056 RunOnMethodBegin netcoreapp3.1 963ns 0.286ns 0.99ns 0.0112 0 0 840 B
#4056 RunOnMethodBegin net472 1.18μs 0.595ns 2.31ns 0.134 0 0 842 B

andrewlock avatar Apr 18 '23 19:04 andrewlock

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4056) - mean (3,019ms)  : 2886, 3151
     .   : milestone, 3019,
    master - mean (3,015ms)  : 2891, 3139
     .   : milestone, 3015,

    section CallTarget+Inlining+NGEN
    This PR (4056) - mean (3,665ms)  : 3588, 3742
     .   : milestone, 3665,
    master - mean (3,683ms)  : 3609, 3757
     .   : milestone, 3683,


gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4056) - mean (3,145ms)  : 3012, 3278
     .   : milestone, 3145,
    master - mean (3,122ms)  : 3008, 3235
     .   : milestone, 3122,

    section CallTarget+Inlining+NGEN
    This PR (4056) - mean (3,551ms)  : 3441, 3660
     .   : milestone, 3551,
    master - mean (3,530ms)  : 3432, 3628
     .   : milestone, 3530,


gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4056) - mean (3,113ms)  : 3014, 3212
     .   : milestone, 3113,
    master - mean (3,100ms)  : 2964, 3236
     .   : milestone, 3100,

    section CallTarget+Inlining+NGEN
    This PR (4056) - mean (3,521ms)  : 3440, 3602
     .   : milestone, 3521,
    master - mean (3,511ms)  : 3435, 3587
     .   : milestone, 3511,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4056) - mean (188ms)  : 183, 193
     .   : milestone, 188,
    master - mean (188ms)  : 184, 191
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (4056) - mean (941ms)  : 906, 975
     .   : milestone, 941,
    master - mean (940ms)  : 918, 963
     .   : milestone, 940,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4056) - mean (368ms)  : 361, 374
     .   : milestone, 368,
    master - mean (368ms)  : 363, 373
     .   : milestone, 368,

    section CallTarget+Inlining+NGEN
    This PR (4056) - mean (1,090ms)  : 1060, 1119
     .   : milestone, 1090,
    master - mean (1,089ms)  : 1063, 1115
     .   : milestone, 1089,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4056) - mean (356ms)  : 349, 364
     .   : milestone, 356,
    master - mean (357ms)  : 353, 362
     .   : milestone, 357,

    section CallTarget+Inlining+NGEN
    This PR (4056) - mean (1,036ms)  : 1010, 1062
     .   : milestone, 1036,
    master - mean (1,042ms)  : 1017, 1068
     .   : milestone, 1042,

andrewlock avatar Apr 20 '23 00:04 andrewlock

Benchmarks Report :snail:

Benchmarks for #4056 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces netcoreapp3.1 601μs 121ns 436ns 0 0 0 2.63 KB
master WriteAndFlushEnrichedTraces net472 794μs 328ns 1.23μs 0.396 0 0 3.22 KB
#4056 WriteAndFlushEnrichedTraces netcoreapp3.1 589μs 400ns 1.5μs 0 0 0 2.63 KB
#4056 WriteAndFlushEnrichedTraces net472 807μs 278ns 1.04μs 0.406 0 0 3.22 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody netcoreapp3.1 38.9μs 28.4ns 106ns 0.0195 0 0 1.66 KB
master AllCycleSimpleBody net472 39μs 57.3ns 222ns 0.27 0 0 1.71 KB
master AllCycleMoreComplexBody netcoreapp3.1 207μs 205ns 767ns 0.103 0 0 9.15 KB
master AllCycleMoreComplexBody net472 214μs 96.7ns 374ns 1.38 0 0 9.31 KB
master ObjectExtractorSimpleBody netcoreapp3.1 168ns 0.0396ns 0.143ns 0.00373 0 0 272 B
master ObjectExtractorSimpleBody net472 147ns 0.279ns 1.08ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.14μs 1.09ns 3.92ns 0.0517 0 0 3.78 KB
master ObjectExtractorMoreComplexBody net472 4.07μs 4.17ns 16.1ns 0.618 0.0061 0 3.89 KB
#4056 AllCycleSimpleBody netcoreapp3.1 37.5μs 59.7ns 223ns 0.0187 0 0 1.66 KB
#4056 AllCycleSimpleBody net472 38.4μs 26.8ns 104ns 0.268 0 0 1.71 KB
#4056 AllCycleMoreComplexBody netcoreapp3.1 205μs 375ns 1.4μs 0.103 0 0 9.14 KB
#4056 AllCycleMoreComplexBody net472 217μs 307ns 1.19μs 1.4 0 0 9.31 KB
#4056 ObjectExtractorSimpleBody netcoreapp3.1 183ns 0.128ns 0.496ns 0.00376 0 0 272 B
#4056 ObjectExtractorSimpleBody net472 146ns 0.187ns 0.725ns 0.0446 0 0 281 B
#4056 ObjectExtractorMoreComplexBody netcoreapp3.1 4.13μs 3.61ns 13.5ns 0.0493 0 0 3.78 KB
#4056 ObjectExtractorMoreComplexBody net472 4.08μs 3.53ns 13.7ns 0.617 0.00613 0 3.89 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest netcoreapp3.1 181μs 187ns 723ns 0.181 0 0 20.36 KB
master SendRequest net472 0.00124ns 0.000312ns 0.00121ns 0 0 0 0 b
#4056 SendRequest netcoreapp3.1 183μs 271ns 1.05μs 0.182 0 0 20.36 KB
#4056 SendRequest net472 0.000414ns 0.000234ns 0.000875ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces netcoreapp3.1 607μs 1.34μs 5.2μs 0.306 0 0 41.88 KB
master WriteAndFlushEnrichedTraces net472 772μs 1.98μs 7.66μs 8.25 2.36 0.393 53.24 KB
#4056 WriteAndFlushEnrichedTraces netcoreapp3.1 598μs 1.78μs 6.91μs 0.309 0 0 41.67 KB
#4056 WriteAndFlushEnrichedTraces net472 774μs 3.17μs 12.3μs 8.44 2.68 0.383 53.22 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery netcoreapp3.1 1.29μs 4.14ns 15.5ns 0.0126 0 0 904 B
master ExecuteNonQuery net472 1.65μs 2.3ns 8.91ns 0.144 0 0 907 B
#4056 ExecuteNonQuery netcoreapp3.1 1.3μs 1.19ns 4.59ns 0.0123 0 0 904 B
#4056 ExecuteNonQuery net472 1.7μs 2.5ns 9.7ns 0.144 0.000852 0 907 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch netcoreapp3.1 1.3μs 0.603ns 2.25ns 0.0142 0 0 1.06 KB
master CallElasticsearch net472 2.14μs 1.52ns 5.91ns 0.176 0.00108 0 1.11 KB
master CallElasticsearchAsync netcoreapp3.1 1.4μs 0.698ns 2.61ns 0.0162 0 0 1.18 KB
master CallElasticsearchAsync net472 2.36μs 4.62ns 17.9ns 0.198 0.00117 0 1.24 KB
#4056 CallElasticsearch netcoreapp3.1 1.3μs 0.527ns 1.97ns 0.0144 0 0 1.06 KB
#4056 CallElasticsearch net472 2.15μs 0.925ns 3.58ns 0.175 0.00107 0 1.11 KB
#4056 CallElasticsearchAsync netcoreapp3.1 1.36μs 0.589ns 2.12ns 0.0157 0 0 1.18 KB
#4056 CallElasticsearchAsync net472 2.47μs 1.46ns 5.65ns 0.197 0.00124 0 1.24 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync netcoreapp3.1 1.39μs 0.298ns 1.07ns 0.0173 0 0 1.28 KB
master ExecuteAsync net472 1.8μs 0.629ns 2.44ns 0.206 0 0 1.3 KB
#4056 ExecuteAsync netcoreapp3.1 1.48μs 0.809ns 3.03ns 0.0176 0 0 1.28 KB
#4056 ExecuteAsync net472 1.75μs 0.527ns 2.04ns 0.206 0.000873 0 1.3 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync netcoreapp3.1 4.21μs 2.92ns 11.3ns 0.0356 0 0 2.66 KB
master SendAsync net472 6.76μs 2.01ns 7.52ns 0.478 0 0 3.03 KB
#4056 SendAsync netcoreapp3.1 4.34μs 1.2ns 4.66ns 0.0346 0 0 2.66 KB
#4056 SendAsync net472 6.74μs 1.24ns 4.79ns 0.478 0 0 3.03 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog netcoreapp3.1 2.1μs 5.14ns 19.2ns 0.0243 0 0 1.83 KB
master EnrichedLog net472 2.53μs 2.3ns 8.59ns 0.284 0 0 1.79 KB
#4056 EnrichedLog netcoreapp3.1 1.94μs 0.502ns 1.88ns 0.0254 0 0 1.83 KB
#4056 EnrichedLog net472 2.45μs 1.54ns 5.78ns 0.284 0 0 1.79 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog netcoreapp3.1 119μs 136ns 509ns 0.0594 0 0 4.42 KB
master EnrichedLog net472 148μs 91.7ns 355ns 0.667 0.222 0 4.63 KB
#4056 EnrichedLog netcoreapp3.1 119μs 133ns 517ns 0.059 0 0 4.42 KB
#4056 EnrichedLog net472 149μs 63.7ns 238ns 0.671 0.224 0 4.63 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog netcoreapp3.1 3.84μs 5.18ns 19.4ns 0.0541 0 0 3.9 KB
master EnrichedLog net472 5.05μs 1.29ns 4.84ns 0.566 0.00252 0 3.56 KB
#4056 EnrichedLog netcoreapp3.1 3.9μs 1.38ns 5.18ns 0.0528 0 0 3.9 KB
#4056 EnrichedLog net472 5μs 0.966ns 3.62ns 0.564 0.00249 0 3.56 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive netcoreapp3.1 1.57μs 0.647ns 2.51ns 0.018 0 0 1.3 KB
master SendReceive net472 2.01μs 1.56ns 6.03ns 0.212 0.001 0 1.34 KB
#4056 SendReceive netcoreapp3.1 1.59μs 0.818ns 3.06ns 0.0175 0 0 1.3 KB
#4056 SendReceive net472 1.97μs 1.41ns 5.45ns 0.212 0.000983 0 1.34 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog netcoreapp3.1 3.63μs 1.14ns 4.41ns 0.0239 0 0 1.78 KB
master EnrichedLog net472 4.38μs 1.38ns 4.97ns 0.349 0 0 2.21 KB
#4056 EnrichedLog netcoreapp3.1 3.65μs 1.26ns 4.89ns 0.0236 0 0 1.78 KB
#4056 EnrichedLog net472 4.43μs 1.68ns 6.53ns 0.349 0 0 2.21 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan netcoreapp3.1 683ns 0.45ns 1.68ns 0.00991 0 0 720 B
master StartFinishSpan net472 940ns 0.476ns 1.84ns 0.121 0 0 762 B
master StartFinishScope netcoreapp3.1 862ns 0.86ns 3.33ns 0.0113 0 0 840 B
master StartFinishScope net472 1.15μs 0.439ns 1.7ns 0.134 0 0 842 B
#4056 StartFinishSpan netcoreapp3.1 712ns 0.271ns 0.978ns 0.00964 0 0 720 B
#4056 StartFinishSpan net472 921ns 0.331ns 1.28ns 0.121 0 0 762 B
#4056 StartFinishScope netcoreapp3.1 838ns 0.537ns 2.08ns 0.0113 0 0 840 B
#4056 StartFinishScope net472 1.13μs 1.05ns 4.07ns 0.134 0 0 842 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin netcoreapp3.1 938ns 0.482ns 1.87ns 0.0113 0 0 840 B
master RunOnMethodBegin net472 1.19μs 0.91ns 3.53ns 0.134 0 0 842 B
#4056 RunOnMethodBegin netcoreapp3.1 985ns 0.862ns 3.11ns 0.0114 0 0 840 B
#4056 RunOnMethodBegin net472 1.2μs 0.787ns 3.05ns 0.133 0 0 842 B

andrewlock avatar Apr 20 '23 00:04 andrewlock

Revisiting this PR, I'd still like to move forward with this change, but I'd also like to verify with telemetry that we're not running on these systems. I've added the status:do-not-merge label in the meantime

zacharycmontoya avatar Oct 17 '24 16:10 zacharycmontoya

Closing due to staleness. We can revisit this in the future if desired

zacharycmontoya avatar Apr 22 '25 17:04 zacharycmontoya