dd-trace-dotnet
dd-trace-dotnet copied to clipboard
Integration test for Oracle
Summary of changes
There was an Oracle Sample app, but it was not used in integration tests. I added a minimal test that at least allows manual testing, and added the required docker things.
Implementation details
I also merged the two Oracle sample projects (core and non-core) into one, using csproj conditions.
The Oracle image does not work (at all) on mac in docker desktop. I used colima to test it, which worked great (but I don't think it's something we want to use in the CI, so I marked the test as non-arm).
The image for the oracle DB is quite big, so we'll have to keep an eye on the CI perf with this on.
Datadog Report
Branch report: vandonr/dsm
Commit report: 8895ad0
Test service: dd-trace-dotnet
:white_check_mark: 0 Failed, 347846 Passed, 2224 Skipped, 22h 47m 45.56s Total Time
Throughput/Crank Report:zap:
Throughput results for AspNetCoreSimpleController comparing the following branches/commits:
Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.
Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!
gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5607) (11.706M) : 0, 11705658
master (12.102M) : 0, 12102247
benchmarks/2.9.0 (11.967M) : 0, 11966853
section Automatic
This PR (5607) (7.840M) : 0, 7840146
master (8.089M) : 0, 8088693
benchmarks/2.9.0 (8.405M) : 0, 8404892
section Trace stats
master (8.443M) : 0, 8442998
section Manual
This PR (5607) (10.257M) : 0, 10257072
master (10.275M) : 0, 10274868
section Manual + Automatic
This PR (5607) (7.504M) : 0, 7503651
master (7.533M) : 0, 7533118
section Version Conflict
master (6.842M) : 0, 6842477
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5607) (9.524M) : 0, 9523746
master (9.603M) : 0, 9602520
benchmarks/2.9.0 (9.255M) : 0, 9255366
section Automatic
This PR (5607) (6.546M) : 0, 6546074
master (6.457M) : 0, 6457136
section Trace stats
master (6.985M) : 0, 6985383
section Manual
This PR (5607) (8.295M) : 0, 8295267
master (8.151M) : 0, 8151308
section Manual + Automatic
This PR (5607) (5.851M) : crit ,0, 5850849
master (6.255M) : 0, 6255035
section Version Conflict
master (5.671M) : 0, 5670523
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5607) (10.217M) : 0, 10217270
master (10.319M) : 0, 10319080
benchmarks/2.9.0 (10.302M) : 0, 10302396
section Automatic
This PR (5607) (7.194M) : 0, 7193551
master (7.083M) : 0, 7083219
benchmarks/2.9.0 (7.635M) : 0, 7634994
section Trace stats
master (7.566M) : 0, 7566202
section Manual
This PR (5607) (8.837M) : 0, 8837202
master (8.698M) : 0, 8698395
section Manual + Automatic
This PR (5607) (6.872M) : 0, 6871931
master (6.776M) : 0, 6776250
section Version Conflict
master (6.140M) : 0, 6139972
Benchmarks Report for tracer :snail:
Benchmarks for #5607 compared to master:
- 2 benchmarks are faster, with geometric mean 1.122
- 2 benchmarks are slower, with geometric mean 1.164
- All benchmarks have the same allocations
The following thresholds were used for comparing the benchmark speeds:
- Mann–Whitney U test with statistical test for significance of 5%
- Only results indicating a difference greater than 10% and 0.3 ns are considered.
Allocation changes below 0.5% are ignored.
Benchmark details
Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartStopWithChild |
net6.0 | 7.67μs | 41.8ns | 243ns | 0.0147 | 0.00733 | 0 | 5.43 KB |
| master | StartStopWithChild |
netcoreapp3.1 | 9.94μs | 55ns | 344ns | 0.0147 | 0.00489 | 0 | 5.62 KB |
| master | StartStopWithChild |
net472 | 16.4μs | 61.3ns | 237ns | 1.05 | 0.337 | 0.0882 | 6.07 KB |
| #5607 | StartStopWithChild |
net6.0 | 7.86μs | 43.3ns | 253ns | 0.0151 | 0.00753 | 0 | 5.42 KB |
| #5607 | StartStopWithChild |
netcoreapp3.1 | 10μs | 55.9ns | 362ns | 0.0236 | 0.00945 | 0 | 5.62 KB |
| #5607 | StartStopWithChild |
net472 | 16.6μs | 71.6ns | 277ns | 1.03 | 0.317 | 0.0917 | 6.06 KB |
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | WriteAndFlushEnrichedTraces |
net6.0 | 477μs | 298ns | 1.08μs | 0 | 0 | 0 | 2.7 KB |
| master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 641μs | 225ns | 871ns | 0 | 0 | 0 | 2.7 KB |
| master | WriteAndFlushEnrichedTraces |
net472 | 820μs | 364ns | 1.36μs | 0.425 | 0 | 0 | 3.3 KB |
| #5607 | WriteAndFlushEnrichedTraces |
net6.0 | 491μs | 117ns | 422ns | 0 | 0 | 0 | 2.7 KB |
| #5607 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 634μs | 305ns | 1.18μs | 0 | 0 | 0 | 2.7 KB |
| #5607 | WriteAndFlushEnrichedTraces |
net472 | 844μs | 350ns | 1.31μs | 0.417 | 0 | 0 | 3.3 KB |
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendRequest |
net6.0 | 173μs | 268ns | 1.04μs | 0.173 | 0 | 0 | 18.45 KB |
| master | SendRequest |
netcoreapp3.1 | 187μs | 256ns | 991ns | 0.185 | 0 | 0 | 20.61 KB |
| master | SendRequest |
net472 | 7.65E‑05ns | 5.49E‑05ns | 0.000212ns | 0 | 0 | 0 | 0 b |
| #5607 | SendRequest |
net6.0 | 172μs | 261ns | 1.01μs | 0.256 | 0 | 0 | 18.45 KB |
| #5607 | SendRequest |
netcoreapp3.1 | 189μs | 385ns | 1.49μs | 0.188 | 0 | 0 | 20.61 KB |
| #5607 | SendRequest |
net472 | 1.43E‑05ns | 1.43E‑05ns | 5.36E‑05ns | 0 | 0 | 0 | 0 b |
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | WriteAndFlushEnrichedTraces |
net6.0 | 538μs | 684ns | 2.65μs | 0.536 | 0 | 0 | 41.52 KB |
| master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 658μs | 1.42μs | 5.5μs | 0.329 | 0 | 0 | 41.68 KB |
| master | WriteAndFlushEnrichedTraces |
net472 | 870μs | 3μs | 11.6μs | 8.45 | 2.53 | 0.422 | 53.23 KB |
| #5607 | WriteAndFlushEnrichedTraces |
net6.0 | 524μs | 347ns | 1.34μs | 0.527 | 0 | 0 | 41.64 KB |
| #5607 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 659μs | 1.82μs | 7.05μs | 0.324 | 0 | 0 | 41.76 KB |
| #5607 | WriteAndFlushEnrichedTraces |
net472 | 839μs | 3.95μs | 15.3μs | 8.17 | 2.45 | 0.408 | 53.23 KB |
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteNonQuery |
net6.0 | 1.22μs | 0.789ns | 3.06ns | 0.0142 | 0 | 0 | 1.02 KB |
| master | ExecuteNonQuery |
netcoreapp3.1 | 1.64μs | 0.842ns | 3.15ns | 0.0132 | 0 | 0 | 1.02 KB |
| master | ExecuteNonQuery |
net472 | 1.92μs | 1.37ns | 4.93ns | 0.157 | 0 | 0 | 987 B |
| #5607 | ExecuteNonQuery |
net6.0 | 1.23μs | 0.53ns | 1.98ns | 0.0143 | 0 | 0 | 1.02 KB |
| #5607 | ExecuteNonQuery |
netcoreapp3.1 | 1.71μs | 0.858ns | 2.97ns | 0.0138 | 0 | 0 | 1.02 KB |
| #5607 | ExecuteNonQuery |
net472 | 2.02μs | 1.03ns | 3.73ns | 0.156 | 0 | 0 | 987 B |
Benchmarks.Trace.ElasticsearchBenchmark - Faster :tada: Same allocations :heavy_check_mark:
Faster :tada: in #5607
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net6.0
1.115
1,330.32
1,193.13
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | CallElasticsearch |
net6.0 | 1.17μs | 0.464ns | 1.74ns | 0.0141 | 0 | 0 | 976 B |
| master | CallElasticsearch |
netcoreapp3.1 | 1.58μs | 1.16ns | 4.33ns | 0.0134 | 0 | 0 | 976 B |
| master | CallElasticsearch |
net472 | 2.53μs | 1.28ns | 4.79ns | 0.158 | 0.00127 | 0 | 995 B |
| master | CallElasticsearchAsync |
net6.0 | 1.33μs | 0.539ns | 2.09ns | 0.0133 | 0 | 0 | 952 B |
| master | CallElasticsearchAsync |
netcoreapp3.1 | 1.6μs | 0.658ns | 2.55ns | 0.0137 | 0 | 0 | 1.02 KB |
| master | CallElasticsearchAsync |
net472 | 2.62μs | 1.69ns | 6.55ns | 0.166 | 0.0013 | 0 | 1.05 KB |
| #5607 | CallElasticsearch |
net6.0 | 1.2μs | 0.477ns | 1.79ns | 0.0138 | 0 | 0 | 976 B |
| #5607 | CallElasticsearch |
netcoreapp3.1 | 1.54μs | 0.63ns | 2.36ns | 0.0129 | 0 | 0 | 976 B |
| #5607 | CallElasticsearch |
net472 | 2.47μs | 1.33ns | 5.15ns | 0.157 | 0.00123 | 0 | 995 B |
| #5607 | CallElasticsearchAsync |
net6.0 | 1.19μs | 1.15ns | 4.3ns | 0.0132 | 0 | 0 | 952 B |
| #5607 | CallElasticsearchAsync |
netcoreapp3.1 | 1.6μs | 1.33ns | 5ns | 0.0134 | 0 | 0 | 1.02 KB |
| #5607 | CallElasticsearchAsync |
net472 | 2.65μs | 1.36ns | 5.26ns | 0.167 | 0.00132 | 0 | 1.05 KB |
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteAsync |
net6.0 | 1.26μs | 0.786ns | 3.04ns | 0.0133 | 0 | 0 | 952 B |
| master | ExecuteAsync |
netcoreapp3.1 | 1.67μs | 1.64ns | 6.34ns | 0.0126 | 0 | 0 | 952 B |
| master | ExecuteAsync |
net472 | 1.76μs | 1.03ns | 3.97ns | 0.145 | 0 | 0 | 915 B |
| #5607 | ExecuteAsync |
net6.0 | 1.28μs | 0.841ns | 3.15ns | 0.0129 | 0 | 0 | 952 B |
| #5607 | ExecuteAsync |
netcoreapp3.1 | 1.6μs | 0.702ns | 2.63ns | 0.0129 | 0 | 0 | 952 B |
| #5607 | ExecuteAsync |
net472 | 1.84μs | 0.986ns | 3.82ns | 0.145 | 0 | 0 | 915 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendAsync |
net6.0 | 4.12μs | 3.79ns | 14.2ns | 0.0311 | 0 | 0 | 2.22 KB |
| master | SendAsync |
netcoreapp3.1 | 5.09μs | 2.45ns | 9.5ns | 0.0381 | 0 | 0 | 2.76 KB |
| master | SendAsync |
net472 | 7.58μs | 1.83ns | 7.1ns | 0.499 | 0 | 0 | 3.15 KB |
| #5607 | SendAsync |
net6.0 | 4.09μs | 10.7ns | 41.4ns | 0.0307 | 0 | 0 | 2.22 KB |
| #5607 | SendAsync |
netcoreapp3.1 | 5.02μs | 4.26ns | 16.5ns | 0.0351 | 0 | 0 | 2.76 KB |
| #5607 | SendAsync |
net472 | 7.57μs | 3.17ns | 12.3ns | 0.497 | 0 | 0 | 3.15 KB |
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 1.42μs | 0.56ns | 2.17ns | 0.0228 | 0 | 0 | 1.64 KB |
| master | EnrichedLog |
netcoreapp3.1 | 2.25μs | 0.918ns | 3.44ns | 0.0226 | 0 | 0 | 1.64 KB |
| master | EnrichedLog |
net472 | 2.56μs | 1.14ns | 4.4ns | 0.249 | 0 | 0 | 1.57 KB |
| #5607 | EnrichedLog |
net6.0 | 1.49μs | 0.78ns | 2.92ns | 0.0234 | 0 | 0 | 1.64 KB |
| #5607 | EnrichedLog |
netcoreapp3.1 | 2.24μs | 0.938ns | 3.51ns | 0.0216 | 0 | 0 | 1.64 KB |
| #5607 | EnrichedLog |
net472 | 2.53μs | 1.82ns | 7.06ns | 0.249 | 0 | 0 | 1.57 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 115μs | 144ns | 557ns | 0.0567 | 0 | 0 | 4.28 KB |
| master | EnrichedLog |
netcoreapp3.1 | 121μs | 159ns | 616ns | 0.0596 | 0 | 0 | 4.28 KB |
| master | EnrichedLog |
net472 | 150μs | 78.4ns | 304ns | 0.672 | 0.224 | 0 | 4.46 KB |
| #5607 | EnrichedLog |
net6.0 | 113μs | 61.8ns | 231ns | 0.0566 | 0 | 0 | 4.28 KB |
| #5607 | EnrichedLog |
netcoreapp3.1 | 118μs | 107ns | 399ns | 0.059 | 0 | 0 | 4.28 KB |
| #5607 | EnrichedLog |
net472 | 146μs | 67.3ns | 252ns | 0.655 | 0.218 | 0 | 4.46 KB |
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 3.03μs | 1.62ns | 6.29ns | 0.0318 | 0 | 0 | 2.2 KB |
| master | EnrichedLog |
netcoreapp3.1 | 4.09μs | 1.63ns | 6.12ns | 0.0307 | 0 | 0 | 2.2 KB |
| master | EnrichedLog |
net472 | 4.83μs | 2.25ns | 8.44ns | 0.319 | 0 | 0 | 2.02 KB |
| #5607 | EnrichedLog |
net6.0 | 2.98μs | 0.596ns | 2.31ns | 0.0312 | 0 | 0 | 2.2 KB |
| #5607 | EnrichedLog |
netcoreapp3.1 | 4.07μs | 0.968ns | 3.62ns | 0.0284 | 0 | 0 | 2.2 KB |
| #5607 | EnrichedLog |
net472 | 4.83μs | 2.26ns | 8.74ns | 0.32 | 0 | 0 | 2.02 KB |
Benchmarks.Trace.RedisBenchmark - Faster :tada: Same allocations :heavy_check_mark:
Faster :tada: in #5607
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.RedisBenchmark.SendReceive‑net6.0
1.129
1,445.18
1,279.59
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendReceive |
net6.0 | 1.45μs | 0.703ns | 2.72ns | 0.016 | 0 | 0 | 1.14 KB |
| master | SendReceive |
netcoreapp3.1 | 1.72μs | 1.6ns | 5.98ns | 0.0151 | 0 | 0 | 1.14 KB |
| master | SendReceive |
net472 | 2.09μs | 0.986ns | 3.82ns | 0.183 | 0 | 0 | 1.16 KB |
| #5607 | SendReceive |
net6.0 | 1.28μs | 0.782ns | 3.03ns | 0.016 | 0 | 0 | 1.14 KB |
| #5607 | SendReceive |
netcoreapp3.1 | 1.79μs | 0.646ns | 2.5ns | 0.0153 | 0 | 0 | 1.14 KB |
| #5607 | SendReceive |
net472 | 2.08μs | 2.6ns | 10.1ns | 0.183 | 0 | 0 | 1.16 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 2.66μs | 0.455ns | 1.64ns | 0.0226 | 0 | 0 | 1.6 KB |
| master | EnrichedLog |
netcoreapp3.1 | 3.97μs | 2ns | 7.48ns | 0.0219 | 0 | 0 | 1.65 KB |
| master | EnrichedLog |
net472 | 4.39μs | 1.55ns | 5.99ns | 0.322 | 0 | 0 | 2.04 KB |
| #5607 | EnrichedLog |
net6.0 | 2.83μs | 1.18ns | 4.43ns | 0.0223 | 0 | 0 | 1.6 KB |
| #5607 | EnrichedLog |
netcoreapp3.1 | 3.99μs | 1.44ns | 5.59ns | 0.0213 | 0 | 0 | 1.65 KB |
| #5607 | EnrichedLog |
net472 | 4.27μs | 3.09ns | 12ns | 0.323 | 0 | 0 | 2.04 KB |
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark:
Slower :warning: in #5607
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0
1.204
468.42
563.85
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0
1.126
401.26
451.92
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartFinishSpan |
net6.0 | 401ns | 0.194ns | 0.753ns | 0.00809 | 0 | 0 | 576 B |
| master | StartFinishSpan |
netcoreapp3.1 | 574ns | 0.749ns | 2.9ns | 0.00797 | 0 | 0 | 576 B |
| master | StartFinishSpan |
net472 | 626ns | 0.658ns | 2.55ns | 0.0917 | 0 | 0 | 578 B |
| master | StartFinishScope |
net6.0 | 468ns | 0.155ns | 0.602ns | 0.00974 | 0 | 0 | 696 B |
| master | StartFinishScope |
netcoreapp3.1 | 669ns | 0.223ns | 0.863ns | 0.00941 | 0 | 0 | 696 B |
| master | StartFinishScope |
net472 | 850ns | 0.363ns | 1.36ns | 0.104 | 0 | 0 | 658 B |
| #5607 | StartFinishSpan |
net6.0 | 452ns | 0.135ns | 0.523ns | 0.00813 | 0 | 0 | 576 B |
| #5607 | StartFinishSpan |
netcoreapp3.1 | 595ns | 0.474ns | 1.84ns | 0.00804 | 0 | 0 | 576 B |
| #5607 | StartFinishSpan |
net472 | 660ns | 0.696ns | 2.69ns | 0.0915 | 0 | 0 | 578 B |
| #5607 | StartFinishScope |
net6.0 | 564ns | 0.253ns | 0.98ns | 0.00961 | 0 | 0 | 696 B |
| #5607 | StartFinishScope |
netcoreapp3.1 | 681ns | 0.521ns | 2.02ns | 0.00961 | 0 | 0 | 696 B |
| #5607 | StartFinishScope |
net472 | 827ns | 0.218ns | 0.817ns | 0.104 | 0 | 0 | 658 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunOnMethodBegin |
net6.0 | 642ns | 0.26ns | 1.01ns | 0.00965 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
netcoreapp3.1 | 846ns | 1.76ns | 6.59ns | 0.00933 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
net472 | 1.05μs | 0.685ns | 2.65ns | 0.104 | 0 | 0 | 658 B |
| #5607 | RunOnMethodBegin |
net6.0 | 635ns | 0.205ns | 0.739ns | 0.00979 | 0 | 0 | 696 B |
| #5607 | RunOnMethodBegin |
netcoreapp3.1 | 913ns | 0.315ns | 1.22ns | 0.00922 | 0 | 0 | 696 B |
| #5607 | RunOnMethodBegin |
net472 | 1.05μs | 0.539ns | 2.09ns | 0.105 | 0 | 0 | 658 B |
Execution-Time Benchmarks Report :stopwatch:
Execution-time results for samples comparing the following branches/commits:
Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
- Welch test with statistical test for significance of 5%
- Only results indicating a difference greater than 5% and 5 ms are considered.
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.
Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).
gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5607) - mean (73ms) : 63, 84
. : milestone, 73,
master - mean (72ms) : 65, 80
. : milestone, 72,
section CallTarget+Inlining+NGEN
This PR (5607) - mean (1,018ms) : 1001, 1034
. : milestone, 1018,
master - mean (1,018ms) : 999, 1036
. : milestone, 1018,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5607) - mean (109ms) : 105, 113
. : milestone, 109,
master - mean (109ms) : 106, 112
. : milestone, 109,
section CallTarget+Inlining+NGEN
This PR (5607) - mean (711ms) : 690, 733
. : milestone, 711,
master - mean (705ms) : 682, 728
. : milestone, 705,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5607) - mean (93ms) : 90, 96
. : milestone, 93,
master - mean (92ms) : 90, 95
. : milestone, 92,
section CallTarget+Inlining+NGEN
This PR (5607) - mean (661ms) : 632, 690
. : milestone, 661,
master - mean (665ms) : 640, 690
. : milestone, 665,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5607) - mean (194ms) : 189, 199
. : milestone, 194,
master - mean (192ms) : 189, 196
. : milestone, 192,
section CallTarget+Inlining+NGEN
This PR (5607) - mean (1,117ms) : 1090, 1144
. : milestone, 1117,
master - mean (1,121ms) : 1098, 1143
. : milestone, 1121,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5607) - mean (277ms) : 272, 281
. : milestone, 277,
master - mean (278ms) : 274, 283
. : milestone, 278,
section CallTarget+Inlining+NGEN
This PR (5607) - mean (914ms) : 880, 948
. : milestone, 914,
master - mean (912ms) : 880, 944
. : milestone, 912,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5607) - mean (267ms) : 260, 274
. : milestone, 267,
master - mean (267ms) : 264, 271
. : milestone, 267,
section CallTarget+Inlining+NGEN
This PR (5607) - mean (895ms) : 866, 923
. : milestone, 895,
master - mean (889ms) : 857, 921
. : milestone, 889,
I think I just found a bug in our oracle instrumentation from the difference in snapshots between netcore21 and the rest of what we're testing.
It seems we're missing the commands executed synchronously with
ExecuteReader (while they were captured in netcore21/the older version of the driver) I'll investigate where this is coming from.