dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard
verified publisher icon DataDog

Try to head off future build issues

Open andrewlock opened this issue 1 year ago • 4 comments

Summary of changes

  • Vendor the dotnet-install scripts into the repo
  • Replace the centos7 repo references with vault based ones

Reason for change

https://dotnet.microsoft.com/ went down recently, which meant we couldn't downnload dotnet-install, which broke a bunch of things, so vendor it.

Centos7 recently shut down their repo feed, which means you can no longer pull packages. Use vault repo instead until we deprecate centos7 entirely

Implementation details

  • Vendor the scripts
  • Replace downloading of the script with direct reference
  • do some sed to replace the centos7 repo

Test coverage

Largely, this is the test, if it all works, I think we're good

Other details

Supersedes https://github.com/DataDog/dd-trace-dotnet/pull/5759 Requires updating the VMs

andrewlock avatar Jul 03 '24 13:07 andrewlock

Datadog Report

Branch report: andrew/ci/try-fix-builds Commit report: 9001630 Test service: dd-trace-dotnet

:white_check_mark: 0 Failed, 298881 Passed, 1587 Skipped, 12h 23m 48.78s Total Time

datadog-ddstaging[bot] avatar Jul 03 '24 13:07 datadog-ddstaging[bot]

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

andrewlock avatar Jul 03 '24 18:07 andrewlock

Benchmarks Report for tracer :snail:

Benchmarks for #5770 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.115
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 7.78μs 44.1ns 330ns 0.0154 0.00768 0 5.43 KB
master StartStopWithChild netcoreapp3.1 9.99μs 55.4ns 355ns 0.0255 0.0153 0.0051 5.62 KB
master StartStopWithChild net472 16μs 51ns 191ns 1.04 0.334 0.111 6.06 KB
#5770 StartStopWithChild net6.0 7.78μs 43.4ns 301ns 0.0159 0.00795 0 5.42 KB
#5770 StartStopWithChild netcoreapp3.1 9.98μs 56.1ns 355ns 0.0147 0.00489 0 5.62 KB
#5770 StartStopWithChild net472 15.9μs 53.4ns 200ns 1.02 0.303 0.0957 6.07 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 468μs 362ns 1.4μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 633μs 477ns 1.79μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 842μs 582ns 2.26μs 0.419 0 0 3.3 KB
#5770 WriteAndFlushEnrichedTraces net6.0 473μs 190ns 736ns 0 0 0 2.7 KB
#5770 WriteAndFlushEnrichedTraces netcoreapp3.1 639μs 297ns 1.15μs 0 0 0 2.7 KB
#5770 WriteAndFlushEnrichedTraces net472 831μs 269ns 1.04μs 0.414 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 187μs 1.06μs 7.12μs 0.213 0 0 18.45 KB
master SendRequest netcoreapp3.1 216μs 1.25μs 10.5μs 0.227 0 0 20.61 KB
master SendRequest net472 0.00159ns 0.000462ns 0.00179ns 0 0 0 0 b
#5770 SendRequest net6.0 189μs 1.04μs 6.15μs 0.184 0 0 18.45 KB
#5770 SendRequest netcoreapp3.1 219μs 1.24μs 8.29μs 0.218 0 0 20.61 KB
#5770 SendRequest net472 0.000204ns 0.000176ns 0.000634ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 570μs 3.1μs 18.1μs 0.561 0 0 41.66 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 683μs 2.89μs 13.6μs 0.349 0 0 41.79 KB
master WriteAndFlushEnrichedTraces net472 883μs 4.07μs 15.2μs 8.3 2.62 0.437 53.34 KB
#5770 WriteAndFlushEnrichedTraces net6.0 577μs 2.81μs 11.6μs 0.566 0 0 41.57 KB
#5770 WriteAndFlushEnrichedTraces netcoreapp3.1 674μs 3.28μs 13.5μs 0.332 0 0 41.81 KB
#5770 WriteAndFlushEnrichedTraces net472 856μs 4μs 15.5μs 8.87 2.53 0.422 53.29 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.28μs 1.45ns 5.63ns 0.0141 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.69μs 1.05ns 4.06ns 0.0142 0 0 1.02 KB
master ExecuteNonQuery net472 2.02μs 2.05ns 7.1ns 0.156 0 0 987 B
#5770 ExecuteNonQuery net6.0 1.26μs 1.68ns 6.52ns 0.0145 0 0 1.02 KB
#5770 ExecuteNonQuery netcoreapp3.1 1.77μs 1.37ns 5.29ns 0.0135 0 0 1.02 KB
#5770 ExecuteNonQuery net472 1.95μs 1.95ns 7.54ns 0.156 0 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.23μs 0.353ns 1.22ns 0.0135 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.57μs 0.618ns 2.39ns 0.0135 0 0 976 B
master CallElasticsearch net472 2.55μs 1.24ns 4.65ns 0.157 0 0 995 B
master CallElasticsearchAsync net6.0 1.34μs 0.981ns 3.67ns 0.0134 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.53μs 0.633ns 2.45ns 0.0137 0 0 1.02 KB
master CallElasticsearchAsync net472 2.43μs 5.08ns 19.7ns 0.166 0 0 1.05 KB
#5770 CallElasticsearch net6.0 1.15μs 0.979ns 3.79ns 0.0138 0 0 976 B
#5770 CallElasticsearch netcoreapp3.1 1.59μs 3.56ns 13.8ns 0.0128 0 0 976 B
#5770 CallElasticsearch net472 2.38μs 1.82ns 7.03ns 0.157 0.0012 0 995 B
#5770 CallElasticsearchAsync net6.0 1.22μs 0.871ns 3.26ns 0.013 0 0 952 B
#5770 CallElasticsearchAsync netcoreapp3.1 1.66μs 0.716ns 2.58ns 0.0136 0 0 1.02 KB
#5770 CallElasticsearchAsync net472 2.53μs 2.16ns 8.38ns 0.167 0.00127 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.2μs 0.567ns 2.12ns 0.0132 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.65μs 1.72ns 6.22ns 0.0125 0 0 952 B
master ExecuteAsync net472 1.8μs 0.948ns 3.67ns 0.145 0 0 915 B
#5770 ExecuteAsync net6.0 1.21μs 0.444ns 1.6ns 0.0134 0 0 952 B
#5770 ExecuteAsync netcoreapp3.1 1.64μs 1.14ns 4.26ns 0.0129 0 0 952 B
#5770 ExecuteAsync net472 1.77μs 0.943ns 3.65ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.03μs 2.33ns 8.7ns 0.0306 0 0 2.22 KB
master SendAsync netcoreapp3.1 5.19μs 1.88ns 7.02ns 0.0363 0 0 2.76 KB
master SendAsync net472 7.75μs 2.25ns 8.73ns 0.499 0 0 3.15 KB
#5770 SendAsync net6.0 3.97μs 1.93ns 7.49ns 0.0299 0 0 2.22 KB
#5770 SendAsync netcoreapp3.1 5.07μs 4.61ns 17.8ns 0.0353 0 0 2.76 KB
#5770 SendAsync net472 7.82μs 1.47ns 5.7ns 0.498 0 0 3.15 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.49μs 0.787ns 3.05ns 0.0231 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.26μs 3.4ns 13.2ns 0.0226 0 0 1.64 KB
master EnrichedLog net472 2.74μs 1.85ns 7.18ns 0.249 0 0 1.57 KB
#5770 EnrichedLog net6.0 1.53μs 4.35ns 16.8ns 0.023 0 0 1.64 KB
#5770 EnrichedLog netcoreapp3.1 2.3μs 1.03ns 3.86ns 0.022 0 0 1.64 KB
#5770 EnrichedLog net472 2.76μs 2ns 7.73ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 114μs 306ns 1.15μs 0.0577 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 120μs 278ns 1.08μs 0.0592 0 0 4.28 KB
master EnrichedLog net472 147μs 212ns 823ns 0.665 0.222 0 4.46 KB
#5770 EnrichedLog net6.0 118μs 376ns 1.46μs 0.0587 0 0 4.28 KB
#5770 EnrichedLog netcoreapp3.1 118μs 274ns 1.06μs 0.059 0 0 4.28 KB
#5770 EnrichedLog net472 150μs 244ns 947ns 0.674 0.225 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3μs 0.771ns 2.88ns 0.03 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.36μs 1.29ns 4.67ns 0.03 0 0 2.2 KB
master EnrichedLog net472 4.9μs 1.8ns 6.98ns 0.319 0 0 2.02 KB
#5770 EnrichedLog net6.0 3.15μs 1.07ns 4.14ns 0.03 0 0 2.2 KB
#5770 EnrichedLog netcoreapp3.1 4.4μs 3.23ns 12.5ns 0.0285 0 0 2.2 KB
#5770 EnrichedLog net472 4.84μs 1.39ns 5.38ns 0.321 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.3μs 0.857ns 3.32ns 0.0156 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.79μs 0.976ns 3.78ns 0.0153 0 0 1.14 KB
master SendReceive net472 2.08μs 1.02ns 3.82ns 0.183 0 0 1.16 KB
#5770 SendReceive net6.0 1.37μs 0.427ns 1.66ns 0.0157 0 0 1.14 KB
#5770 SendReceive netcoreapp3.1 1.73μs 1.08ns 4.2ns 0.0156 0 0 1.14 KB
#5770 SendReceive net472 2.25μs 1.07ns 4.12ns 0.183 0.00112 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.71μs 0.516ns 2ns 0.0217 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 3.91μs 2.67ns 10.3ns 0.0216 0 0 1.65 KB
master EnrichedLog net472 4.37μs 1.25ns 4.66ns 0.323 0 0 2.04 KB
#5770 EnrichedLog net6.0 2.74μs 0.588ns 2.28ns 0.0218 0 0 1.6 KB
#5770 EnrichedLog netcoreapp3.1 3.95μs 3.78ns 14.7ns 0.0217 0 0 1.65 KB
#5770 EnrichedLog net472 4.42μs 1.75ns 6.78ns 0.324 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 397ns 0.213ns 0.825ns 0.00801 0 0 576 B
master StartFinishSpan netcoreapp3.1 666ns 0.438ns 1.7ns 0.00774 0 0 576 B
master StartFinishSpan net472 567ns 0.285ns 1.1ns 0.0918 0 0 578 B
master StartFinishScope net6.0 490ns 0.149ns 0.578ns 0.00974 0 0 696 B
master StartFinishScope netcoreapp3.1 697ns 0.358ns 1.39ns 0.00942 0 0 696 B
master StartFinishScope net472 913ns 0.876ns 3.39ns 0.104 0 0 658 B
#5770 StartFinishSpan net6.0 393ns 0.215ns 0.835ns 0.00812 0 0 576 B
#5770 StartFinishSpan netcoreapp3.1 613ns 0.252ns 0.943ns 0.00771 0 0 576 B
#5770 StartFinishSpan net472 570ns 0.269ns 1.04ns 0.0917 0 0 578 B
#5770 StartFinishScope net6.0 486ns 0.283ns 1.1ns 0.00976 0 0 696 B
#5770 StartFinishScope netcoreapp3.1 725ns 1.52ns 5.88ns 0.00944 0 0 696 B
#5770 StartFinishScope net472 875ns 0.835ns 3.01ns 0.105 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Faster :tada: Same allocations :heavy_check_mark:

Faster :tada: in #5770

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.115 677.94 608.21

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 678ns 0.279ns 1.08ns 0.00987 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 945ns 0.656ns 2.54ns 0.00948 0 0 696 B
master RunOnMethodBegin net472 1.11μs 0.407ns 1.58ns 0.104 0 0 658 B
#5770 RunOnMethodBegin net6.0 608ns 0.408ns 1.58ns 0.00985 0 0 696 B
#5770 RunOnMethodBegin netcoreapp3.1 940ns 0.353ns 1.27ns 0.00944 0 0 696 B
#5770 RunOnMethodBegin net472 1.09μs 1.66ns 6.44ns 0.104 0 0 658 B

andrewlock avatar Jul 03 '24 23:07 andrewlock

Throughput/Crank Report :zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5770) (11.706M)   : 0, 11705613
    master (11.924M)   : 0, 11924044
    benchmarks/2.9.0 (11.883M)   : 0, 11882814

    section Automatic
    This PR (5770) (7.950M)   : 0, 7950178
    master (7.828M)   : 0, 7827944
    benchmarks/2.9.0 (8.446M)   : 0, 8445613

    section Trace stats
    master (8.153M)   : 0, 8153092

    section Manual
    master (11.617M)   : 0, 11616636

    section Manual + Automatic
    This PR (5770) (7.256M)   : 0, 7256160
    master (7.365M)   : 0, 7364918

    section DD_TRACE_ENABLED=0
    master (10.799M)   : 0, 10798925


gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5770) (9.438M)   : 0, 9438129
    master (9.551M)   : 0, 9551120
    benchmarks/2.9.0 (9.760M)   : 0, 9759697

    section Automatic
    This PR (5770) (6.588M)   : 0, 6587606
    master (6.610M)   : 0, 6609773

    section Trace stats
    master (6.890M)   : 0, 6889536

    section Manual
    master (9.385M)   : 0, 9384897

    section Manual + Automatic
    This PR (5770) (6.197M)   : 0, 6196862
    master (6.104M)   : 0, 6103569

    section DD_TRACE_ENABLED=0
    master (8.918M)   : 0, 8917738


gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5770) (9.999M)   : 0, 9998839

    section Automatic
    This PR (5770) (6.705M)   : 0, 6705428

    section Manual + Automatic
    This PR (5770) (6.291M)   : 0, 6291119

andrewlock avatar Jul 04 '24 02:07 andrewlock

When this is merged, we should also bump to the latest .NET 8 build version as it likely includes fixes we need

andrewlock avatar Aug 02 '24 11:08 andrewlock