dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard
verified publisher icon DataDog

[Dynamic Instrumentation] DEBUG-2530 EL `instanceof` supports all types

Open dudikeleti opened this issue 1 year ago • 3 comments

Summary of changes

This PR add the ability to use instanceof expression for any type + uses a cache for types so we won't have to lookup for types in all assemblies if we have already seen this type before.

Reason for change

Currently the instanceof expression supports only types from mscorlib or for current executing assembly. We want to let the customer to use this expression with any type.

Test coverage

DebuggerExpressionLanguageTests MemoryCheckerTests ThreadSafeConfigurableCacheTests

dudikeleti avatar Aug 02 '24 16:08 dudikeleti

Datadog Report

Branch report: dudik/el-instanceof-support-more-types Commit report: de27be0 Test service: dd-trace-dotnet

:x: 52 Failed (0 Known Flaky), 282639 Passed, 2160 Skipped, 23h 24m 3.21s Total Time :snowflake: 13 New Flaky

:x: Failed Tests (52)

This report shows up to 5 failed tests.

  • CheckUnixMemory_ValidMemAvailable_ReturnsExpectedResult - Datadog.Trace.Tests.Debugger.MemoryCheckerTests - Details

    Expand for error 
    ssert.True() Failure
xpected: True
ctual:   False

  • CheckUnixMemory_ValidMemAvailable_ReturnsExpectedResult - Datadog.Trace.Tests.Debugger.MemoryCheckerTests - Details

    Expand for error 
    ssert.True() Failure
xpected: True
ctual:   False

  • CheckUnixMemory_ValidMemAvailable_ReturnsExpectedResult - Datadog.Trace.Tests.Debugger.MemoryCheckerTests - Details

    Expand for error 
    ssert.True() Failure
xpected: True
ctual:   False

  • CheckUnixMemory_ValidMemAvailable_ReturnsExpectedResult - Datadog.Trace.Tests.Debugger.MemoryCheckerTests - Details

    Expand for error 
    ssert.True() Failure
xpected: True
ctual:   False

  • CheckUnixMemory_ValidMemAvailable_ReturnsExpectedResult - Datadog.Trace.Tests.Debugger.MemoryCheckerTests - Details

    Expand for error 
    ssert.True() Failure
xpected: True
ctual:   False

New Flaky Tests (13)

  • TestConditions - Datadog.Trace.Tests.Debugger.DebuggerExpressionLanguageTests

  • TestConditions - Datadog.Trace.Tests.Debugger.DebuggerExpressionLanguageTests - Last Failure

    Expand for error 
    esults do not match.
ifferences:
eceived: DebuggerExpressionLanguageTests.IsInstanceOfNotQualifiedName.received.txt
erified: DebuggerExpressionLanguageTests.IsInstanceOfNotQualifiedName.verified.txt
eceived Content:
ondition:
son:

 "instanceof": [
   {
..

  • TestConditions - Datadog.Trace.Tests.Debugger.DebuggerExpressionLanguageTests - Last Failure

    Expand for error 
    esults do not match.
ifferences:
eceived: DebuggerExpressionLanguageTests.IsInstanceOfNotQualifiedName.received.txt
erified: DebuggerExpressionLanguageTests.IsInstanceOfNotQualifiedName.verified.txt
eceived Content:
ondition:
son:

 "instanceof": [
   {
..

  • TestConditions - Datadog.Trace.Tests.Debugger.DebuggerExpressionLanguageTests - Last Failure

    Expand for error 
    esults do not match.
ifferences:
eceived: DebuggerExpressionLanguageTests.IsInstanceOfNotQualifiedName.received.txt
erified: DebuggerExpressionLanguageTests.IsInstanceOfNotQualifiedName.verified.txt
eceived Content:
ondition:
son:

 "instanceof": [
   {
..

  • TestConditions - Datadog.Trace.Tests.Debugger.DebuggerExpressionLanguageTests

datadog-ddstaging[bot] avatar Aug 02 '24 16:08 datadog-ddstaging[bot]

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5844) - mean (73ms)  : 65, 81
     .   : milestone, 73,
    master - mean (74ms)  : 64, 84
     .   : milestone, 74,

    section CallTarget+Inlining+NGEN
    This PR (5844) - mean (1,076ms)  : 1046, 1105
     .   : milestone, 1076,
    master - mean (1,063ms)  : 1041, 1084
     .   : milestone, 1063,


gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5844) - mean (109ms)  : 106, 113
     .   : milestone, 109,
    master - mean (110ms)  : 105, 115
     .   : milestone, 110,

    section CallTarget+Inlining+NGEN
    This PR (5844) - mean (748ms)  : 719, 777
     .   : milestone, 748,
    master - mean (747ms)  : 727, 767
     .   : milestone, 747,


gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5844) - mean (93ms)  : 88, 98
     .   : milestone, 93,
    master - mean (92ms)  : 88, 96
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (5844) - mean (702ms)  : 683, 720
     .   : milestone, 702,
    master - mean (703ms)  : 684, 721
     .   : milestone, 703,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5844) - mean (192ms)  : 189, 196
     .   : milestone, 192,
    master - mean (192ms)  : 189, 196
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (5844) - mean (1,166ms)  : 1135, 1197
     .   : milestone, 1166,
    master - mean (1,169ms)  : 1139, 1200
     .   : milestone, 1169,


gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5844) - mean (277ms)  : 274, 281
     .   : milestone, 277,
    master - mean (275ms)  : 271, 280
     .   : milestone, 275,

    section CallTarget+Inlining+NGEN
    This PR (5844) - mean (924ms)  : 895, 953
     .   : milestone, 924,
    master - mean (921ms)  : 896, 947
     .   : milestone, 921,


gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5844) - mean (265ms)  : 260, 269
     .   : milestone, 265,
    master - mean (266ms)  : 262, 270
     .   : milestone, 266,

    section CallTarget+Inlining+NGEN
    This PR (5844) - mean (904ms)  : 884, 923
     .   : milestone, 904,
    master - mean (905ms)  : 883, 928
     .   : milestone, 905,

andrewlock avatar Aug 02 '24 16:08 andrewlock

Benchmarks Report for tracer :snail:

Benchmarks for #5844 compared to master:

  • 2 benchmarks are slower, with geometric mean 1.138
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 7.56μs 41.9ns 269ns 0.015 0.00751 0 5.43 KB
master StartStopWithChild netcoreapp3.1 9.91μs 56.7ns 416ns 0.0197 0.00986 0 5.62 KB
master StartStopWithChild net472 16μs 51.9ns 201ns 1.01 0.289 0.086 6.06 KB
#5844 StartStopWithChild net6.0 7.62μs 43.1ns 299ns 0.0156 0.00778 0 5.43 KB
#5844 StartStopWithChild netcoreapp3.1 10.1μs 56.4ns 370ns 0.0197 0.00986 0 5.62 KB
#5844 StartStopWithChild net472 15.9μs 54.5ns 211ns 1.03 0.311 0.104 6.07 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 471μs 199ns 746ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 631μs 274ns 989ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 826μs 350ns 1.31μs 0.411 0 0 3.3 KB
#5844 WriteAndFlushEnrichedTraces net6.0 484μs 305ns 1.18μs 0 0 0 2.7 KB
#5844 WriteAndFlushEnrichedTraces netcoreapp3.1 644μs 714ns 2.77μs 0 0 0 2.7 KB
#5844 WriteAndFlushEnrichedTraces net472 819μs 555ns 2.15μs 0.408 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 204μs 1.19μs 11.1μs 0.189 0 0 18.45 KB
master SendRequest netcoreapp3.1 219μs 1.26μs 10.3μs 0.207 0 0 20.61 KB
master SendRequest net472 0.00431ns 0.00126ns 0.00487ns 0 0 0 0 b
#5844 SendRequest net6.0 196μs 1.14μs 10.2μs 0.205 0 0 18.45 KB
#5844 SendRequest netcoreapp3.1 215μs 1.24μs 10μs 0.224 0 0 20.61 KB
#5844 SendRequest net472 0.00185ns 0.000875ns 0.00339ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 563μs 2.39μs 8.93μs 0.558 0 0 41.62 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 699μs 3.8μs 21.8μs 0.377 0 0 41.76 KB
master WriteAndFlushEnrichedTraces net472 862μs 3.21μs 12.4μs 8.45 2.53 0.422 53.3 KB
#5844 WriteAndFlushEnrichedTraces net6.0 568μs 2.75μs 11.7μs 0.568 0 0 41.71 KB
#5844 WriteAndFlushEnrichedTraces netcoreapp3.1 685μs 3.55μs 19.5μs 0.332 0 0 41.69 KB
#5844 WriteAndFlushEnrichedTraces net472 875μs 2.83μs 10.6μs 8.45 2.53 0.422 53.29 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.18μs 0.842ns 3.26ns 0.0142 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.73μs 2.02ns 7.84ns 0.0138 0 0 1.02 KB
master ExecuteNonQuery net472 2.04μs 2.84ns 11ns 0.157 0 0 987 B
#5844 ExecuteNonQuery net6.0 1.25μs 0.756ns 2.93ns 0.0144 0 0 1.02 KB
#5844 ExecuteNonQuery netcoreapp3.1 1.75μs 1.8ns 6.95ns 0.0139 0 0 1.02 KB
#5844 ExecuteNonQuery net472 2μs 1.89ns 7.33ns 0.157 0 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.18μs 0.374ns 1.45ns 0.0136 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.53μs 0.643ns 2.41ns 0.013 0 0 976 B
master CallElasticsearch net472 2.59μs 2.34ns 8.76ns 0.158 0 0 995 B
master CallElasticsearchAsync net6.0 1.33μs 1.68ns 6.29ns 0.0134 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.64μs 0.491ns 1.77ns 0.0139 0 0 1.02 KB
master CallElasticsearchAsync net472 2.73μs 2.7ns 10.5ns 0.166 0 0 1.05 KB
#5844 CallElasticsearch net6.0 1.2μs 0.673ns 2.52ns 0.0137 0 0 976 B
#5844 CallElasticsearch netcoreapp3.1 1.64μs 0.963ns 3.6ns 0.0131 0 0 976 B
#5844 CallElasticsearch net472 2.55μs 8.5ns 32.9ns 0.157 0 0 995 B
#5844 CallElasticsearchAsync net6.0 1.3μs 0.545ns 2.11ns 0.0131 0 0 952 B
#5844 CallElasticsearchAsync netcoreapp3.1 1.6μs 6.17ns 23.9ns 0.0142 0 0 1.02 KB
#5844 CallElasticsearchAsync net472 2.74μs 3.05ns 11.8ns 0.166 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.33μs 0.631ns 2.27ns 0.0133 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.58μs 0.816ns 3.05ns 0.0126 0 0 952 B
master ExecuteAsync net472 1.76μs 0.609ns 2.28ns 0.145 0 0 915 B
#5844 ExecuteAsync net6.0 1.31μs 0.928ns 3.47ns 0.0131 0 0 952 B
#5844 ExecuteAsync netcoreapp3.1 1.62μs 2.29ns 8.89ns 0.0122 0 0 952 B
#5844 ExecuteAsync net472 1.76μs 0.932ns 3.61ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.22μs 2.09ns 8.08ns 0.0298 0 0 2.22 KB
master SendAsync netcoreapp3.1 5.02μs 1.47ns 5.49ns 0.0351 0 0 2.76 KB
master SendAsync net472 7.59μs 1.78ns 6.89ns 0.497 0 0 3.15 KB
#5844 SendAsync net6.0 4.12μs 1.68ns 6.51ns 0.0308 0 0 2.22 KB
#5844 SendAsync netcoreapp3.1 4.98μs 1.85ns 7.16ns 0.0371 0 0 2.76 KB
#5844 SendAsync net472 7.65μs 4ns 15.5ns 0.499 0 0 3.15 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.55μs 1.07ns 4.13ns 0.0234 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.24μs 1.56ns 6.05ns 0.0224 0 0 1.64 KB
master EnrichedLog net472 2.7μs 2.14ns 7.72ns 0.249 0 0 1.57 KB
#5844 EnrichedLog net6.0 1.51μs 1.03ns 3.86ns 0.0233 0 0 1.64 KB
#5844 EnrichedLog netcoreapp3.1 2.18μs 2.4ns 9.29ns 0.022 0 0 1.64 KB
#5844 EnrichedLog net472 2.65μs 0.836ns 3.24ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 115μs 162ns 626ns 0.057 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 120μs 181ns 700ns 0.0598 0 0 4.28 KB
master EnrichedLog net472 146μs 204ns 761ns 0.664 0.221 0 4.46 KB
#5844 EnrichedLog net6.0 115μs 317ns 1.23μs 0.057 0 0 4.28 KB
#5844 EnrichedLog netcoreapp3.1 122μs 282ns 1.09μs 0 0 0 4.28 KB
#5844 EnrichedLog net472 149μs 295ns 1.14μs 0.675 0.225 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.89μs 1.6ns 5.98ns 0.0304 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.21μs 2.17ns 8.42ns 0.0294 0 0 2.2 KB
master EnrichedLog net472 4.71μs 1.63ns 6.3ns 0.319 0 0 2.02 KB
#5844 EnrichedLog net6.0 3.11μs 0.801ns 3ns 0.0308 0 0 2.2 KB
#5844 EnrichedLog netcoreapp3.1 4.13μs 1.92ns 7.44ns 0.0289 0 0 2.2 KB
#5844 EnrichedLog net472 4.68μs 1.07ns 4.14ns 0.319 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.36μs 0.937ns 3.63ns 0.0157 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.81μs 0.898ns 3.48ns 0.0153 0 0 1.14 KB
master SendReceive net472 2.07μs 1.51ns 5.84ns 0.183 0.00103 0 1.16 KB
#5844 SendReceive net6.0 1.37μs 4.2ns 16.3ns 0.0162 0 0 1.14 KB
#5844 SendReceive netcoreapp3.1 1.77μs 1.29ns 4.67ns 0.0158 0 0 1.14 KB
#5844 SendReceive net472 2.07μs 2.49ns 9.3ns 0.183 0.00106 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.96μs 0.937ns 3.63ns 0.0222 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 3.94μs 1.88ns 7.28ns 0.0217 0 0 1.65 KB
master EnrichedLog net472 4.32μs 2.55ns 9.53ns 0.323 0 0 2.04 KB
#5844 EnrichedLog net6.0 2.71μs 2.04ns 7.91ns 0.0218 0 0 1.6 KB
#5844 EnrichedLog netcoreapp3.1 3.99μs 3.44ns 13.3ns 0.0219 0 0 1.65 KB
#5844 EnrichedLog net472 4.34μs 8.58ns 33.2ns 0.322 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #5844

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 1.158 659.67 763.81

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 465ns 0.174ns 0.673ns 0.00817 0 0 576 B
master StartFinishSpan netcoreapp3.1 603ns 0.179ns 0.693ns 0.00787 0 0 576 B
master StartFinishSpan net472 625ns 0.246ns 0.952ns 0.0917 0 0 578 B
master StartFinishScope net6.0 497ns 0.248ns 0.959ns 0.00973 0 0 696 B
master StartFinishScope netcoreapp3.1 660ns 0.362ns 1.25ns 0.00955 0 0 696 B
master StartFinishScope net472 862ns 0.628ns 2.43ns 0.105 0 0 658 B
#5844 StartFinishSpan net6.0 451ns 0.0842ns 0.304ns 0.00819 0 0 576 B
#5844 StartFinishSpan netcoreapp3.1 563ns 0.631ns 2.44ns 0.00768 0 0 576 B
#5844 StartFinishSpan net472 618ns 1.56ns 5.85ns 0.0917 0 0 578 B
#5844 StartFinishScope net6.0 486ns 0.106ns 0.412ns 0.00977 0 0 696 B
#5844 StartFinishScope netcoreapp3.1 763ns 0.526ns 2.04ns 0.00956 0 0 696 B
#5844 StartFinishScope net472 915ns 1.33ns 5.16ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower :warning: Same allocations :heavy_check_mark:

Slower :warning: in #5844

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.118 589.59 659.03

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 590ns 0.203ns 0.786ns 0.00981 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 878ns 0.51ns 1.97ns 0.00927 0 0 696 B
master RunOnMethodBegin net472 1.06μs 0.306ns 1.18ns 0.104 0 0 658 B
#5844 RunOnMethodBegin net6.0 659ns 0.476ns 1.84ns 0.00961 0 0 696 B
#5844 RunOnMethodBegin netcoreapp3.1 945ns 1.26ns 4.56ns 0.00944 0 0 696 B
#5844 RunOnMethodBegin net472 1.16μs 0.33ns 1.28ns 0.104 0 0 658 B

andrewlock avatar Aug 02 '24 17:08 andrewlock