dd-trace-java icon indicating copy to clipboard operation
dd-trace-java copied to clipboard

Published 20 hours ago • verified publisher icon DataDog

Implement Remote Config tracing_enabled feature

Open nayeem-kamal opened this issue 11 months ago • 1 comments

What Does This Do

Motivation

Additional Notes

Jira ticket: [PROJ-IDENT]

nayeem-kamal avatar Mar 04 '24 13:03 nayeem-kamal

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nayeem-kamal/RC-experimental-tracing-enabled
git_commit_date 1709558898 1709137061
git_commit_sha aeb7876f22 083a28ed9e
release_version 1.31.0-SNAPSHOT~aeb7876f22 1.31.0-SNAPSHOT~083a28ed9e
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1709563676 1709563676
ci_job_id 449627086 449627086
ci_pipeline_id 29400793 29400793
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 10 performance improvements and 4 performance regressions! Performance is the same for 32 metrics, 8 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent.start better
[-34.101ms; -24.624ms] or [-2.823%; -2.038%]		 1.179s 1.208s
scenario:startup:insecure-bank:iast:BytebuddyAgent better
[-29.557ms; -23.608ms] or [-3.686%; -2.944%]		 775.344ms 801.926ms
scenario:startup:insecure-bank:iast:AppSec better
[-6.762ms; -1.826ms] or [-12.433%; -3.357%]		 50.094ms 54.388ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent.start better
[-34.668ms; -25.417ms] or [-2.894%; -2.122%]		 1.168s 1.198s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:BytebuddyAgent better
[-26.461ms; -21.520ms] or [-3.343%; -2.718%]		 767.645ms 791.636ms
scenario:startup:insecure-bank:tracing:BytebuddyAgent better
[-29.926ms; -26.336ms] or [-4.302%; -3.786%]		 667.569ms 695.701ms
scenario:startup:insecure-bank:tracing:Remote Config better
[-58.544µs; -25.603µs] or [-7.821%; -3.420%]		 706.456µs 748.529µs
scenario:startup:petclinic:appsec:Agent.start better
[-48.190ms; -41.751ms] or [-4.020%; -3.483%]		 1.154s 1.199s
scenario:startup:petclinic:appsec:BytebuddyAgent better
[-33.919ms; -30.161ms] or [-4.868%; -4.328%]		 664.788ms 696.828ms
scenario:startup:petclinic:appsec:Remote Config worse
[+19.858µs; +34.858µs] or [+3.262%; +5.726%]		 636.143µs 608.785µs
scenario:startup:petclinic:iast:Remote Config worse
[+12.483µs; +70.920µs] or [+2.112%; +11.998%]		 632.789µs 591.088µs
scenario:startup:petclinic:profiling:ProfilingAgent worse
[+21.609ms; +26.391ms] or [+23.113%; +28.227%]		 117.493ms 93.493ms
scenario:startup:petclinic:profiling:Profiling worse
[+21.610ms; +26.392ms] or [+23.108%; +28.221%]		 117.518ms 93.517ms
scenario:startup:petclinic:tracing:BytebuddyAgent better
[-43.620ms; -25.114ms] or [-6.237%; -3.591%]		 665.054ms 699.421ms
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.31.0-SNAPSHOT~083a28ed9e, baseline=1.31.0-SNAPSHOT~aeb7876f22

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.087 s) : 0, 1086724
Total [baseline] (9.175 s) : 0, 9174977
Agent [candidate] (1.055 s) : 0, 1054736
Total [candidate] (9.139 s) : 0, 9138599
section appsec
Agent [baseline] (1.199 s) : 0, 1198771
Total [baseline] (9.309 s) : 0, 9309041
Agent [candidate] (1.154 s) : 0, 1153800
Total [candidate] (9.313 s) : 0, 9313249
section iast
Agent [baseline] (1.208 s) : 0, 1207901
Total [baseline] (9.328 s) : 0, 9327646
Agent [candidate] (1.184 s) : 0, 1184230
Total [candidate] (9.427 s) : 0, 9426680
section profiling
Agent [baseline] (1.278 s) : 0, 1278471
Total [baseline] (9.38 s) : 0, 9380155
Agent [candidate] (1.283 s) : 0, 1283217
Total [candidate] (9.438 s) : 0, 9438393
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.087 s -
Agent appsec 1.199 s 112.047 ms (10.3%)
Agent iast 1.208 s 121.178 ms (11.2%)
Agent profiling 1.278 s 191.747 ms (17.6%)
Total tracing 9.175 s -
Total appsec 9.309 s 134.064 ms (1.5%)
Total iast 9.328 s 152.669 ms (1.7%)
Total profiling 9.38 s 205.178 ms (2.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.055 s -
Agent appsec 1.154 s 99.064 ms (9.4%)
Agent iast 1.184 s 129.494 ms (12.3%)
Agent profiling 1.283 s 228.481 ms (21.7%)
Total tracing 9.139 s -
Total appsec 9.313 s 174.65 ms (1.9%)
Total iast 9.427 s 288.081 ms (3.2%)
Total profiling 9.438 s 299.794 ms (3.3%) 
gantt
    title petclinic - break down per module: candidate=1.31.0-SNAPSHOT~083a28ed9e, baseline=1.31.0-SNAPSHOT~aeb7876f22

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (699.421 ms) : 0, 699421
BytebuddyAgent [candidate] (665.054 ms) : 0, 665054
GlobalTracer [baseline] (292.541 ms) : 0, 292541
GlobalTracer [candidate] (295.572 ms) : 0, 295572
AppSec [baseline] (51.767 ms) : 0, 51767
AppSec [candidate] (51.522 ms) : 0, 51522
Remote Config [baseline] (722.793 µs) : 0, 723
Remote Config [candidate] (707.687 µs) : 0, 708
Telemetry [baseline] (7.758 ms) : 0, 7758
Telemetry [candidate] (7.736 ms) : 0, 7736
section appsec
BytebuddyAgent [baseline] (696.828 ms) : 0, 696828
BytebuddyAgent [candidate] (664.788 ms) : 0, 664788
GlobalTracer [baseline] (291.343 ms) : 0, 291343
GlobalTracer [candidate] (296.079 ms) : 0, 296079
AppSec [baseline] (150.961 ms) : 0, 150961
AppSec [candidate] (151.296 ms) : 0, 151296
Remote Config [baseline] (608.785 µs) : 0, 609
Remote Config [candidate] (636.143 µs) : 0, 636
Telemetry [baseline] (6.865 ms) : 0, 6865
Telemetry [candidate] (6.808 ms) : 0, 6808
IAST [baseline] (17.761 ms) : 0, 17761
section iast
BytebuddyAgent [baseline] (801.431 ms) : 0, 801431
BytebuddyAgent [candidate] (780.489 ms) : 0, 780489
GlobalTracer [baseline] (288.635 ms) : 0, 288635
GlobalTracer [candidate] (288.958 ms) : 0, 288958
AppSec [baseline] (51.428 ms) : 0, 51428
AppSec [candidate] (50.842 ms) : 0, 50842
Remote Config [baseline] (591.088 µs) : 0, 591
Remote Config [candidate] (632.789 µs) : 0, 633
Telemetry [baseline] (7.454 ms) : 0, 7454
Telemetry [candidate] (9.808 ms) : 0, 9808
IAST [baseline] (23.918 ms) : 0, 23918
IAST [candidate] (18.938 ms) : 0, 18938
section profiling
ProfilingAgent [baseline] (93.493 ms) : 0, 93493
ProfilingAgent [candidate] (117.493 ms) : 0, 117493
BytebuddyAgent [baseline] (692.53 ms) : 0, 692530
BytebuddyAgent [candidate] (666.579 ms) : 0, 666579
GlobalTracer [baseline] (375.268 ms) : 0, 375268
GlobalTracer [candidate] (383.064 ms) : 0, 383064
AppSec [baseline] (52.717 ms) : 0, 52717
AppSec [candidate] (52.708 ms) : 0, 52708
Remote Config [baseline] (805.922 µs) : 0, 806
Remote Config [candidate] (667.809 µs) : 0, 668
Telemetry [baseline] (7.416 ms) : 0, 7416
Telemetry [candidate] (7.68 ms) : 0, 7680
Profiling [baseline] (93.517 ms) : 0, 93517
Profiling [candidate] (117.518 ms) : 0, 117518

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-03-04T14:24:22 2024-03-04T14:42:57
git_branch master nayeem-kamal/RC-experimental-tracing-enabled
git_commit_date 1709558898 1709137061
git_commit_sha aeb7876f22 083a28ed9e
release_version 1.31.0-SNAPSHOT~aeb7876f22 1.31.0-SNAPSHOT~083a28ed9e
start_time 2024-03-04T14:24:09 2024-03-04T14:42:43
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1709563676 1709563676
ci_job_id 449627086 449627086
ci_pipeline_id 29400793 29400793
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 14 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~083a28ed9e, baseline=1.31.0-SNAPSHOT~aeb7876f22
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.347 ms) : 1328, 1366
.   : milestone, 1347,
appsec (1.805 ms) : 1782, 1828
.   : milestone, 1805,
iast (1.491 ms) : 1468, 1515
.   : milestone, 1491,
profiling (1.514 ms) : 1491, 1537
.   : milestone, 1514,
tracing (1.515 ms) : 1492, 1538
.   : milestone, 1515,
section candidate
no_agent (1.344 ms) : 1325, 1363
.   : milestone, 1344,
appsec (1.79 ms) : 1766, 1814
.   : milestone, 1790,
iast (1.511 ms) : 1487, 1535
.   : milestone, 1511,
profiling (1.517 ms) : 1493, 1542
.   : milestone, 1517,
tracing (1.492 ms) : 1468, 1516
.   : milestone, 1492,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.347 ms [1.328 ms, 1.366 ms] -
appsec 1.805 ms [1.782 ms, 1.828 ms] 458.284 µs (34.0%)
iast 1.491 ms [1.468 ms, 1.515 ms] 144.47 µs (10.7%)
profiling 1.514 ms [1.491 ms, 1.537 ms] 167.315 µs (12.4%)
tracing 1.515 ms [1.492 ms, 1.538 ms] 168.49 µs (12.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.344 ms [1.325 ms, 1.363 ms] -
appsec 1.79 ms [1.766 ms, 1.814 ms] 445.668 µs (33.2%)
iast 1.511 ms [1.487 ms, 1.535 ms] 166.849 µs (12.4%)
profiling 1.517 ms [1.493 ms, 1.542 ms] 173.172 µs (12.9%)
tracing 1.492 ms [1.468 ms, 1.516 ms] 147.987 µs (11.0%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~083a28ed9e, baseline=1.31.0-SNAPSHOT~aeb7876f22
    dateFormat X
    axisFormat %s
section baseline
no_agent (364.106 µs) : 344, 385
.   : milestone, 364,
iast (476.086 µs) : 455, 497
.   : milestone, 476,
iast_FULL (537.417 µs) : 516, 558
.   : milestone, 537,
iast_GLOBAL (501.0 µs) : 479, 523
.   : milestone, 501,
iast_HARDCODED_SECRET_DISABLED (474.872 µs) : 454, 496
.   : milestone, 475,
iast_INACTIVE (447.171 µs) : 427, 468
.   : milestone, 447,
iast_TELEMETRY_OFF (475.806 µs) : 455, 497
.   : milestone, 476,
tracing (450.378 µs) : 429, 472
.   : milestone, 450,
section candidate
no_agent (365.957 µs) : 346, 386
.   : milestone, 366,
iast (487.936 µs) : 467, 509
.   : milestone, 488,
iast_FULL (539.097 µs) : 518, 560
.   : milestone, 539,
iast_GLOBAL (501.199 µs) : 480, 522
.   : milestone, 501,
iast_HARDCODED_SECRET_DISABLED (468.84 µs) : 448, 489
.   : milestone, 469,
iast_INACTIVE (448.106 µs) : 428, 468
.   : milestone, 448,
iast_TELEMETRY_OFF (469.89 µs) : 449, 490
.   : milestone, 470,
tracing (442.496 µs) : 422, 463
.   : milestone, 442,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.106 µs [343.524 µs, 384.689 µs] -
iast 476.086 µs [455.251 µs, 496.922 µs] 111.98 µs (30.8%)
iast_FULL 537.417 µs [516.41 µs, 558.425 µs] 173.311 µs (47.6%)
iast_GLOBAL 501.0 µs [479.357 µs, 522.643 µs] 136.893 µs (37.6%)
iast_HARDCODED_SECRET_DISABLED 474.872 µs [453.972 µs, 495.772 µs] 110.765 µs (30.4%)
iast_INACTIVE 447.171 µs [426.839 µs, 467.503 µs] 83.065 µs (22.8%)
iast_TELEMETRY_OFF 475.806 µs [454.826 µs, 496.785 µs] 111.699 µs (30.7%)
tracing 450.378 µs [428.846 µs, 471.909 µs] 86.271 µs (23.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.957 µs [345.878 µs, 386.036 µs] -
iast 487.936 µs [467.259 µs, 508.614 µs] 121.979 µs (33.3%)
iast_FULL 539.097 µs [518.411 µs, 559.783 µs] 173.14 µs (47.3%)
iast_GLOBAL 501.199 µs [480.155 µs, 522.243 µs] 135.242 µs (37.0%)
iast_HARDCODED_SECRET_DISABLED 468.84 µs [448.482 µs, 489.197 µs] 102.883 µs (28.1%)
iast_INACTIVE 448.106 µs [427.768 µs, 468.444 µs] 82.149 µs (22.4%)
iast_TELEMETRY_OFF 469.89 µs [449.473 µs, 490.308 µs] 103.933 µs (28.4%)
tracing 442.496 µs [422.372 µs, 462.621 µs] 76.539 µs (20.9%)

pr-commenter[bot] avatar Mar 04 '24 14:03 pr-commenter[bot]