dd-trace-java icon indicating copy to clipboard operation
dd-trace-java copied to clipboard

Published 20 hours ago • verified publisher icon DataDog

Api Data Source Security

Open ValentinZakharov opened this issue 1 year ago • 1 comments
trafficstars

What Does This Do

The innovation week project

Motivation

Additional Notes

Jira ticket: [PROJ-IDENT]

ValentinZakharov avatar Nov 28 '23 08:11 ValentinZakharov

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master vzakharov/api_data_source_security
git_commit_date 1701696880 1701715830
git_commit_sha cc4d3ab92e 0648da357d
release_version 1.25.0-SNAPSHOT~cc4d3ab92e 1.25.0-SNAPSHOT~0648da357d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1701718480 1701718480
ci_job_id 380765004 380765004
ci_pipeline_id 24202335 24202335
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 2 performance improvements and 4 performance regressions! Performance is the same for 36 metrics, 12 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:tracing:AppSec worse
[+6.539ms; +8.034ms] or [+13.466%; +16.543%]		 55.848ms 48.562ms
scenario:startup:petclinic:appsec:GlobalTracer better
[-10.564ms; -7.682ms] or [-3.447%; -2.506%]		 297.392ms 306.515ms
scenario:startup:petclinic:appsec:AppSec worse
[+9.398ms; +11.077ms] or [+6.833%; +8.054%]		 147.781ms 137.544ms
scenario:startup:petclinic:profiling:GlobalTracer better
[-10.697ms; -8.263ms] or [-2.831%; -2.187%]		 368.396ms 377.876ms
scenario:startup:petclinic:profiling:AppSec worse
[+6.466ms; +7.372ms] or [+13.325%; +15.191%]		 55.445ms 48.527ms
scenario:startup:petclinic:tracing:AppSec worse
[+4.789ms; +6.801ms] or [+9.774%; +13.880%]		 54.793ms 48.998ms
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.25.0-SNAPSHOT~0648da357d, baseline=1.25.0-SNAPSHOT~cc4d3ab92e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.048 s) : 0, 1048301
Total [baseline] (9.296 s) : 0, 9295777
Agent [candidate] (1.038 s) : 0, 1038408
Total [candidate] (9.311 s) : 0, 9311005
section appsec
Agent [baseline] (1.134 s) : 0, 1133648
Total [baseline] (9.394 s) : 0, 9393571
Agent [candidate] (1.13 s) : 0, 1130415
Total [candidate] (9.432 s) : 0, 9431931
section iast
Agent [baseline] (1.159 s) : 0, 1159373
Total [baseline] (9.6 s) : 0, 9600332
Agent [candidate] (1.159 s) : 0, 1159449
Total [candidate] (9.541 s) : 0, 9541395
section profiling
Agent [baseline] (1.235 s) : 0, 1234540
Total [baseline] (9.61 s) : 0, 9610459
Agent [candidate] (1.233 s) : 0, 1232635
Total [candidate] (9.602 s) : 0, 9602143
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.048 s -
Agent appsec 1.134 s 85.346 ms (8.1%)
Agent iast 1.159 s 111.071 ms (10.6%)
Agent profiling 1.235 s 186.239 ms (17.8%)
Total tracing 9.296 s -
Total appsec 9.394 s 97.794 ms (1.1%)
Total iast 9.6 s 304.555 ms (3.3%)
Total profiling 9.61 s 314.682 ms (3.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.038 s -
Agent appsec 1.13 s 92.007 ms (8.9%)
Agent iast 1.159 s 121.041 ms (11.7%)
Agent profiling 1.233 s 194.227 ms (18.7%)
Total tracing 9.311 s -
Total appsec 9.432 s 120.927 ms (1.3%)
Total iast 9.541 s 230.391 ms (2.5%)
Total profiling 9.602 s 291.139 ms (3.1%) 
gantt
    title petclinic - break down per module: candidate=1.25.0-SNAPSHOT~0648da357d, baseline=1.25.0-SNAPSHOT~cc4d3ab92e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (649.241 ms) : 0, 649241
BytebuddyAgent [candidate] (643.545 ms) : 0, 643545
GlobalTracer [baseline] (307.671 ms) : 0, 307671
GlobalTracer [candidate] (297.83 ms) : 0, 297830
AppSec [baseline] (48.998 ms) : 0, 48998
AppSec [candidate] (54.793 ms) : 0, 54793
Remote Config [baseline] (679.97 µs) : 0, 680
Remote Config [candidate] (697.637 µs) : 0, 698
Telemetry [baseline] (7.183 ms) : 0, 7183
Telemetry [candidate] (7.307 ms) : 0, 7307
section appsec
BytebuddyAgent [baseline] (646.387 ms) : 0, 646387
BytebuddyAgent [candidate] (643.615 ms) : 0, 643615
GlobalTracer [baseline] (306.515 ms) : 0, 306515
GlobalTracer [candidate] (297.392 ms) : 0, 297392
AppSec [baseline] (137.544 ms) : 0, 137544
AppSec [candidate] (147.781 ms) : 0, 147781
Remote Config [baseline] (646.0 µs) : 0, 646
Remote Config [candidate] (662.676 µs) : 0, 663
Telemetry [baseline] (8.212 ms) : 0, 8212
Telemetry [candidate] (6.791 ms) : 0, 6791
section iast
BytebuddyAgent [baseline] (766.349 ms) : 0, 766349
BytebuddyAgent [candidate] (766.338 ms) : 0, 766338
GlobalTracer [baseline] (285.105 ms) : 0, 285105
GlobalTracer [candidate] (279.008 ms) : 0, 279008
AppSec [baseline] (46.114 ms) : 0, 46114
AppSec [candidate] (56.454 ms) : 0, 56454
Remote Config [baseline] (627.006 µs) : 0, 627
Remote Config [candidate] (584.037 µs) : 0, 584
Telemetry [baseline] (6.58 ms) : 0, 6580
Telemetry [candidate] (6.531 ms) : 0, 6531
IAST [baseline] (20.211 ms) : 0, 20211
IAST [candidate] (16.148 ms) : 0, 16148
section profiling
BytebuddyAgent [baseline] (656.99 ms) : 0, 656990
BytebuddyAgent [candidate] (658.645 ms) : 0, 658645
GlobalTracer [baseline] (377.876 ms) : 0, 377876
GlobalTracer [candidate] (368.396 ms) : 0, 368396
AppSec [baseline] (48.527 ms) : 0, 48527
AppSec [candidate] (55.445 ms) : 0, 55445
Remote Config [baseline] (714.492 µs) : 0, 714
Remote Config [candidate] (674.244 µs) : 0, 674
Telemetry [baseline] (7.416 ms) : 0, 7416
Telemetry [candidate] (7.396 ms) : 0, 7396
ProfilingAgent [baseline] (88.725 ms) : 0, 88725
ProfilingAgent [candidate] (87.591 ms) : 0, 87591
Profiling [baseline] (88.75 ms) : 0, 88750
Profiling [candidate] (87.614 ms) : 0, 87614
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.25.0-SNAPSHOT~0648da357d, baseline=1.25.0-SNAPSHOT~cc4d3ab92e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1043245
Total [baseline] (8.703 s) : 0, 8703471
Agent [candidate] (1.045 s) : 0, 1044517
Total [candidate] (8.706 s) : 0, 8706068
section iast
Agent [baseline] (1.161 s) : 0, 1161477
Total [baseline] (9.215 s) : 0, 9215439
Agent [candidate] (1.162 s) : 0, 1162386
Total [candidate] (9.24 s) : 0, 9239546
section iast_TELEMETRY_OFF
Agent [baseline] (1.159 s) : 0, 1159310
Total [baseline] (9.283 s) : 0, 9283085
Agent [candidate] (1.154 s) : 0, 1153725
Total [candidate] (9.233 s) : 0, 9232948
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.043 s -
Agent iast 1.161 s 118.231 ms (11.3%)
Agent iast_TELEMETRY_OFF 1.159 s 116.064 ms (11.1%)
Total tracing 8.703 s -
Total iast 9.215 s 511.969 ms (5.9%)
Total iast_TELEMETRY_OFF 9.283 s 579.615 ms (6.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.045 s -
Agent iast 1.162 s 117.869 ms (11.3%)
Agent iast_TELEMETRY_OFF 1.154 s 109.209 ms (10.5%)
Total tracing 8.706 s -
Total iast 9.24 s 533.478 ms (6.1%)
Total iast_TELEMETRY_OFF 9.233 s 526.88 ms (6.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.25.0-SNAPSHOT~0648da357d, baseline=1.25.0-SNAPSHOT~cc4d3ab92e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (645.498 ms) : 0, 645498
BytebuddyAgent [candidate] (647.319 ms) : 0, 647319
GlobalTracer [baseline] (307.034 ms) : 0, 307034
GlobalTracer [candidate] (298.875 ms) : 0, 298875
AppSec [baseline] (48.562 ms) : 0, 48562
AppSec [candidate] (55.848 ms) : 0, 55848
Remote Config [baseline] (678.021 µs) : 0, 678
Remote Config [candidate] (713.068 µs) : 0, 713
Telemetry [baseline] (7.234 ms) : 0, 7234
Telemetry [candidate] (7.349 ms) : 0, 7349
section iast
BytebuddyAgent [baseline] (767.453 ms) : 0, 767453
BytebuddyAgent [candidate] (767.037 ms) : 0, 767037
GlobalTracer [baseline] (285.383 ms) : 0, 285383
GlobalTracer [candidate] (279.507 ms) : 0, 279507
AppSec [baseline] (46.141 ms) : 0, 46141
AppSec [candidate] (57.358 ms) : 0, 57358
Remote Config [baseline] (593.925 µs) : 0, 594
Remote Config [candidate] (606.092 µs) : 0, 606
Telemetry [baseline] (7.329 ms) : 0, 7329
Telemetry [candidate] (6.58 ms) : 0, 6580
IAST [baseline] (20.194 ms) : 0, 20194
IAST [candidate] (16.962 ms) : 0, 16962
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (763.792 ms) : 0, 763792
BytebuddyAgent [candidate] (760.345 ms) : 0, 760345
GlobalTracer [baseline] (287.051 ms) : 0, 287051
GlobalTracer [candidate] (280.153 ms) : 0, 280153
AppSec [baseline] (47.98 ms) : 0, 47980
AppSec [candidate] (57.096 ms) : 0, 57096
Remote Config [baseline] (591.275 µs) : 0, 591
Remote Config [candidate] (568.982 µs) : 0, 569
Telemetry [baseline] (7.948 ms) : 0, 7948
Telemetry [candidate] (7.35 ms) : 0, 7350
IAST [baseline] (17.411 ms) : 0, 17411
IAST [candidate] (13.709 ms) : 0, 13709

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2023-12-04T19:13:54 2023-12-04T19:30:27
git_branch master vzakharov/api_data_source_security
git_commit_date 1701696880 1701715830
git_commit_sha cc4d3ab92e 0648da357d
release_version 1.25.0-SNAPSHOT~cc4d3ab92e 1.25.0-SNAPSHOT~0648da357d
start_time 2023-12-04T19:13:41 2023-12-04T19:30:14
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1701718480 1701718480
ci_job_id 380765004 380765004
ci_pipeline_id 24202335 24202335
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 14 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.25.0-SNAPSHOT~0648da357d, baseline=1.25.0-SNAPSHOT~cc4d3ab92e
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.347 ms) : 1328, 1366
.   : milestone, 1347,
appsec (1.753 ms) : 1728, 1779
.   : milestone, 1753,
iast (1.536 ms) : 1511, 1560
.   : milestone, 1536,
profiling (1.568 ms) : 1542, 1594
.   : milestone, 1568,
tracing (1.476 ms) : 1451, 1502
.   : milestone, 1476,
section candidate
no_agent (1.363 ms) : 1343, 1382
.   : milestone, 1363,
appsec (1.747 ms) : 1722, 1772
.   : milestone, 1747,
iast (1.506 ms) : 1482, 1531
.   : milestone, 1506,
profiling (1.537 ms) : 1511, 1563
.   : milestone, 1537,
tracing (1.513 ms) : 1488, 1538
.   : milestone, 1513,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.347 ms [1.328 ms, 1.366 ms] -
appsec 1.753 ms [1.728 ms, 1.779 ms] 406.787 µs (30.2%)
iast 1.536 ms [1.511 ms, 1.56 ms] 189.052 µs (14.0%)
profiling 1.568 ms [1.542 ms, 1.594 ms] 221.453 µs (16.4%)
tracing 1.476 ms [1.451 ms, 1.502 ms] 129.781 µs (9.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.363 ms [1.343 ms, 1.382 ms] -
appsec 1.747 ms [1.722 ms, 1.772 ms] 384.285 µs (28.2%)
iast 1.506 ms [1.482 ms, 1.531 ms] 143.868 µs (10.6%)
profiling 1.537 ms [1.511 ms, 1.563 ms] 174.386 µs (12.8%)
tracing 1.513 ms [1.488 ms, 1.538 ms] 150.545 µs (11.0%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.25.0-SNAPSHOT~0648da357d, baseline=1.25.0-SNAPSHOT~cc4d3ab92e
    dateFormat X
    axisFormat %s
section baseline
no_agent (364.398 µs) : 344, 385
.   : milestone, 364,
iast (468.262 µs) : 448, 489
.   : milestone, 468,
iast_FULL (528.137 µs) : 508, 548
.   : milestone, 528,
iast_INACTIVE (444.963 µs) : 424, 466
.   : milestone, 445,
iast_TELEMETRY_OFF (463.888 µs) : 443, 485
.   : milestone, 464,
tracing (446.082 µs) : 425, 467
.   : milestone, 446,
section candidate
no_agent (359.017 µs) : 339, 379
.   : milestone, 359,
iast (463.993 µs) : 444, 484
.   : milestone, 464,
iast_FULL (530.755 µs) : 510, 552
.   : milestone, 531,
iast_INACTIVE (448.043 µs) : 426, 470
.   : milestone, 448,
iast_TELEMETRY_OFF (462.213 µs) : 441, 484
.   : milestone, 462,
tracing (438.819 µs) : 418, 460
.   : milestone, 439,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.398 µs [344.155 µs, 384.641 µs] -
iast 468.262 µs [447.59 µs, 488.934 µs] 103.864 µs (28.5%)
iast_FULL 528.137 µs [507.969 µs, 548.305 µs] 163.739 µs (44.9%)
iast_INACTIVE 444.963 µs [423.804 µs, 466.122 µs] 80.565 µs (22.1%)
iast_TELEMETRY_OFF 463.888 µs [442.992 µs, 484.784 µs] 99.49 µs (27.3%)
tracing 446.082 µs [424.808 µs, 467.356 µs] 81.684 µs (22.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 359.017 µs [339.476 µs, 378.557 µs] -
iast 463.993 µs [443.733 µs, 484.253 µs] 104.976 µs (29.2%)
iast_FULL 530.755 µs [509.67 µs, 551.84 µs] 171.738 µs (47.8%)
iast_INACTIVE 448.043 µs [426.149 µs, 469.937 µs] 89.026 µs (24.8%)
iast_TELEMETRY_OFF 462.213 µs [440.882 µs, 483.543 µs] 103.196 µs (28.7%)
tracing 438.819 µs [417.934 µs, 459.703 µs] 79.802 µs (22.2%)

pr-commenter[bot] avatar Nov 28 '23 09:11 pr-commenter[bot]