Benchmarks
Startup
Parameters
|
Baseline |
Candidate |
| baseline_or_candidate |
baseline |
candidate |
| git_branch |
master |
mtoff/span-addlink |
| git_commit_date |
1725979768 |
1726005002 |
| git_commit_sha |
04db2f0bfb |
028c414378 |
| release_version |
1.40.0-SNAPSHOT~04db2f0bfb |
1.39.0-SNAPSHOT~028c414378 |
See matching parameters
|
Baseline |
Candidate |
| application |
insecure-bank |
insecure-bank |
| ci_job_date |
1726007465 |
1726007465 |
| ci_job_id |
634719968 |
634719968 |
| ci_pipeline_id |
44033883 |
44033883 |
| cpu_model |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| module |
Agent |
Agent |
| parent |
None |
None |
| variant |
iast |
iast |
Summary
Found 0 performance improvements and 1 performance regressions! Performance is the same for 46 metrics, 16 unstable metrics.
| scenario |
Δ mean execution_time |
candidate mean execution_time |
baseline mean execution_time |
| scenario:startup:petclinic:appsec:Remote Config |
worse [+17.972µs; +66.384µs] or [+2.805%; +10.362%] |
682.835µs |
640.657µs |
Startup time reports for petclinic
gantt
title petclinic - global startup overhead: candidate=1.39.0-SNAPSHOT~028c414378, baseline=1.40.0-SNAPSHOT~04db2f0bfb
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.065 s) : 0, 1065340
Total [baseline] (10.396 s) : 0, 10395677
Agent [candidate] (1.071 s) : 0, 1071215
Total [candidate] (10.477 s) : 0, 10476710
section appsec
Agent [baseline] (1.195 s) : 0, 1195155
Total [baseline] (10.632 s) : 0, 10631848
Agent [candidate] (1.193 s) : 0, 1192633
Total [candidate] (10.576 s) : 0, 10576485
section iast
Agent [baseline] (1.209 s) : 0, 1208980
Total [baseline] (10.922 s) : 0, 10922007
Agent [candidate] (1.189 s) : 0, 1189463
Total [candidate] (10.888 s) : 0, 10887558
section profiling
Agent [baseline] (1.264 s) : 0, 1263991
Total [baseline] (10.629 s) : 0, 10628779
Agent [candidate] (1.272 s) : 0, 1272000
Total [candidate] (10.716 s) : 0, 10715790
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.065 s |
- |
| Agent |
appsec |
1.195 s |
129.815 ms (12.2%) |
| Agent |
iast |
1.209 s |
143.639 ms (13.5%) |
| Agent |
profiling |
1.264 s |
198.65 ms (18.6%) |
| Total |
tracing |
10.396 s |
- |
| Total |
appsec |
10.632 s |
236.17 ms (2.3%) |
| Total |
iast |
10.922 s |
526.33 ms (5.1%) |
| Total |
profiling |
10.629 s |
233.102 ms (2.2%) |
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.071 s |
- |
| Agent |
appsec |
1.193 s |
121.418 ms (11.3%) |
| Agent |
iast |
1.189 s |
118.248 ms (11.0%) |
| Agent |
profiling |
1.272 s |
200.785 ms (18.7%) |
| Total |
tracing |
10.477 s |
- |
| Total |
appsec |
10.576 s |
99.775 ms (1.0%) |
| Total |
iast |
10.888 s |
410.847 ms (3.9%) |
| Total |
profiling |
10.716 s |
239.08 ms (2.3%) |
gantt
title petclinic - break down per module: candidate=1.39.0-SNAPSHOT~028c414378, baseline=1.40.0-SNAPSHOT~04db2f0bfb
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (680.295 ms) : 0, 680295
BytebuddyAgent [candidate] (683.482 ms) : 0, 683482
GlobalTracer [baseline] (311.12 ms) : 0, 311120
GlobalTracer [candidate] (313.217 ms) : 0, 313217
AppSec [baseline] (51.912 ms) : 0, 51912
AppSec [candidate] (52.34 ms) : 0, 52340
Remote Config [baseline] (709.223 µs) : 0, 709
Remote Config [candidate] (694.164 µs) : 0, 694
Telemetry [baseline] (7.613 ms) : 0, 7613
Telemetry [candidate] (7.656 ms) : 0, 7656
section appsec
BytebuddyAgent [baseline] (698.146 ms) : 0, 698146
BytebuddyAgent [candidate] (691.871 ms) : 0, 691871
GlobalTracer [baseline] (303.198 ms) : 0, 303198
GlobalTracer [candidate] (306.222 ms) : 0, 306222
AppSec [baseline] (160.896 ms) : 0, 160896
AppSec [candidate] (163.704 ms) : 0, 163704
IAST [baseline] (21.337 ms) : 0, 21337
IAST [candidate] (18.91 ms) : 0, 18910
Remote Config [baseline] (640.657 µs) : 0, 641
Remote Config [candidate] (682.835 µs) : 0, 683
Telemetry [baseline] (7.968 ms) : 0, 7968
Telemetry [candidate] (6.901 ms) : 0, 6901
section iast
BytebuddyAgent [baseline] (804.746 ms) : 0, 804746
BytebuddyAgent [candidate] (790.511 ms) : 0, 790511
GlobalTracer [baseline] (304.075 ms) : 0, 304075
GlobalTracer [candidate] (300.789 ms) : 0, 300789
AppSec [baseline] (53.976 ms) : 0, 53976
AppSec [candidate] (49.594 ms) : 0, 49594
IAST [baseline] (24.105 ms) : 0, 24105
IAST [candidate] (25.287 ms) : 0, 25287
Remote Config [baseline] (635.499 µs) : 0, 635
Remote Config [candidate] (611.913 µs) : 0, 612
Telemetry [baseline] (7.531 ms) : 0, 7531
Telemetry [candidate] (8.876 ms) : 0, 8876
section profiling
BytebuddyAgent [baseline] (674.1 ms) : 0, 674100
BytebuddyAgent [candidate] (677.92 ms) : 0, 677920
GlobalTracer [baseline] (395.63 ms) : 0, 395630
GlobalTracer [candidate] (396.942 ms) : 0, 396942
AppSec [baseline] (52.482 ms) : 0, 52482
AppSec [candidate] (52.896 ms) : 0, 52896
Remote Config [baseline] (712.751 µs) : 0, 713
Remote Config [candidate] (733.248 µs) : 0, 733
Telemetry [baseline] (7.453 ms) : 0, 7453
Telemetry [candidate] (7.455 ms) : 0, 7455
ProfilingAgent [baseline] (95.518 ms) : 0, 95518
ProfilingAgent [candidate] (98.033 ms) : 0, 98033
Profiling [baseline] (95.541 ms) : 0, 95541
Profiling [candidate] (98.057 ms) : 0, 98057
Startup time reports for insecure-bank
gantt
title insecure-bank - global startup overhead: candidate=1.39.0-SNAPSHOT~028c414378, baseline=1.40.0-SNAPSHOT~04db2f0bfb
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.068 s) : 0, 1067707
Total [baseline] (8.555 s) : 0, 8555420
Agent [candidate] (1.064 s) : 0, 1063641
Total [candidate] (8.563 s) : 0, 8562780
section iast
Agent [baseline] (1.19 s) : 0, 1190326
Total [baseline] (8.992 s) : 0, 8992208
Agent [candidate] (1.188 s) : 0, 1188464
Total [candidate] (9.019 s) : 0, 9018686
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.194 s) : 0, 1194024
Total [baseline] (8.974 s) : 0, 8973710
Agent [candidate] (1.202 s) : 0, 1202135
Total [candidate] (9.032 s) : 0, 9032104
section iast_TELEMETRY_OFF
Agent [baseline] (1.194 s) : 0, 1193732
Total [baseline] (8.993 s) : 0, 8993435
Agent [candidate] (1.183 s) : 0, 1183243
Total [candidate] (9.006 s) : 0, 9005775
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.068 s |
- |
| Agent |
iast |
1.19 s |
122.619 ms (11.5%) |
| Agent |
iast_HARDCODED_SECRET_DISABLED |
1.194 s |
126.317 ms (11.8%) |
| Agent |
iast_TELEMETRY_OFF |
1.194 s |
126.025 ms (11.8%) |
| Total |
tracing |
8.555 s |
- |
| Total |
iast |
8.992 s |
436.788 ms (5.1%) |
| Total |
iast_HARDCODED_SECRET_DISABLED |
8.974 s |
418.29 ms (4.9%) |
| Total |
iast_TELEMETRY_OFF |
8.993 s |
438.015 ms (5.1%) |
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.064 s |
- |
| Agent |
iast |
1.188 s |
124.823 ms (11.7%) |
| Agent |
iast_HARDCODED_SECRET_DISABLED |
1.202 s |
138.494 ms (13.0%) |
| Agent |
iast_TELEMETRY_OFF |
1.183 s |
119.602 ms (11.2%) |
| Total |
tracing |
8.563 s |
- |
| Total |
iast |
9.019 s |
455.906 ms (5.3%) |
| Total |
iast_HARDCODED_SECRET_DISABLED |
9.032 s |
469.325 ms (5.5%) |
| Total |
iast_TELEMETRY_OFF |
9.006 s |
442.996 ms (5.2%) |
gantt
title insecure-bank - break down per module: candidate=1.39.0-SNAPSHOT~028c414378, baseline=1.40.0-SNAPSHOT~04db2f0bfb
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (681.76 ms) : 0, 681760
BytebuddyAgent [candidate] (678.392 ms) : 0, 678392
GlobalTracer [baseline] (312.011 ms) : 0, 312011
GlobalTracer [candidate] (311.218 ms) : 0, 311218
AppSec [baseline] (51.92 ms) : 0, 51920
AppSec [candidate] (52.056 ms) : 0, 52056
Remote Config [baseline] (713.689 µs) : 0, 714
Remote Config [candidate] (687.367 µs) : 0, 687
Telemetry [baseline] (7.558 ms) : 0, 7558
Telemetry [candidate] (7.544 ms) : 0, 7544
section iast
BytebuddyAgent [baseline] (792.14 ms) : 0, 792140
BytebuddyAgent [candidate] (789.7 ms) : 0, 789700
GlobalTracer [baseline] (299.292 ms) : 0, 299292
GlobalTracer [candidate] (299.331 ms) : 0, 299331
AppSec [baseline] (53.223 ms) : 0, 53223
AppSec [candidate] (53.283 ms) : 0, 53283
IAST [baseline] (23.876 ms) : 0, 23876
IAST [candidate] (24.437 ms) : 0, 24437
Remote Config [baseline] (630.958 µs) : 0, 631
Remote Config [candidate] (644.981 µs) : 0, 645
Telemetry [baseline] (7.438 ms) : 0, 7438
Telemetry [candidate] (7.325 ms) : 0, 7325
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (795.334 ms) : 0, 795334
BytebuddyAgent [candidate] (797.478 ms) : 0, 797478
GlobalTracer [baseline] (299.821 ms) : 0, 299821
GlobalTracer [candidate] (304.151 ms) : 0, 304151
AppSec [baseline] (55.054 ms) : 0, 55054
AppSec [candidate] (51.633 ms) : 0, 51633
IAST [baseline] (21.98 ms) : 0, 21980
IAST [candidate] (25.555 ms) : 0, 25555
Remote Config [baseline] (596.006 µs) : 0, 596
Remote Config [candidate] (615.798 µs) : 0, 616
Telemetry [baseline] (7.409 ms) : 0, 7409
Telemetry [candidate] (8.863 ms) : 0, 8863
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (789.643 ms) : 0, 789643
BytebuddyAgent [candidate] (786.11 ms) : 0, 786110
GlobalTracer [baseline] (304.972 ms) : 0, 304972
GlobalTracer [candidate] (299.332 ms) : 0, 299332
AppSec [baseline] (53.13 ms) : 0, 53130
AppSec [candidate] (50.205 ms) : 0, 50205
IAST [baseline] (24.292 ms) : 0, 24292
IAST [candidate] (23.615 ms) : 0, 23615
Remote Config [baseline] (622.002 µs) : 0, 622
Remote Config [candidate] (617.598 µs) : 0, 618
Telemetry [baseline] (7.319 ms) : 0, 7319
Telemetry [candidate] (9.669 ms) : 0, 9669
Load
Parameters
|
Baseline |
Candidate |
| baseline_or_candidate |
baseline |
candidate |
| end_time |
2024-09-10T22:01:40 |
2024-09-10T22:08:32 |
| git_branch |
master |
mtoff/span-addlink |
| git_commit_date |
1725979768 |
1726005002 |
| git_commit_sha |
04db2f0bfb |
028c414378 |
| release_version |
1.40.0-SNAPSHOT~04db2f0bfb |
1.39.0-SNAPSHOT~028c414378 |
| start_time |
2024-09-10T22:01:26 |
2024-09-10T22:08:19 |
See matching parameters
|
Baseline |
Candidate |
| application |
insecure-bank |
insecure-bank |
| ci_job_date |
1726006462 |
1726006462 |
| ci_job_id |
634719969 |
634719969 |
| ci_pipeline_id |
44033883 |
44033883 |
| cpu_model |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| variant |
iast |
iast |
Summary
Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 15 unstable metrics.
Request duration reports for insecure-bank
gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~028c414378, baseline=1.40.0-SNAPSHOT~04db2f0bfb
dateFormat X
axisFormat %s
section baseline
no_agent (378.914 µs) : 359, 399
. : milestone, 379,
iast (498.289 µs) : 476, 521
. : milestone, 498,
iast_FULL (563.441 µs) : 542, 585
. : milestone, 563,
iast_GLOBAL (518.541 µs) : 497, 540
. : milestone, 519,
iast_HARDCODED_SECRET_DISABLED (500.73 µs) : 478, 523
. : milestone, 501,
iast_INACTIVE (454.944 µs) : 434, 476
. : milestone, 455,
iast_TELEMETRY_OFF (490.302 µs) : 468, 513
. : milestone, 490,
tracing (446.181 µs) : 426, 467
. : milestone, 446,
section candidate
no_agent (371.352 µs) : 352, 391
. : milestone, 371,
iast (492.097 µs) : 469, 515
. : milestone, 492,
iast_FULL (558.926 µs) : 538, 580
. : milestone, 559,
iast_GLOBAL (507.811 µs) : 487, 529
. : milestone, 508,
iast_HARDCODED_SECRET_DISABLED (491.272 µs) : 469, 514
. : milestone, 491,
iast_INACTIVE (456.32 µs) : 435, 478
. : milestone, 456,
iast_TELEMETRY_OFF (484.606 µs) : 462, 507
. : milestone, 485,
tracing (448.308 µs) : 428, 469
. : milestone, 448,
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
378.914 µs [358.606 µs, 399.222 µs] |
- |
| iast |
498.289 µs [475.869 µs, 520.709 µs] |
119.376 µs (31.5%) |
| iast_FULL |
563.441 µs [542.154 µs, 584.728 µs] |
184.527 µs (48.7%) |
| iast_GLOBAL |
518.541 µs [497.065 µs, 540.016 µs] |
139.627 µs (36.8%) |
| iast_HARDCODED_SECRET_DISABLED |
500.73 µs [478.434 µs, 523.026 µs] |
121.816 µs (32.1%) |
| iast_INACTIVE |
454.944 µs [433.69 µs, 476.198 µs] |
76.031 µs (20.1%) |
| iast_TELEMETRY_OFF |
490.302 µs [467.743 µs, 512.86 µs] |
111.388 µs (29.4%) |
| tracing |
446.181 µs [425.633 µs, 466.729 µs] |
67.268 µs (17.8%) |
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
371.352 µs [351.729 µs, 390.975 µs] |
- |
| iast |
492.097 µs [469.409 µs, 514.784 µs] |
120.745 µs (32.5%) |
| iast_FULL |
558.926 µs [537.757 µs, 580.094 µs] |
187.573 µs (50.5%) |
| iast_GLOBAL |
507.811 µs [486.942 µs, 528.68 µs] |
136.459 µs (36.7%) |
| iast_HARDCODED_SECRET_DISABLED |
491.272 µs [468.653 µs, 513.892 µs] |
119.92 µs (32.3%) |
| iast_INACTIVE |
456.32 µs [435.034 µs, 477.606 µs] |
84.968 µs (22.9%) |
| iast_TELEMETRY_OFF |
484.606 µs [462.249 µs, 506.963 µs] |
113.254 µs (30.5%) |
| tracing |
448.308 µs [427.694 µs, 468.923 µs] |
76.956 µs (20.7%) |
Dacapo
Parameters
|
Baseline |
Candidate |
| baseline_or_candidate |
baseline |
candidate |
| git_branch |
master |
mtoff/span-addlink |
| git_commit_date |
1725979768 |
1726005002 |
| git_commit_sha |
04db2f0bfb |
028c414378 |
| release_version |
1.40.0-SNAPSHOT~04db2f0bfb |
1.39.0-SNAPSHOT~028c414378 |
See matching parameters
|
Baseline |
Candidate |
| application |
biojava |
biojava |
| ci_job_date |
1726006996 |
1726006996 |
| ci_job_id |
634719970 |
634719970 |
| ci_pipeline_id |
44033883 |
44033883 |
| cpu_model |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| variant |
appsec |
appsec |
Summary
Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.
Execution time for biojava
gantt
title biojava - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~028c414378, baseline=1.40.0-SNAPSHOT~04db2f0bfb
dateFormat X
axisFormat %s
section baseline
no_agent (14.869 s) : 14869000, 14869000
. : milestone, 14869000,
appsec (15.125 s) : 15125000, 15125000
. : milestone, 15125000,
iast (18.976 s) : 18976000, 18976000
. : milestone, 18976000,
iast_GLOBAL (17.877 s) : 17877000, 17877000
. : milestone, 17877000,
profiling (15.746 s) : 15746000, 15746000
. : milestone, 15746000,
tracing (15.177 s) : 15177000, 15177000
. : milestone, 15177000,
section candidate
no_agent (15.775 s) : 15775000, 15775000
. : milestone, 15775000,
appsec (15.372 s) : 15372000, 15372000
. : milestone, 15372000,
iast (19.083 s) : 19083000, 19083000
. : milestone, 19083000,
iast_GLOBAL (17.801 s) : 17801000, 17801000
. : milestone, 17801000,
profiling (15.299 s) : 15299000, 15299000
. : milestone, 15299000,
tracing (15.426 s) : 15426000, 15426000
. : milestone, 15426000,
| Variant |
Execution Time [CI 0.99] |
Δ no_agent |
| no_agent |
14.869 s [14.869 s, 14.869 s] |
- |
| appsec |
15.125 s [15.125 s, 15.125 s] |
256.0 ms (1.7%) |
| iast |
18.976 s [18.976 s, 18.976 s] |
4.107 s (27.6%) |
| iast_GLOBAL |
17.877 s [17.877 s, 17.877 s] |
3.008 s (20.2%) |
| profiling |
15.746 s [15.746 s, 15.746 s] |
877.0 ms (5.9%) |
| tracing |
15.177 s [15.177 s, 15.177 s] |
308.0 ms (2.1%) |
| Variant |
Execution Time [CI 0.99] |
Δ no_agent |
| no_agent |
15.775 s [15.775 s, 15.775 s] |
- |
| appsec |
15.372 s [15.372 s, 15.372 s] |
-403.0 ms (-2.6%) |
| iast |
19.083 s [19.083 s, 19.083 s] |
3.308 s (21.0%) |
| iast_GLOBAL |
17.801 s [17.801 s, 17.801 s] |
2.026 s (12.8%) |
| profiling |
15.299 s [15.299 s, 15.299 s] |
-476.0 ms (-3.0%) |
| tracing |
15.426 s [15.426 s, 15.426 s] |
-349.0 ms (-2.2%) |
Execution time for tomcat
gantt
title tomcat - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~028c414378, baseline=1.40.0-SNAPSHOT~04db2f0bfb
dateFormat X
axisFormat %s
section baseline
no_agent (1.461 ms) : 1449, 1472
. : milestone, 1461,
appsec (2.277 ms) : 2237, 2318
. : milestone, 2277,
iast (2.035 ms) : 1986, 2083
. : milestone, 2035,
iast_GLOBAL (2.089 ms) : 2038, 2139
. : milestone, 2089,
profiling (1.919 ms) : 1879, 1959
. : milestone, 1919,
tracing (1.906 ms) : 1867, 1944
. : milestone, 1906,
section candidate
no_agent (1.469 ms) : 1457, 1480
. : milestone, 1469,
appsec (2.269 ms) : 2230, 2309
. : milestone, 2269,
iast (2.05 ms) : 2000, 2100
. : milestone, 2050,
iast_GLOBAL (2.09 ms) : 2041, 2140
. : milestone, 2090,
profiling (1.918 ms) : 1878, 1959
. : milestone, 1918,
tracing (1.898 ms) : 1860, 1936
. : milestone, 1898,
| Variant |
Execution Time [CI 0.99] |
Δ no_agent |
| no_agent |
1.461 ms [1.449 ms, 1.472 ms] |
- |
| appsec |
2.277 ms [2.237 ms, 2.318 ms] |
816.63 µs (55.9%) |
| iast |
2.035 ms [1.986 ms, 2.083 ms] |
573.751 µs (39.3%) |
| iast_GLOBAL |
2.089 ms [2.038 ms, 2.139 ms] |
627.987 µs (43.0%) |
| profiling |
1.919 ms [1.879 ms, 1.959 ms] |
458.17 µs (31.4%) |
| tracing |
1.906 ms [1.867 ms, 1.944 ms] |
444.889 µs (30.5%) |
| Variant |
Execution Time [CI 0.99] |
Δ no_agent |
| no_agent |
1.469 ms [1.457 ms, 1.48 ms] |
- |
| appsec |
2.269 ms [2.23 ms, 2.309 ms] |
800.852 µs (54.5%) |
| iast |
2.05 ms [2.0 ms, 2.1 ms] |
581.667 µs (39.6%) |
| iast_GLOBAL |
2.09 ms [2.041 ms, 2.14 ms] |
621.856 µs (42.3%) |
| profiling |
1.918 ms [1.878 ms, 1.959 ms] |
449.739 µs (30.6%) |
| tracing |
1.898 ms [1.86 ms, 1.936 ms] |
429.673 µs (29.3%) |