Move macrobenchmarks to `apm-sdks-benchmarks` repo

Open sarahchen6 opened this issue 3 months ago • 1 comments

What Does This Do

Now that our macrobenchmarks have been moved to the apm-sdks-benchmarks repo (https://github.com/DataDog/apm-sdks-benchmarks/pull/34), we can remove the local implementation.

Motivation

Additional Notes

Before totally moving macrobenchmarks to the apm-sdks-benchmarks repo, we need to ensure OE reporting is not disrupted. Currently, OE may report apm-sdks-benchmarks instead of dd-trace-java for the benchmarks.

To check this PR, see the manually triggered macrobenchmark runs in CI: https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-java/-/pipelines/86577685. Note that we'll have to confirm that they're automatically run on master and when scheduled (ref).

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: https://datadoghq.atlassian.net/browse/APMLP-715

Dec 02 '25 17:12 sarahchen6

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/move-macrobenchmarks
git_commit_date	1765910716	1765916968
git_commit_sha	5477fa0d27	c3f9431502
release_version	1.57.0-SNAPSHOT~5477fa0d27	1.57.0-SNAPSHOT~c3f9431502

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1765918880	1765918880
ci_job_id	1300579801	1300579801
ci_pipeline_id	87158656	87158656
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-w66xqn13 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-w66xqn13 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 8 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~c3f9431502, baseline=1.57.0-SNAPSHOT~5477fa0d27

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.079 s) : 0, 1078501
Total [baseline] (10.883 s) : 0, 10883410
Agent [candidate] (1.084 s) : 0, 1084243
Total [candidate] (10.875 s) : 0, 10874914
section appsec
Agent [baseline] (1.263 s) : 0, 1262647
Total [baseline] (11.019 s) : 0, 11019081
Agent [candidate] (1.259 s) : 0, 1259004
Total [candidate] (11.039 s) : 0, 11038548
section iast
Agent [baseline] (1.216 s) : 0, 1216372
Total [baseline] (11.246 s) : 0, 11245527
Agent [candidate] (1.216 s) : 0, 1215611
Total [candidate] (11.156 s) : 0, 11156125
section profiling
Agent [baseline] (1.209 s) : 0, 1209391
Total [baseline] (10.999 s) : 0, 10999457
Agent [candidate] (1.208 s) : 0, 1207918
Total [candidate] (10.995 s) : 0, 10995136

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.079 s	-
Agent	appsec	1.263 s	184.146 ms (17.1%)
Agent	iast	1.216 s	137.871 ms (12.8%)
Agent	profiling	1.209 s	130.89 ms (12.1%)
Total	tracing	10.883 s	-
Total	appsec	11.019 s	135.671 ms (1.2%)
Total	iast	11.246 s	362.118 ms (3.3%)
Total	profiling	10.999 s	116.047 ms (1.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.084 s	-
Agent	appsec	1.259 s	174.761 ms (16.1%)
Agent	iast	1.216 s	131.368 ms (12.1%)
Agent	profiling	1.208 s	123.675 ms (11.4%)
Total	tracing	10.875 s	-
Total	appsec	11.039 s	163.634 ms (1.5%)
Total	iast	11.156 s	281.211 ms (2.6%)
Total	profiling	10.995 s	120.222 ms (1.1%)

gantt
    title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~c3f9431502, baseline=1.57.0-SNAPSHOT~5477fa0d27

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.171 ms) : 0, 1171
crashtracking [candidate] (1.183 ms) : 0, 1183
BytebuddyAgent [baseline] (647.008 ms) : 0, 647008
BytebuddyAgent [candidate] (650.481 ms) : 0, 650481
GlobalTracer [baseline] (281.72 ms) : 0, 281720
GlobalTracer [candidate] (283.343 ms) : 0, 283343
AppSec [baseline] (32.146 ms) : 0, 32146
AppSec [candidate] (32.433 ms) : 0, 32433
Debugger [baseline] (67.594 ms) : 0, 67594
Debugger [candidate] (68.167 ms) : 0, 68167
Remote Config [baseline] (619.318 µs) : 0, 619
Remote Config [candidate] (595.348 µs) : 0, 595
Telemetry [baseline] (9.027 ms) : 0, 9027
Telemetry [candidate] (8.857 ms) : 0, 8857
Flare Poller [baseline] (3.794 ms) : 0, 3794
Flare Poller [candidate] (3.758 ms) : 0, 3758
section appsec
crashtracking [baseline] (1.185 ms) : 0, 1185
crashtracking [candidate] (1.172 ms) : 0, 1172
BytebuddyAgent [baseline] (688.282 ms) : 0, 688282
BytebuddyAgent [candidate] (686.144 ms) : 0, 686144
GlobalTracer [baseline] (258.613 ms) : 0, 258613
GlobalTracer [candidate] (257.585 ms) : 0, 257585
AppSec [baseline] (171.544 ms) : 0, 171544
AppSec [candidate] (171.853 ms) : 0, 171853
Debugger [baseline] (69.242 ms) : 0, 69242
Debugger [candidate] (68.605 ms) : 0, 68605
Remote Config [baseline] (698.134 µs) : 0, 698
Remote Config [candidate] (723.075 µs) : 0, 723
Telemetry [baseline] (9.125 ms) : 0, 9125
Telemetry [candidate] (9.067 ms) : 0, 9067
Flare Poller [baseline] (3.801 ms) : 0, 3801
Flare Poller [candidate] (3.918 ms) : 0, 3918
IAST [baseline] (24.634 ms) : 0, 24634
IAST [candidate] (24.469 ms) : 0, 24469
section iast
crashtracking [baseline] (1.176 ms) : 0, 1176
crashtracking [candidate] (1.171 ms) : 0, 1171
BytebuddyAgent [baseline] (786.275 ms) : 0, 786275
BytebuddyAgent [candidate] (784.771 ms) : 0, 784771
GlobalTracer [baseline] (254.67 ms) : 0, 254670
GlobalTracer [candidate] (254.7 ms) : 0, 254700
AppSec [baseline] (34.201 ms) : 0, 34201
AppSec [candidate] (34.044 ms) : 0, 34044
Debugger [baseline] (65.426 ms) : 0, 65426
Debugger [candidate] (66.289 ms) : 0, 66289
Remote Config [baseline] (593.549 µs) : 0, 594
Remote Config [candidate] (573.118 µs) : 0, 573
Telemetry [baseline] (8.418 ms) : 0, 8418
Telemetry [candidate] (8.414 ms) : 0, 8414
Flare Poller [baseline] (3.464 ms) : 0, 3464
Flare Poller [candidate] (3.491 ms) : 0, 3491
IAST [baseline] (26.881 ms) : 0, 26881
IAST [candidate] (26.997 ms) : 0, 26997
section profiling
ProfilingAgent [baseline] (97.163 ms) : 0, 97163
ProfilingAgent [candidate] (97.509 ms) : 0, 97509
crashtracking [baseline] (1.219 ms) : 0, 1219
crashtracking [candidate] (1.243 ms) : 0, 1243
BytebuddyAgent [baseline] (704.487 ms) : 0, 704487
BytebuddyAgent [candidate] (702.209 ms) : 0, 702209
GlobalTracer [baseline] (221.751 ms) : 0, 221751
GlobalTracer [candidate] (222.024 ms) : 0, 222024
AppSec [baseline] (32.37 ms) : 0, 32370
AppSec [candidate] (32.568 ms) : 0, 32568
Debugger [baseline] (69.116 ms) : 0, 69116
Debugger [candidate] (68.907 ms) : 0, 68907
Remote Config [baseline] (653.579 µs) : 0, 654
Remote Config [candidate] (644.156 µs) : 0, 644
Telemetry [baseline] (8.905 ms) : 0, 8905
Telemetry [candidate] (8.869 ms) : 0, 8869
Flare Poller [baseline] (3.785 ms) : 0, 3785
Flare Poller [candidate] (3.779 ms) : 0, 3779
Profiling [baseline] (97.747 ms) : 0, 97747
Profiling [candidate] (98.092 ms) : 0, 98092

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~c3f9431502, baseline=1.57.0-SNAPSHOT~5477fa0d27

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.082 s) : 0, 1081586
Total [baseline] (8.768 s) : 0, 8767730
Agent [candidate] (1.081 s) : 0, 1081072
Total [candidate] (8.744 s) : 0, 8744486
section iast
Agent [baseline] (1.219 s) : 0, 1219159
Total [baseline] (9.344 s) : 0, 9343803
Agent [candidate] (1.219 s) : 0, 1219033
Total [candidate] (9.322 s) : 0, 9322235

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.082 s	-
Agent	iast	1.219 s	137.573 ms (12.7%)
Total	tracing	8.768 s	-
Total	iast	9.344 s	576.073 ms (6.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.081 s	-
Agent	iast	1.219 s	137.961 ms (12.8%)
Total	tracing	8.744 s	-
Total	iast	9.322 s	577.75 ms (6.6%)

gantt
    title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~c3f9431502, baseline=1.57.0-SNAPSHOT~5477fa0d27

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.185 ms) : 0, 1185
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (649.541 ms) : 0, 649541
BytebuddyAgent [candidate] (649.345 ms) : 0, 649345
GlobalTracer [baseline] (282.31 ms) : 0, 282310
GlobalTracer [candidate] (282.631 ms) : 0, 282631
AppSec [baseline] (32.405 ms) : 0, 32405
AppSec [candidate] (32.178 ms) : 0, 32178
Debugger [baseline] (66.443 ms) : 0, 66443
Debugger [candidate] (66.677 ms) : 0, 66677
Remote Config [baseline] (617.574 µs) : 0, 618
Remote Config [candidate] (650.298 µs) : 0, 650
Telemetry [baseline] (9.077 ms) : 0, 9077
Telemetry [candidate] (9.121 ms) : 0, 9121
Flare Poller [baseline] (4.595 ms) : 0, 4595
Flare Poller [candidate] (3.876 ms) : 0, 3876
section iast
crashtracking [baseline] (1.203 ms) : 0, 1203
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (788.686 ms) : 0, 788686
BytebuddyAgent [candidate] (788.597 ms) : 0, 788597
GlobalTracer [baseline] (254.921 ms) : 0, 254921
GlobalTracer [candidate] (255.169 ms) : 0, 255169
AppSec [baseline] (33.4 ms) : 0, 33400
AppSec [candidate] (35.029 ms) : 0, 35029
Debugger [baseline] (66.162 ms) : 0, 66162
Debugger [candidate] (64.248 ms) : 0, 64248
Remote Config [baseline] (606.577 µs) : 0, 607
Remote Config [candidate] (582.531 µs) : 0, 583
Telemetry [baseline] (8.487 ms) : 0, 8487
Telemetry [candidate] (8.399 ms) : 0, 8399
Flare Poller [baseline] (3.487 ms) : 0, 3487
Flare Poller [candidate] (3.499 ms) : 0, 3499
IAST [baseline] (26.991 ms) : 0, 26991
IAST [candidate] (27.07 ms) : 0, 27070

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/move-macrobenchmarks
git_commit_date	1765910716	1765916968
git_commit_sha	5477fa0d27	c3f9431502
release_version	1.57.0-SNAPSHOT~5477fa0d27	1.57.0-SNAPSHOT~c3f9431502

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1765919351	1765919351
ci_job_id	1300579802	1300579802
ci_pipeline_id	87158656	87158656
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-y96cyhlz 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-y96cyhlz 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 0 performance regressions! Performance is the same for 18 metrics, 16 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:petclinic:code_origins:high_load	better [-2.049ms; -1.060ms] or [-10.947%; -5.663%]	better [-2.599ms; -0.815ms] or [-8.671%; -2.718%]	unstable [-8.370op/s; +46.682op/s] or [-3.409%; +19.015%]	17.163ms	28.271ms	264.656op/s	18.717ms	29.978ms	245.500op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~c3f9431502, baseline=1.57.0-SNAPSHOT~5477fa0d27
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.174 ms) : 1162, 1186
.   : milestone, 1174,
iast (3.176 ms) : 3133, 3220
.   : milestone, 3176,
iast_FULL (5.704 ms) : 5647, 5761
.   : milestone, 5704,
iast_GLOBAL (3.539 ms) : 3487, 3590
.   : milestone, 3539,
profiling (1.937 ms) : 1922, 1953
.   : milestone, 1937,
tracing (1.841 ms) : 1824, 1858
.   : milestone, 1841,
section candidate
no_agent (1.183 ms) : 1172, 1194
.   : milestone, 1183,
iast (3.146 ms) : 3110, 3183
.   : milestone, 3146,
iast_FULL (5.685 ms) : 5628, 5742
.   : milestone, 5685,
iast_GLOBAL (3.599 ms) : 3544, 3654
.   : milestone, 3599,
profiling (1.993 ms) : 1976, 2010
.   : milestone, 1993,
tracing (1.762 ms) : 1748, 1776
.   : milestone, 1762,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.174 ms [1.162 ms, 1.186 ms]	-
iast	3.176 ms [3.133 ms, 3.22 ms]	2.003 ms (170.6%)
iast_FULL	5.704 ms [5.647 ms, 5.761 ms]	4.53 ms (386.0%)
iast_GLOBAL	3.539 ms [3.487 ms, 3.59 ms]	2.365 ms (201.5%)
profiling	1.937 ms [1.922 ms, 1.953 ms]	763.676 µs (65.1%)
tracing	1.841 ms [1.824 ms, 1.858 ms]	667.336 µs (56.9%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.183 ms [1.172 ms, 1.194 ms]	-
iast	3.146 ms [3.11 ms, 3.183 ms]	1.963 ms (165.8%)
iast_FULL	5.685 ms [5.628 ms, 5.742 ms]	4.502 ms (380.4%)
iast_GLOBAL	3.599 ms [3.544 ms, 3.654 ms]	2.416 ms (204.1%)
profiling	1.993 ms [1.976 ms, 2.01 ms]	809.603 µs (68.4%)
tracing	1.762 ms [1.748 ms, 1.776 ms]	578.265 µs (48.9%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~c3f9431502, baseline=1.57.0-SNAPSHOT~5477fa0d27
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.906 ms) : 17722, 18090
.   : milestone, 17906,
appsec (18.562 ms) : 18373, 18751
.   : milestone, 18562,
code_origins (19.015 ms) : 18825, 19205
.   : milestone, 19015,
iast (17.966 ms) : 17787, 18146
.   : milestone, 17966,
profiling (18.434 ms) : 18247, 18621
.   : milestone, 18434,
tracing (17.691 ms) : 17514, 17868
.   : milestone, 17691,
section candidate
no_agent (18.136 ms) : 17948, 18324
.   : milestone, 18136,
appsec (18.406 ms) : 18223, 18588
.   : milestone, 18406,
code_origins (17.629 ms) : 17458, 17799
.   : milestone, 17629,
iast (17.51 ms) : 17337, 17682
.   : milestone, 17510,
profiling (18.634 ms) : 18450, 18818
.   : milestone, 18634,
tracing (17.541 ms) : 17363, 17719
.   : milestone, 17541,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	17.906 ms [17.722 ms, 18.09 ms]	-
appsec	18.562 ms [18.373 ms, 18.751 ms]	656.026 µs (3.7%)
code_origins	19.015 ms [18.825 ms, 19.205 ms]	1.109 ms (6.2%)
iast	17.966 ms [17.787 ms, 18.146 ms]	60.142 µs (0.3%)
profiling	18.434 ms [18.247 ms, 18.621 ms]	527.914 µs (2.9%)
tracing	17.691 ms [17.514 ms, 17.868 ms]	-214.664 µs (-1.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.136 ms [17.948 ms, 18.324 ms]	-
appsec	18.406 ms [18.223 ms, 18.588 ms]	269.737 µs (1.5%)
code_origins	17.629 ms [17.458 ms, 17.799 ms]	-507.348 µs (-2.8%)
iast	17.51 ms [17.337 ms, 17.682 ms]	-626.403 µs (-3.5%)
profiling	18.634 ms [18.45 ms, 18.818 ms]	498.11 µs (2.7%)
tracing	17.541 ms [17.363 ms, 17.719 ms]	-594.969 µs (-3.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/move-macrobenchmarks
git_commit_date	1765910716	1765916968
git_commit_sha	5477fa0d27	c3f9431502
release_version	1.57.0-SNAPSHOT~5477fa0d27	1.57.0-SNAPSHOT~c3f9431502

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1765918998	1765918998
ci_job_id	1300579803	1300579803
ci_pipeline_id	87158656	87158656
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-qz6b9z9r 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-qz6b9z9r 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~c3f9431502, baseline=1.57.0-SNAPSHOT~5477fa0d27
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.471 ms) : 1459, 1482
.   : milestone, 1471,
appsec (3.702 ms) : 3482, 3922
.   : milestone, 3702,
iast (2.206 ms) : 2142, 2270
.   : milestone, 2206,
iast_GLOBAL (2.248 ms) : 2182, 2313
.   : milestone, 2248,
profiling (2.067 ms) : 2014, 2119
.   : milestone, 2067,
tracing (2.029 ms) : 1979, 2080
.   : milestone, 2029,
section candidate
no_agent (1.473 ms) : 1461, 1484
.   : milestone, 1473,
appsec (3.663 ms) : 3448, 3877
.   : milestone, 3663,
iast (2.208 ms) : 2144, 2273
.   : milestone, 2208,
iast_GLOBAL (2.254 ms) : 2189, 2318
.   : milestone, 2254,
profiling (2.066 ms) : 2014, 2119
.   : milestone, 2066,
tracing (2.033 ms) : 1982, 2083
.   : milestone, 2033,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.471 ms [1.459 ms, 1.482 ms]	-
appsec	3.702 ms [3.482 ms, 3.922 ms]	2.231 ms (151.7%)
iast	2.206 ms [2.142 ms, 2.27 ms]	735.079 µs (50.0%)
iast_GLOBAL	2.248 ms [2.182 ms, 2.313 ms]	776.592 µs (52.8%)
profiling	2.067 ms [2.014 ms, 2.119 ms]	595.711 µs (40.5%)
tracing	2.029 ms [1.979 ms, 2.08 ms]	558.356 µs (38.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.473 ms [1.461 ms, 1.484 ms]	-
appsec	3.663 ms [3.448 ms, 3.877 ms]	2.19 ms (148.6%)
iast	2.208 ms [2.144 ms, 2.273 ms]	735.249 µs (49.9%)
iast_GLOBAL	2.254 ms [2.189 ms, 2.318 ms]	780.506 µs (53.0%)
profiling	2.066 ms [2.014 ms, 2.119 ms]	593.377 µs (40.3%)
tracing	2.033 ms [1.982 ms, 2.083 ms]	559.835 µs (38.0%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~c3f9431502, baseline=1.57.0-SNAPSHOT~5477fa0d27
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.371 s) : 15371000, 15371000
.   : milestone, 15371000,
appsec (14.84 s) : 14840000, 14840000
.   : milestone, 14840000,
iast (18.381 s) : 18381000, 18381000
.   : milestone, 18381000,
iast_GLOBAL (17.703 s) : 17703000, 17703000
.   : milestone, 17703000,
profiling (14.629 s) : 14629000, 14629000
.   : milestone, 14629000,
tracing (14.591 s) : 14591000, 14591000
.   : milestone, 14591000,
section candidate
no_agent (15.084 s) : 15084000, 15084000
.   : milestone, 15084000,
appsec (14.373 s) : 14373000, 14373000
.   : milestone, 14373000,
iast (18.273 s) : 18273000, 18273000
.   : milestone, 18273000,
iast_GLOBAL (17.656 s) : 17656000, 17656000
.   : milestone, 17656000,
profiling (14.626 s) : 14626000, 14626000
.   : milestone, 14626000,
tracing (14.911 s) : 14911000, 14911000
.   : milestone, 14911000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.371 s [15.371 s, 15.371 s]	-
appsec	14.84 s [14.84 s, 14.84 s]	-531.0 ms (-3.5%)
iast	18.381 s [18.381 s, 18.381 s]	3.01 s (19.6%)
iast_GLOBAL	17.703 s [17.703 s, 17.703 s]	2.332 s (15.2%)
profiling	14.629 s [14.629 s, 14.629 s]	-742.0 ms (-4.8%)
tracing	14.591 s [14.591 s, 14.591 s]	-780.0 ms (-5.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.084 s [15.084 s, 15.084 s]	-
appsec	14.373 s [14.373 s, 14.373 s]	-711.0 ms (-4.7%)
iast	18.273 s [18.273 s, 18.273 s]	3.189 s (21.1%)
iast_GLOBAL	17.656 s [17.656 s, 17.656 s]	2.572 s (17.1%)
profiling	14.626 s [14.626 s, 14.626 s]	-458.0 ms (-3.0%)
tracing	14.911 s [14.911 s, 14.911 s]	-173.0 ms (-1.1%)

Dec 02 '25 18:12 pr-commenter[bot]