What Does This Do

Motivation

Additional Notes

Contributor Checklist

[x] Format the title according the contribution guidelines
[x] Assign the type: and (comp: or inst:) labels in addition to any usefull labels
[ ] Squash your commits prior merging or merge using GitHub's Squash and merge
[x] Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
~[ ] Update the public documentation in case of new configuration flag or behavior~

Jira ticket: APMJAVA-1342

Aug 26 '24 10:08 smola

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	smola/gradle-java11
git_commit_date	1743177610	1743179438
git_commit_sha	2a848ef2e2	d67f8fd0fd
release_version	1.48.0-SNAPSHOT~2a848ef2e2	1.48.0-SNAPSHOT~d67f8fd0fd

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1743182477	1743182477
ci_job_id	870518233	870518233
ci_pipeline_id	60362083	60362083
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-kve8stnm-project-304-concurrent-2-q7b92ly4 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-kve8stnm-project-304-concurrent-2-q7b92ly4 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 67 metrics, 4 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1057132
Total [baseline] (8.703 s) : 0, 8702787
Agent [candidate] (1.062 s) : 0, 1061773
Total [candidate] (8.712 s) : 0, 8711951
section iast
Agent [baseline] (1.18 s) : 0, 1179537
Total [baseline] (9.22 s) : 0, 9220332
Agent [candidate] (1.18 s) : 0, 1179818
Total [candidate] (9.233 s) : 0, 9233216
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.189 s) : 0, 1188698
Total [baseline] (9.245 s) : 0, 9244756
Agent [candidate] (1.189 s) : 0, 1189132
Total [candidate] (9.216 s) : 0, 9216382
section iast_TELEMETRY_OFF
Agent [baseline] (1.186 s) : 0, 1185586
Total [baseline] (9.248 s) : 0, 9248241
Agent [candidate] (1.175 s) : 0, 1175197
Total [candidate] (9.246 s) : 0, 9245552

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.057 s	-
Agent	iast	1.18 s	122.406 ms (11.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.189 s	131.567 ms (12.4%)
Agent	iast_TELEMETRY_OFF	1.186 s	128.454 ms (12.2%)
Total	tracing	8.703 s	-
Total	iast	9.22 s	517.545 ms (5.9%)
Total	iast_HARDCODED_SECRET_DISABLED	9.245 s	541.97 ms (6.2%)
Total	iast_TELEMETRY_OFF	9.248 s	545.455 ms (6.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.062 s	-
Agent	iast	1.18 s	118.045 ms (11.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.189 s	127.36 ms (12.0%)
Agent	iast_TELEMETRY_OFF	1.175 s	113.424 ms (10.7%)
Total	tracing	8.712 s	-
Total	iast	9.233 s	521.265 ms (6.0%)
Total	iast_HARDCODED_SECRET_DISABLED	9.216 s	504.431 ms (5.8%)
Total	iast_TELEMETRY_OFF	9.246 s	533.601 ms (6.1%)

gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.874 ms) : 0, 720874
BytebuddyAgent [candidate] (725.347 ms) : 0, 725347
GlobalTracer [baseline] (239.806 ms) : 0, 239806
GlobalTracer [candidate] (241.049 ms) : 0, 241049
AppSec [baseline] (55.44 ms) : 0, 55440
AppSec [candidate] (56.288 ms) : 0, 56288
Debugger [baseline] (4.493 ms) : 0, 4493
Debugger [candidate] (4.526 ms) : 0, 4526
Remote Config [baseline] (682.79 µs) : 0, 683
Remote Config [candidate] (697.906 µs) : 0, 698
Telemetry [baseline] (14.413 ms) : 0, 14413
Telemetry [candidate] (13.072 ms) : 0, 13072
section iast
BytebuddyAgent [baseline] (837.658 ms) : 0, 837658
BytebuddyAgent [candidate] (837.832 ms) : 0, 837832
GlobalTracer [baseline] (228.889 ms) : 0, 228889
GlobalTracer [candidate] (229.392 ms) : 0, 229392
IAST [baseline] (22.842 ms) : 0, 22842
IAST [candidate] (22.901 ms) : 0, 22901
AppSec [baseline] (55.785 ms) : 0, 55785
AppSec [candidate] (55.552 ms) : 0, 55552
Debugger [baseline] (4.23 ms) : 0, 4230
Debugger [candidate] (4.198 ms) : 0, 4198
Remote Config [baseline] (630.614 µs) : 0, 631
Remote Config [candidate] (609.871 µs) : 0, 610
Telemetry [baseline] (8.911 ms) : 0, 8911
Telemetry [candidate] (8.816 ms) : 0, 8816
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (844.65 ms) : 0, 844650
BytebuddyAgent [candidate] (844.964 ms) : 0, 844964
GlobalTracer [baseline] (230.439 ms) : 0, 230439
GlobalTracer [candidate] (230.41 ms) : 0, 230410
IAST [baseline] (23.022 ms) : 0, 23022
IAST [candidate] (23.187 ms) : 0, 23187
AppSec [baseline] (56.092 ms) : 0, 56092
AppSec [candidate] (56.205 ms) : 0, 56205
Debugger [baseline] (4.235 ms) : 0, 4235
Debugger [candidate] (4.249 ms) : 0, 4249
Remote Config [baseline] (621.907 µs) : 0, 622
Remote Config [candidate] (614.126 µs) : 0, 614
Telemetry [baseline] (8.995 ms) : 0, 8995
Telemetry [candidate] (8.867 ms) : 0, 8867
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (841.772 ms) : 0, 841772
BytebuddyAgent [candidate] (833.981 ms) : 0, 833981
GlobalTracer [baseline] (230.569 ms) : 0, 230569
GlobalTracer [candidate] (228.894 ms) : 0, 228894
IAST [baseline] (22.608 ms) : 0, 22608
IAST [candidate] (22.273 ms) : 0, 22273
AppSec [baseline] (56.442 ms) : 0, 56442
AppSec [candidate] (55.914 ms) : 0, 55914
Debugger [baseline] (4.163 ms) : 0, 4163
Debugger [candidate] (4.188 ms) : 0, 4188
Remote Config [baseline] (621.459 µs) : 0, 621
Remote Config [candidate] (630.0 µs) : 0, 630
Telemetry [baseline] (8.802 ms) : 0, 8802
Telemetry [candidate] (8.869 ms) : 0, 8869

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1057074
Total [baseline] (10.439 s) : 0, 10439368
Agent [candidate] (1.051 s) : 0, 1050980
Total [candidate] (10.424 s) : 0, 10423749
section appsec
Agent [baseline] (1.195 s) : 0, 1194541
Total [baseline] (10.74 s) : 0, 10740155
Agent [candidate] (1.193 s) : 0, 1192851
Total [candidate] (10.761 s) : 0, 10761108
section iast
Agent [baseline] (1.185 s) : 0, 1185419
Total [baseline] (11.055 s) : 0, 11054602
Agent [candidate] (1.196 s) : 0, 1196191
Total [candidate] (11.033 s) : 0, 11033122
section profiling
Agent [baseline] (1.275 s) : 0, 1274755
Total [baseline] (10.847 s) : 0, 10846765
Agent [candidate] (1.276 s) : 0, 1275672
Total [candidate] (10.837 s) : 0, 10836891

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.057 s	-
Agent	appsec	1.195 s	137.467 ms (13.0%)
Agent	iast	1.185 s	128.345 ms (12.1%)
Agent	profiling	1.275 s	217.681 ms (20.6%)
Total	tracing	10.439 s	-
Total	appsec	10.74 s	300.787 ms (2.9%)
Total	iast	11.055 s	615.235 ms (5.9%)
Total	profiling	10.847 s	407.398 ms (3.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.051 s	-
Agent	appsec	1.193 s	141.871 ms (13.5%)
Agent	iast	1.196 s	145.211 ms (13.8%)
Agent	profiling	1.276 s	224.692 ms (21.4%)
Total	tracing	10.424 s	-
Total	appsec	10.761 s	337.358 ms (3.2%)
Total	iast	11.033 s	609.373 ms (5.8%)
Total	profiling	10.837 s	413.142 ms (4.0%)

gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.563 ms) : 0, 720563
BytebuddyAgent [candidate] (717.703 ms) : 0, 717703
GlobalTracer [baseline] (240.103 ms) : 0, 240103
GlobalTracer [candidate] (238.655 ms) : 0, 238655
AppSec [baseline] (54.674 ms) : 0, 54674
AppSec [candidate] (54.171 ms) : 0, 54171
Debugger [baseline] (4.454 ms) : 0, 4454
Debugger [candidate] (4.412 ms) : 0, 4412
Remote Config [baseline] (696.173 µs) : 0, 696
Remote Config [candidate] (698.912 µs) : 0, 699
Telemetry [baseline] (15.934 ms) : 0, 15934
Telemetry [candidate] (14.763 ms) : 0, 14763
section appsec
BytebuddyAgent [baseline] (737.74 ms) : 0, 737740
BytebuddyAgent [candidate] (737.217 ms) : 0, 737217
GlobalTracer [baseline] (235.568 ms) : 0, 235568
GlobalTracer [candidate] (235.244 ms) : 0, 235244
IAST [baseline] (21.458 ms) : 0, 21458
IAST [candidate] (21.607 ms) : 0, 21607
AppSec [baseline] (176.431 ms) : 0, 176431
AppSec [candidate] (175.968 ms) : 0, 175968
Debugger [baseline] (4.314 ms) : 0, 4314
Debugger [candidate] (4.267 ms) : 0, 4267
Remote Config [baseline] (650.895 µs) : 0, 651
Remote Config [candidate] (639.206 µs) : 0, 639
Telemetry [baseline] (8.571 ms) : 0, 8571
Telemetry [candidate] (8.197 ms) : 0, 8197
section iast
BytebuddyAgent [baseline] (841.666 ms) : 0, 841666
BytebuddyAgent [candidate] (850.51 ms) : 0, 850510
GlobalTracer [baseline] (230.539 ms) : 0, 230539
GlobalTracer [candidate] (231.45 ms) : 0, 231450
IAST [baseline] (23.013 ms) : 0, 23013
IAST [candidate] (23.227 ms) : 0, 23227
AppSec [baseline] (55.982 ms) : 0, 55982
AppSec [candidate] (56.446 ms) : 0, 56446
Debugger [baseline] (4.22 ms) : 0, 4220
Debugger [candidate] (4.216 ms) : 0, 4216
Remote Config [baseline] (613.279 µs) : 0, 613
Remote Config [candidate] (622.124 µs) : 0, 622
Telemetry [baseline] (8.849 ms) : 0, 8849
Telemetry [candidate] (8.97 ms) : 0, 8970
section profiling
BytebuddyAgent [baseline] (711.388 ms) : 0, 711388
BytebuddyAgent [candidate] (711.269 ms) : 0, 711269
GlobalTracer [baseline] (347.396 ms) : 0, 347396
GlobalTracer [candidate] (349.181 ms) : 0, 349181
AppSec [baseline] (54.009 ms) : 0, 54009
AppSec [candidate] (53.195 ms) : 0, 53195
Debugger [baseline] (4.305 ms) : 0, 4305
Debugger [candidate] (4.295 ms) : 0, 4295
Remote Config [baseline] (715.121 µs) : 0, 715
Remote Config [candidate] (696.462 µs) : 0, 696
Telemetry [baseline] (8.941 ms) : 0, 8941
Telemetry [candidate] (9.0 ms) : 0, 9000
ProfilingAgent [baseline] (102.104 ms) : 0, 102104
ProfilingAgent [candidate] (102.304 ms) : 0, 102304
Profiling [baseline] (102.129 ms) : 0, 102129
Profiling [candidate] (102.329 ms) : 0, 102329

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-03-28T16:51:34	2025-03-28T16:59:21
git_branch	master	smola/gradle-java11
git_commit_date	1743177610	1743179438
git_commit_sha	2a848ef2e2	d67f8fd0fd
release_version	1.48.0-SNAPSHOT~2a848ef2e2	1.48.0-SNAPSHOT~d67f8fd0fd
start_time	2025-03-28T16:51:20	2025-03-28T16:59:07

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1743181560	1743181560
ci_job_id	870518234	870518234
ci_pipeline_id	60362083	60362083
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-5gca-jya-project-304-concurrent-0-xpj4ivbg 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-5gca-jya-project-304-concurrent-0-xpj4ivbg 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 16 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:profiling	better [-91.656µs; -40.047µs] or [-5.792%; -2.531%]	unstable [-422.819op/s; +650.739op/s] or [-14.270%; +21.962%]	1.517ms	3076.923op/s	1.582ms	2962.963op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2
    dateFormat X
    axisFormat %s
section baseline
no_agent (384.088 µs) : 363, 405
.   : milestone, 384,
iast (522.044 µs) : 500, 544
.   : milestone, 522,
iast_FULL (730.527 µs) : 709, 753
.   : milestone, 731,
iast_GLOBAL (571.459 µs) : 549, 594
.   : milestone, 571,
iast_HARDCODED_SECRET_DISABLED (516.802 µs) : 495, 539
.   : milestone, 517,
iast_INACTIVE (469.74 µs) : 448, 491
.   : milestone, 470,
iast_TELEMETRY_OFF (503.291 µs) : 481, 525
.   : milestone, 503,
tracing (458.548 µs) : 438, 479
.   : milestone, 459,
section candidate
no_agent (390.604 µs) : 371, 410
.   : milestone, 391,
iast (517.441 µs) : 495, 539
.   : milestone, 517,
iast_FULL (731.734 µs) : 710, 754
.   : milestone, 732,
iast_GLOBAL (568.688 µs) : 546, 591
.   : milestone, 569,
iast_HARDCODED_SECRET_DISABLED (514.958 µs) : 493, 537
.   : milestone, 515,
iast_INACTIVE (469.43 µs) : 448, 491
.   : milestone, 469,
iast_TELEMETRY_OFF (507.359 µs) : 486, 529
.   : milestone, 507,
tracing (464.27 µs) : 443, 486
.   : milestone, 464,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	384.088 µs [363.428 µs, 404.747 µs]	-
iast	522.044 µs [500.035 µs, 544.052 µs]	137.956 µs (35.9%)
iast_FULL	730.527 µs [708.521 µs, 752.533 µs]	346.439 µs (90.2%)
iast_GLOBAL	571.459 µs [548.939 µs, 593.979 µs]	187.371 µs (48.8%)
iast_HARDCODED_SECRET_DISABLED	516.802 µs [494.503 µs, 539.102 µs]	132.715 µs (34.6%)
iast_INACTIVE	469.74 µs [448.008 µs, 491.472 µs]	85.652 µs (22.3%)
iast_TELEMETRY_OFF	503.291 µs [481.382 µs, 525.201 µs]	119.203 µs (31.0%)
tracing	458.548 µs [437.633 µs, 479.462 µs]	74.46 µs (19.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	390.604 µs [370.917 µs, 410.291 µs]	-
iast	517.441 µs [495.45 µs, 539.431 µs]	126.837 µs (32.5%)
iast_FULL	731.734 µs [709.921 µs, 753.548 µs]	341.13 µs (87.3%)
iast_GLOBAL	568.688 µs [546.264 µs, 591.111 µs]	178.084 µs (45.6%)
iast_HARDCODED_SECRET_DISABLED	514.958 µs [493.109 µs, 536.806 µs]	124.354 µs (31.8%)
iast_INACTIVE	469.43 µs [447.636 µs, 491.225 µs]	78.826 µs (20.2%)
iast_TELEMETRY_OFF	507.359 µs [485.705 µs, 529.012 µs]	116.755 µs (29.9%)
tracing	464.27 µs [442.709 µs, 485.832 µs]	73.666 µs (18.9%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.363 ms) : 1343, 1383
.   : milestone, 1363,
appsec (1.747 ms) : 1723, 1771
.   : milestone, 1747,
appsec_no_iast (1.722 ms) : 1698, 1746
.   : milestone, 1722,
code_origins (1.696 ms) : 1668, 1724
.   : milestone, 1696,
iast (1.511 ms) : 1486, 1535
.   : milestone, 1511,
profiling (1.582 ms) : 1557, 1607
.   : milestone, 1582,
tracing (1.499 ms) : 1475, 1524
.   : milestone, 1499,
section candidate
no_agent (1.357 ms) : 1338, 1376
.   : milestone, 1357,
appsec (1.73 ms) : 1706, 1753
.   : milestone, 1730,
appsec_no_iast (1.728 ms) : 1704, 1752
.   : milestone, 1728,
code_origins (1.699 ms) : 1672, 1726
.   : milestone, 1699,
iast (1.52 ms) : 1495, 1544
.   : milestone, 1520,
profiling (1.517 ms) : 1494, 1540
.   : milestone, 1517,
tracing (1.49 ms) : 1464, 1516
.   : milestone, 1490,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.363 ms [1.343 ms, 1.383 ms]	-
appsec	1.747 ms [1.723 ms, 1.771 ms]	384.141 µs (28.2%)
appsec_no_iast	1.722 ms [1.698 ms, 1.746 ms]	359.152 µs (26.3%)
code_origins	1.696 ms [1.668 ms, 1.724 ms]	333.429 µs (24.5%)
iast	1.511 ms [1.486 ms, 1.535 ms]	147.934 µs (10.9%)
profiling	1.582 ms [1.557 ms, 1.607 ms]	219.417 µs (16.1%)
tracing	1.499 ms [1.475 ms, 1.524 ms]	136.195 µs (10.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.357 ms [1.338 ms, 1.376 ms]	-
appsec	1.73 ms [1.706 ms, 1.753 ms]	372.71 µs (27.5%)
appsec_no_iast	1.728 ms [1.704 ms, 1.752 ms]	371.17 µs (27.4%)
code_origins	1.699 ms [1.672 ms, 1.726 ms]	342.276 µs (25.2%)
iast	1.52 ms [1.495 ms, 1.544 ms]	162.77 µs (12.0%)
profiling	1.517 ms [1.494 ms, 1.54 ms]	159.767 µs (11.8%)
tracing	1.49 ms [1.464 ms, 1.516 ms]	133.225 µs (9.8%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	smola/gradle-java11
git_commit_date	1743177610	1743179438
git_commit_sha	2a848ef2e2	d67f8fd0fd
release_version	1.48.0-SNAPSHOT~2a848ef2e2	1.48.0-SNAPSHOT~d67f8fd0fd

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1743181924	1743181924
ci_job_id	870518235	870518235
ci_pipeline_id	60362083	60362083
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-kve8stnm-project-304-concurrent-3-yjlwaord 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-kve8stnm-project-304-concurrent-3-yjlwaord 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.475 s) : 15475000, 15475000
.   : milestone, 15475000,
appsec (14.945 s) : 14945000, 14945000
.   : milestone, 14945000,
iast (18.683 s) : 18683000, 18683000
.   : milestone, 18683000,
iast_GLOBAL (18.093 s) : 18093000, 18093000
.   : milestone, 18093000,
profiling (15.503 s) : 15503000, 15503000
.   : milestone, 15503000,
tracing (14.911 s) : 14911000, 14911000
.   : milestone, 14911000,
section candidate
no_agent (15.626 s) : 15626000, 15626000
.   : milestone, 15626000,
appsec (15.091 s) : 15091000, 15091000
.   : milestone, 15091000,
iast (18.471 s) : 18471000, 18471000
.   : milestone, 18471000,
iast_GLOBAL (18.144 s) : 18144000, 18144000
.   : milestone, 18144000,
profiling (15.594 s) : 15594000, 15594000
.   : milestone, 15594000,
tracing (15.257 s) : 15257000, 15257000
.   : milestone, 15257000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.475 s [15.475 s, 15.475 s]	-
appsec	14.945 s [14.945 s, 14.945 s]	-530.0 ms (-3.4%)
iast	18.683 s [18.683 s, 18.683 s]	3.208 s (20.7%)
iast_GLOBAL	18.093 s [18.093 s, 18.093 s]	2.618 s (16.9%)
profiling	15.503 s [15.503 s, 15.503 s]	28.0 ms (0.2%)
tracing	14.911 s [14.911 s, 14.911 s]	-564.0 ms (-3.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.626 s [15.626 s, 15.626 s]	-
appsec	15.091 s [15.091 s, 15.091 s]	-535.0 ms (-3.4%)
iast	18.471 s [18.471 s, 18.471 s]	2.845 s (18.2%)
iast_GLOBAL	18.144 s [18.144 s, 18.144 s]	2.518 s (16.1%)
profiling	15.594 s [15.594 s, 15.594 s]	-32.0 ms (-0.2%)
tracing	15.257 s [15.257 s, 15.257 s]	-369.0 ms (-2.4%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.479 ms) : 1468, 1491
.   : milestone, 1479,
appsec (2.379 ms) : 2334, 2424
.   : milestone, 2379,
iast (2.142 ms) : 2085, 2199
.   : milestone, 2142,
iast_GLOBAL (2.193 ms) : 2135, 2251
.   : milestone, 2193,
profiling (1.997 ms) : 1952, 2043
.   : milestone, 1997,
tracing (1.966 ms) : 1922, 2009
.   : milestone, 1966,
section candidate
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (2.369 ms) : 2324, 2414
.   : milestone, 2369,
iast (2.15 ms) : 2093, 2207
.   : milestone, 2150,
iast_GLOBAL (2.196 ms) : 2138, 2253
.   : milestone, 2196,
profiling (2.009 ms) : 1963, 2056
.   : milestone, 2009,
tracing (1.983 ms) : 1939, 2027
.   : milestone, 1983,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.479 ms [1.468 ms, 1.491 ms]	-
appsec	2.379 ms [2.334 ms, 2.424 ms]	899.746 µs (60.8%)
iast	2.142 ms [2.085 ms, 2.199 ms]	662.753 µs (44.8%)
iast_GLOBAL	2.193 ms [2.135 ms, 2.251 ms]	713.827 µs (48.3%)
profiling	1.997 ms [1.952 ms, 2.043 ms]	518.162 µs (35.0%)
tracing	1.966 ms [1.922 ms, 2.009 ms]	486.832 µs (32.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.475 ms [1.464 ms, 1.487 ms]	-
appsec	2.369 ms [2.324 ms, 2.414 ms]	894.099 µs (60.6%)
iast	2.15 ms [2.093 ms, 2.207 ms]	674.611 µs (45.7%)
iast_GLOBAL	2.196 ms [2.138 ms, 2.253 ms]	720.705 µs (48.9%)
profiling	2.009 ms [1.963 ms, 2.056 ms]	534.186 µs (36.2%)
tracing	1.983 ms [1.939 ms, 2.027 ms]	507.722 µs (34.4%)

Aug 30 '24 21:08 pr-commenter[bot]

This pull request has been marked as stale because it has not had activity over the past quarter. It will be closed in 7 days if no further activity occurs. Feel free to reopen the PR if you are still working on it.

Jan 01 '25 03:01 github-actions[bot]

Debugger benchmarks

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
ci_job_date	1737357243	1737357619
end_time	2025-01-20T07:15:19	2025-01-20T07:21:36
git_branch	master	smola/gradle-java11
git_commit_sha	0b239ae5cc	352736e805689800823bb36719081242a4b6f3a3
start_time	2025-01-20T07:14:04	2025-01-20T07:20:20

See matching parameters

	Baseline	Candidate
ci_job_id	772223798	772223798
ci_pipeline_id	53445931	53445931
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date	1737222884	1737222884

Summary

Found 0 performance improvements and 6 performance regressions! Performance is the same for 3 metrics, 6 unstable metrics.

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:basic	worse [+3.576µs; +11.815µs] or [+1.324%; +4.374%]	same	same	unstable [-23.734µs; +39.109µs] or [-4.673%; +7.700%]	unstable [-239.733op/s; +111.528op/s] or [-9.350%; +4.350%]
scenario:loop	worse [+442.270µs; +522.596µs] or [+4.356%; +5.147%]	worse [+449.109µs; +550.782µs] or [+4.353%; +5.338%]	worse [+410.327µs; +544.125µs] or [+3.935%; +5.218%]	worse [+195.678µs; +703.388µs] or [+1.812%; +6.512%]	worse [-5.491op/s; -3.248op/s] or [-5.738%; -3.394%]

See unchanged results

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:noprobe	unstable [-26.397µs; +21.584µs] or [-9.744%; +7.968%]	unstable [-39.342µs; +33.189µs] or [-12.663%; +10.682%]	unstable [-52.143µs; +42.935µs] or [-16.038%; +13.206%]	unstable [-94.545µs; +97.240µs] or [-16.186%; +16.647%]	same

Request duration reports for reports

gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (310.695 µs) : 270, 352
.   : milestone, 311,
basic (301.977 µs) : 293, 310
.   : milestone, 302,
loop (10.317 ms) : 10257, 10378
.   : milestone, 10317,
section candidate
noprobe (307.618 µs) : 281, 334
.   : milestone, 308,
basic (309.887 µs) : 301, 319
.   : milestone, 310,
loop (10.817 ms) : 10788, 10847
.   : milestone, 10817,

baseline results

Scenario	Request median duration [CI 0.99]
noprobe	310.695 µs [269.778 µs, 351.611 µs]
basic	301.977 µs [293.457 µs, 310.498 µs]
loop	10.317 ms [10.257 ms, 10.378 ms]

candidate results

Scenario	Request median duration [CI 0.99]
noprobe	307.618 µs [280.843 µs, 334.393 µs]
basic	309.887 µs [301.101 µs, 318.674 µs]
loop	10.817 ms [10.788 ms, 10.847 ms]

Jan 15 '25 15:01 pr-commenter[bot]

Kafka / producer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	smola/gradle-java11
git_commit_date	1743177610	1743179438
git_commit_sha	2a848ef2e2386013e6176526d0171609f5a336ce	d67f8fd0fd334df6057542e3c187b4bf67572bed

See matching parameters

	Baseline	Candidate
ci_job_date	1743181110	1743181110
ci_job_id	870518237	870518237
ci_pipeline_id	60362083	60362083
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce	same

Jan 20 '25 07:01 pr-commenter[bot]

Kafka / consumer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	smola/gradle-java11
git_commit_date	1743177610	1743179438
git_commit_sha	2a848ef2e2386013e6176526d0171609f5a336ce	d67f8fd0fd334df6057542e3c187b4bf67572bed

See matching parameters

	Baseline	Candidate
ci_job_date	1743181168	1743181168
ci_job_id	870518238	870518238
ci_pipeline_id	60362083	60362083
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same

Jan 20 '25 07:01 pr-commenter[bot]

This pull request has been marked as stale because it has not had activity over the past quarter. It will be closed in 7 days if no further activity occurs. Feel free to reopen the PR if you are still working on it.

Jul 01 '25 03:07 github-actions[bot]

This pull request has been closed because it has not had activity over the past quarter. Feel free to reopen the PR if you are still working on it.

Aug 01 '25 03:08 github-actions[bot]

dd-trace-java
dd-trace-java copied to clipboard

Decouple gradle JDK from compile/test JDK

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Debugger benchmarks

Parameters

Summary

Kafka / producer-benchmark

Parameters

Summary

Kafka / consumer-benchmark

Parameters

Summary

dd-trace-java dd-trace-java copied to clipboard

Decouple gradle JDK from compile/test JDK

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Debugger benchmarks

Parameters

Summary

Kafka / producer-benchmark

Parameters

Summary

Kafka / consumer-benchmark

Parameters

Summary

dd-trace-java
dd-trace-java copied to clipboard