Make sure to always upload parametric tests logs and results

Open PerfectSlayer opened this issue 1 year ago • 1 comments

What Does This Do

We need more logs to help investigate how run the parametric tests

Motivation

Parametric tests timed out most of the time

Additional Notes

Contributor Checklist

[ ] Format the title according the contribution guidelines
[ ] Assign the type: and (comp: or inst:) labels in addition to any usefull labels
[ ] Squash your commits prior merging or merge using GitHub's Squash and merge
[ ] Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
[ ] Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

Oct 10 '24 15:10 PerfectSlayer

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bbujon/system-tests
git_commit_date	1730118169	1730124132
git_commit_sha	38d810659d	341d84e882
release_version	1.42.0-SNAPSHOT~38d810659d	1.42.0-SNAPSHOT~341d84e882

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1730126733	1730126733
ci_job_id	686448316	686448316
ci_pipeline_id	47602790	47602790
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 metrics, 9 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.42.0-SNAPSHOT~341d84e882, baseline=1.42.0-SNAPSHOT~38d810659d

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.085 s) : 0, 1085271
Total [baseline] (8.601 s) : 0, 8601066
Agent [candidate] (1.083 s) : 0, 1083002
Total [candidate] (8.607 s) : 0, 8606623
section iast
Agent [baseline] (1.207 s) : 0, 1207326
Total [baseline] (9.138 s) : 0, 9137685
Agent [candidate] (1.206 s) : 0, 1205792
Total [candidate] (9.109 s) : 0, 9109163
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.206 s) : 0, 1205580
Total [baseline] (9.113 s) : 0, 9113088
Agent [candidate] (1.205 s) : 0, 1205170
Total [candidate] (9.093 s) : 0, 9093410
section iast_TELEMETRY_OFF
Agent [baseline] (1.212 s) : 0, 1212334
Total [baseline] (9.174 s) : 0, 9173821
Agent [candidate] (1.205 s) : 0, 1205095
Total [candidate] (9.113 s) : 0, 9112616

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.085 s	-
Agent	iast	1.207 s	122.055 ms (11.2%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.206 s	120.31 ms (11.1%)
Agent	iast_TELEMETRY_OFF	1.212 s	127.064 ms (11.7%)
Total	tracing	8.601 s	-
Total	iast	9.138 s	536.619 ms (6.2%)
Total	iast_HARDCODED_SECRET_DISABLED	9.113 s	512.022 ms (6.0%)
Total	iast_TELEMETRY_OFF	9.174 s	572.755 ms (6.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.083 s	-
Agent	iast	1.206 s	122.79 ms (11.3%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.205 s	122.168 ms (11.3%)
Agent	iast_TELEMETRY_OFF	1.205 s	122.092 ms (11.3%)
Total	tracing	8.607 s	-
Total	iast	9.109 s	502.541 ms (5.8%)
Total	iast_HARDCODED_SECRET_DISABLED	9.093 s	486.787 ms (5.7%)
Total	iast_TELEMETRY_OFF	9.113 s	505.994 ms (5.9%)

gantt
    title insecure-bank - break down per module: candidate=1.42.0-SNAPSHOT~341d84e882, baseline=1.42.0-SNAPSHOT~38d810659d

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (689.231 ms) : 0, 689231
BytebuddyAgent [candidate] (687.341 ms) : 0, 687341
GlobalTracer [baseline] (316.549 ms) : 0, 316549
GlobalTracer [candidate] (315.433 ms) : 0, 315433
AppSec [baseline] (55.095 ms) : 0, 55095
AppSec [candidate] (55.289 ms) : 0, 55289
Remote Config [baseline] (677.397 µs) : 0, 677
Remote Config [candidate] (665.053 µs) : 0, 665
Telemetry [baseline] (9.948 ms) : 0, 9948
Telemetry [candidate] (10.504 ms) : 0, 10504
section iast
BytebuddyAgent [baseline] (803.311 ms) : 0, 803311
BytebuddyAgent [candidate] (801.873 ms) : 0, 801873
GlobalTracer [baseline] (304.069 ms) : 0, 304069
GlobalTracer [candidate] (303.738 ms) : 0, 303738
AppSec [baseline] (58.226 ms) : 0, 58226
AppSec [candidate] (56.769 ms) : 0, 56769
IAST [baseline] (19.903 ms) : 0, 19903
IAST [candidate] (21.675 ms) : 0, 21675
Remote Config [baseline] (606.562 µs) : 0, 607
Remote Config [candidate] (594.105 µs) : 0, 594
Telemetry [baseline] (7.42 ms) : 0, 7420
Telemetry [candidate] (7.401 ms) : 0, 7401
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (801.837 ms) : 0, 801837
BytebuddyAgent [candidate] (801.162 ms) : 0, 801162
GlobalTracer [baseline] (303.543 ms) : 0, 303543
GlobalTracer [candidate] (303.71 ms) : 0, 303710
AppSec [baseline] (57.512 ms) : 0, 57512
AppSec [candidate] (58.453 ms) : 0, 58453
IAST [baseline] (20.021 ms) : 0, 20021
IAST [candidate] (19.954 ms) : 0, 19954
Remote Config [baseline] (614.008 µs) : 0, 614
Remote Config [candidate] (625.473 µs) : 0, 625
Telemetry [baseline] (8.294 ms) : 0, 8294
Telemetry [candidate] (7.515 ms) : 0, 7515
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (806.081 ms) : 0, 806081
BytebuddyAgent [candidate] (800.477 ms) : 0, 800477
GlobalTracer [baseline] (305.699 ms) : 0, 305699
GlobalTracer [candidate] (304.336 ms) : 0, 304336
AppSec [baseline] (58.738 ms) : 0, 58738
AppSec [candidate] (57.184 ms) : 0, 57184
IAST [baseline] (19.922 ms) : 0, 19922
IAST [candidate] (21.336 ms) : 0, 21336
Remote Config [baseline] (615.72 µs) : 0, 616
Remote Config [candidate] (605.514 µs) : 0, 606
Telemetry [baseline] (7.429 ms) : 0, 7429
Telemetry [candidate] (7.387 ms) : 0, 7387

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.42.0-SNAPSHOT~341d84e882, baseline=1.42.0-SNAPSHOT~38d810659d

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.083 s) : 0, 1083423
Total [baseline] (10.389 s) : 0, 10388729
Agent [candidate] (1.083 s) : 0, 1082988
Total [candidate] (10.393 s) : 0, 10392600
section appsec
Agent [baseline] (1.215 s) : 0, 1214591
Total [baseline] (10.672 s) : 0, 10671780
Agent [candidate] (1.215 s) : 0, 1214741
Total [candidate] (10.685 s) : 0, 10684566
section iast
Agent [baseline] (1.206 s) : 0, 1205910
Total [baseline] (10.884 s) : 0, 10883692
Agent [candidate] (1.224 s) : 0, 1223549
Total [candidate] (10.901 s) : 0, 10900863
section profiling
Agent [baseline] (1.28 s) : 0, 1279871
Total [baseline] (10.712 s) : 0, 10711764
Agent [candidate] (1.28 s) : 0, 1280482
Total [candidate] (10.806 s) : 0, 10805679

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.083 s	-
Agent	appsec	1.215 s	131.167 ms (12.1%)
Agent	iast	1.206 s	122.487 ms (11.3%)
Agent	profiling	1.28 s	196.448 ms (18.1%)
Total	tracing	10.389 s	-
Total	appsec	10.672 s	283.051 ms (2.7%)
Total	iast	10.884 s	494.963 ms (4.8%)
Total	profiling	10.712 s	323.035 ms (3.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.083 s	-
Agent	appsec	1.215 s	131.754 ms (12.2%)
Agent	iast	1.224 s	140.561 ms (13.0%)
Agent	profiling	1.28 s	197.495 ms (18.2%)
Total	tracing	10.393 s	-
Total	appsec	10.685 s	291.966 ms (2.8%)
Total	iast	10.901 s	508.263 ms (4.9%)
Total	profiling	10.806 s	413.079 ms (4.0%)

gantt
    title petclinic - break down per module: candidate=1.42.0-SNAPSHOT~341d84e882, baseline=1.42.0-SNAPSHOT~38d810659d

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (688.405 ms) : 0, 688405
BytebuddyAgent [candidate] (686.645 ms) : 0, 686645
GlobalTracer [baseline] (316.202 ms) : 0, 316202
GlobalTracer [candidate] (315.612 ms) : 0, 315612
AppSec [baseline] (55.266 ms) : 0, 55266
AppSec [candidate] (55.055 ms) : 0, 55055
Remote Config [baseline] (666.301 µs) : 0, 666
Remote Config [candidate] (660.356 µs) : 0, 660
Telemetry [baseline] (9.143 ms) : 0, 9143
Telemetry [candidate] (11.327 ms) : 0, 11327
section appsec
BytebuddyAgent [baseline] (703.441 ms) : 0, 703441
BytebuddyAgent [candidate] (703.706 ms) : 0, 703706
GlobalTracer [baseline] (311.867 ms) : 0, 311867
GlobalTracer [candidate] (312.389 ms) : 0, 312389
AppSec [baseline] (166.849 ms) : 0, 166849
AppSec [candidate] (166.915 ms) : 0, 166915
IAST [baseline] (19.304 ms) : 0, 19304
IAST [candidate] (19.255 ms) : 0, 19255
Remote Config [baseline] (638.018 µs) : 0, 638
Remote Config [candidate] (636.371 µs) : 0, 636
Telemetry [baseline] (8.433 ms) : 0, 8433
Telemetry [candidate] (7.727 ms) : 0, 7727
section iast
BytebuddyAgent [baseline] (802.566 ms) : 0, 802566
BytebuddyAgent [candidate] (815.494 ms) : 0, 815494
GlobalTracer [baseline] (303.612 ms) : 0, 303612
GlobalTracer [candidate] (307.214 ms) : 0, 307214
AppSec [baseline] (58.062 ms) : 0, 58062
AppSec [candidate] (58.504 ms) : 0, 58504
IAST [baseline] (19.917 ms) : 0, 19917
IAST [candidate] (20.281 ms) : 0, 20281
Remote Config [baseline] (601.374 µs) : 0, 601
Remote Config [candidate] (614.153 µs) : 0, 614
Telemetry [baseline] (7.393 ms) : 0, 7393
Telemetry [candidate] (7.482 ms) : 0, 7482
section profiling
BytebuddyAgent [baseline] (681.735 ms) : 0, 681735
BytebuddyAgent [candidate] (681.996 ms) : 0, 681996
GlobalTracer [baseline] (398.87 ms) : 0, 398870
GlobalTracer [candidate] (398.659 ms) : 0, 398659
AppSec [baseline] (55.352 ms) : 0, 55352
AppSec [candidate] (55.537 ms) : 0, 55537
Remote Config [baseline] (663.053 µs) : 0, 663
Remote Config [candidate] (664.688 µs) : 0, 665
Telemetry [baseline] (13.987 ms) : 0, 13987
Telemetry [candidate] (13.613 ms) : 0, 13613
ProfilingAgent [baseline] (90.35 ms) : 0, 90350
ProfilingAgent [candidate] (91.044 ms) : 0, 91044
Profiling [baseline] (90.374 ms) : 0, 90374
Profiling [candidate] (91.067 ms) : 0, 91067

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-10-28T14:15:49	2024-10-28T14:22:44
git_branch	master	bbujon/system-tests
git_commit_date	1730118169	1730124132
git_commit_sha	38d810659d	341d84e882
release_version	1.42.0-SNAPSHOT~38d810659d	1.42.0-SNAPSHOT~341d84e882
start_time	2024-10-28T14:15:36	2024-10-28T14:22:31

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1730125713	1730125713
ci_job_id	686448319	686448319
ci_pipeline_id	47602790	47602790
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.42.0-SNAPSHOT~341d84e882, baseline=1.42.0-SNAPSHOT~38d810659d
    dateFormat X
    axisFormat %s
section baseline
no_agent (371.757 µs) : 351, 393
.   : milestone, 372,
iast (485.061 µs) : 464, 506
.   : milestone, 485,
iast_FULL (646.597 µs) : 625, 668
.   : milestone, 647,
iast_GLOBAL (509.966 µs) : 489, 531
.   : milestone, 510,
iast_HARDCODED_SECRET_DISABLED (487.728 µs) : 467, 509
.   : milestone, 488,
iast_INACTIVE (445.301 µs) : 425, 466
.   : milestone, 445,
iast_TELEMETRY_OFF (475.277 µs) : 454, 497
.   : milestone, 475,
tracing (439.184 µs) : 419, 459
.   : milestone, 439,
section candidate
no_agent (367.452 µs) : 348, 387
.   : milestone, 367,
iast (482.053 µs) : 461, 503
.   : milestone, 482,
iast_FULL (643.87 µs) : 623, 665
.   : milestone, 644,
iast_GLOBAL (518.781 µs) : 496, 541
.   : milestone, 519,
iast_HARDCODED_SECRET_DISABLED (481.576 µs) : 461, 502
.   : milestone, 482,
iast_INACTIVE (447.839 µs) : 427, 469
.   : milestone, 448,
iast_TELEMETRY_OFF (477.777 µs) : 456, 499
.   : milestone, 478,
tracing (439.195 µs) : 419, 459
.   : milestone, 439,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	371.757 µs [350.531 µs, 392.983 µs]	-
iast	485.061 µs [463.891 µs, 506.231 µs]	113.304 µs (30.5%)
iast_FULL	646.597 µs [625.374 µs, 667.821 µs]	274.84 µs (73.9%)
iast_GLOBAL	509.966 µs [488.731 µs, 531.201 µs]	138.209 µs (37.2%)
iast_HARDCODED_SECRET_DISABLED	487.728 µs [466.517 µs, 508.938 µs]	115.971 µs (31.2%)
iast_INACTIVE	445.301 µs [424.603 µs, 465.999 µs]	73.544 µs (19.8%)
iast_TELEMETRY_OFF	475.277 µs [453.876 µs, 496.677 µs]	103.52 µs (27.8%)
tracing	439.184 µs [418.998 µs, 459.37 µs]	67.427 µs (18.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	367.452 µs [347.989 µs, 386.916 µs]	-
iast	482.053 µs [460.939 µs, 503.168 µs]	114.601 µs (31.2%)
iast_FULL	643.87 µs [622.601 µs, 665.139 µs]	276.418 µs (75.2%)
iast_GLOBAL	518.781 µs [496.442 µs, 541.121 µs]	151.329 µs (41.2%)
iast_HARDCODED_SECRET_DISABLED	481.576 µs [460.664 µs, 502.488 µs]	114.123 µs (31.1%)
iast_INACTIVE	447.839 µs [426.936 µs, 468.742 µs]	80.387 µs (21.9%)
iast_TELEMETRY_OFF	477.777 µs [456.38 µs, 499.173 µs]	110.324 µs (30.0%)
tracing	439.195 µs [418.972 µs, 459.417 µs]	71.743 µs (19.5%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.42.0-SNAPSHOT~341d84e882, baseline=1.42.0-SNAPSHOT~38d810659d
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.341 ms) : 1322, 1360
.   : milestone, 1341,
appsec (1.726 ms) : 1703, 1749
.   : milestone, 1726,
appsec_no_iast (1.731 ms) : 1708, 1755
.   : milestone, 1731,
iast (1.493 ms) : 1470, 1515
.   : milestone, 1493,
profiling (1.468 ms) : 1445, 1492
.   : milestone, 1468,
tracing (1.477 ms) : 1453, 1502
.   : milestone, 1477,
section candidate
no_agent (1.343 ms) : 1323, 1362
.   : milestone, 1343,
appsec (1.74 ms) : 1717, 1764
.   : milestone, 1740,
appsec_no_iast (1.718 ms) : 1693, 1742
.   : milestone, 1718,
iast (1.486 ms) : 1464, 1509
.   : milestone, 1486,
profiling (1.487 ms) : 1463, 1510
.   : milestone, 1487,
tracing (1.477 ms) : 1454, 1501
.   : milestone, 1477,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.341 ms [1.322 ms, 1.36 ms]	-
appsec	1.726 ms [1.703 ms, 1.749 ms]	385.375 µs (28.7%)
appsec_no_iast	1.731 ms [1.708 ms, 1.755 ms]	390.344 µs (29.1%)
iast	1.493 ms [1.47 ms, 1.515 ms]	151.65 µs (11.3%)
profiling	1.468 ms [1.445 ms, 1.492 ms]	127.568 µs (9.5%)
tracing	1.477 ms [1.453 ms, 1.502 ms]	136.558 µs (10.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.343 ms [1.323 ms, 1.362 ms]	-
appsec	1.74 ms [1.717 ms, 1.764 ms]	397.728 µs (29.6%)
appsec_no_iast	1.718 ms [1.693 ms, 1.742 ms]	374.807 µs (27.9%)
iast	1.486 ms [1.464 ms, 1.509 ms]	143.424 µs (10.7%)
profiling	1.487 ms [1.463 ms, 1.51 ms]	144.076 µs (10.7%)
tracing	1.477 ms [1.454 ms, 1.501 ms]	134.455 µs (10.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bbujon/system-tests
git_commit_date	1730118169	1730124132
git_commit_sha	38d810659d	341d84e882
release_version	1.42.0-SNAPSHOT~38d810659d	1.42.0-SNAPSHOT~341d84e882

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1730126260	1730126260
ci_job_id	686448323	686448323
ci_pipeline_id	47602790	47602790
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.42.0-SNAPSHOT~341d84e882, baseline=1.42.0-SNAPSHOT~38d810659d
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.117 s) : 15117000, 15117000
.   : milestone, 15117000,
appsec (15.038 s) : 15038000, 15038000
.   : milestone, 15038000,
iast (18.925 s) : 18925000, 18925000
.   : milestone, 18925000,
iast_GLOBAL (17.863 s) : 17863000, 17863000
.   : milestone, 17863000,
profiling (15.073 s) : 15073000, 15073000
.   : milestone, 15073000,
tracing (15.014 s) : 15014000, 15014000
.   : milestone, 15014000,
section candidate
no_agent (15.556 s) : 15556000, 15556000
.   : milestone, 15556000,
appsec (15.139 s) : 15139000, 15139000
.   : milestone, 15139000,
iast (18.604 s) : 18604000, 18604000
.   : milestone, 18604000,
iast_GLOBAL (18.132 s) : 18132000, 18132000
.   : milestone, 18132000,
profiling (15.008 s) : 15008000, 15008000
.   : milestone, 15008000,
tracing (15.345 s) : 15345000, 15345000
.   : milestone, 15345000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.117 s [15.117 s, 15.117 s]	-
appsec	15.038 s [15.038 s, 15.038 s]	-79.0 ms (-0.5%)
iast	18.925 s [18.925 s, 18.925 s]	3.808 s (25.2%)
iast_GLOBAL	17.863 s [17.863 s, 17.863 s]	2.746 s (18.2%)
profiling	15.073 s [15.073 s, 15.073 s]	-44.0 ms (-0.3%)
tracing	15.014 s [15.014 s, 15.014 s]	-103.0 ms (-0.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.556 s [15.556 s, 15.556 s]	-
appsec	15.139 s [15.139 s, 15.139 s]	-417.0 ms (-2.7%)
iast	18.604 s [18.604 s, 18.604 s]	3.048 s (19.6%)
iast_GLOBAL	18.132 s [18.132 s, 18.132 s]	2.576 s (16.6%)
profiling	15.008 s [15.008 s, 15.008 s]	-548.0 ms (-3.5%)
tracing	15.345 s [15.345 s, 15.345 s]	-211.0 ms (-1.4%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.42.0-SNAPSHOT~341d84e882, baseline=1.42.0-SNAPSHOT~38d810659d
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1461, 1484
.   : milestone, 1473,
appsec (2.333 ms) : 2291, 2374
.   : milestone, 2333,
iast (2.086 ms) : 2033, 2138
.   : milestone, 2086,
iast_GLOBAL (2.137 ms) : 2084, 2190
.   : milestone, 2137,
profiling (1.939 ms) : 1897, 1981
.   : milestone, 1939,
tracing (1.924 ms) : 1885, 1964
.   : milestone, 1924,
section candidate
no_agent (1.478 ms) : 1466, 1489
.   : milestone, 1478,
appsec (2.362 ms) : 2321, 2404
.   : milestone, 2362,
iast (2.082 ms) : 2031, 2134
.   : milestone, 2082,
iast_GLOBAL (2.141 ms) : 2089, 2193
.   : milestone, 2141,
profiling (1.954 ms) : 1911, 1996
.   : milestone, 1954,
tracing (1.922 ms) : 1882, 1961
.   : milestone, 1922,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.473 ms [1.461 ms, 1.484 ms]	-
appsec	2.333 ms [2.291 ms, 2.374 ms]	860.068 µs (58.4%)
iast	2.086 ms [2.033 ms, 2.138 ms]	612.684 µs (41.6%)
iast_GLOBAL	2.137 ms [2.084 ms, 2.19 ms]	663.88 µs (45.1%)
profiling	1.939 ms [1.897 ms, 1.981 ms]	466.05 µs (31.6%)
tracing	1.924 ms [1.885 ms, 1.964 ms]	451.447 µs (30.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.478 ms [1.466 ms, 1.489 ms]	-
appsec	2.362 ms [2.321 ms, 2.404 ms]	884.691 µs (59.9%)
iast	2.082 ms [2.031 ms, 2.134 ms]	604.741 µs (40.9%)
iast_GLOBAL	2.141 ms [2.089 ms, 2.193 ms]	663.154 µs (44.9%)
profiling	1.954 ms [1.911 ms, 1.996 ms]	476.03 µs (32.2%)
tracing	1.922 ms [1.882 ms, 1.961 ms]	444.152 µs (30.1%)

Oct 10 '24 15:10 pr-commenter[bot]

dd-trace-java dd-trace-java copied to clipboard

Make sure to always upload parametric tests logs and results

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

dd-trace-java
dd-trace-java copied to clipboard