[WIP] Debug log received traces in smoke tests

[bantonsson]

trafficstars

[pr-commenter[bot]]

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	ban/run-with-verbose-spans-in-smoke-tests
git_commit_date	1705397650	1705398472
git_commit_sha	17330cb335	2553c52c54
release_version	1.28.0-SNAPSHOT~17330cb335	1.28.0-SNAPSHOT~2553c52c54

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1705401263	1705401263
ci_job_id	409159184	409159184
ci_pipeline_id	26507174	26507174
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 9 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.28.0-SNAPSHOT~2553c52c54, baseline=1.28.0-SNAPSHOT~17330cb335

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063062
Total [baseline] (8.748 s) : 0, 8747844
Agent [candidate] (1.062 s) : 0, 1061760
Total [candidate] (8.771 s) : 0, 8771087
section iast
Agent [baseline] (1.178 s) : 0, 1177770
Total [baseline] (9.261 s) : 0, 9261464
Agent [candidate] (1.181 s) : 0, 1181088
Total [candidate] (9.29 s) : 0, 9290350
section iast_TELEMETRY_OFF
Agent [baseline] (1.167 s) : 0, 1167255
Total [baseline] (9.269 s) : 0, 9268876
Agent [candidate] (1.167 s) : 0, 1167323
Total [candidate] (9.268 s) : 0, 9267813

• baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.063 s	-
Agent	iast	1.178 s	114.708 ms (10.8%)
Agent	iast_TELEMETRY_OFF	1.167 s	104.194 ms (9.8%)
Total	tracing	8.748 s	-
Total	iast	9.261 s	513.621 ms (5.9%)
Total	iast_TELEMETRY_OFF	9.269 s	521.032 ms (6.0%)

• candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.062 s	-
Agent	iast	1.181 s	119.328 ms (11.2%)
Agent	iast_TELEMETRY_OFF	1.167 s	105.563 ms (9.9%)
Total	tracing	8.771 s	-
Total	iast	9.29 s	519.263 ms (5.9%)
Total	iast_TELEMETRY_OFF	9.268 s	496.725 ms (5.7%)

gantt
    title insecure-bank - break down per module: candidate=1.28.0-SNAPSHOT~2553c52c54, baseline=1.28.0-SNAPSHOT~17330cb335

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (671.648 ms) : 0, 671648
BytebuddyAgent [candidate] (670.617 ms) : 0, 670617
GlobalTracer [baseline] (297.719 ms) : 0, 297719
GlobalTracer [candidate] (297.87 ms) : 0, 297870
AppSec [baseline] (51.088 ms) : 0, 51088
AppSec [candidate] (50.78 ms) : 0, 50780
Remote Config [baseline] (660.605 µs) : 0, 661
Remote Config [candidate] (664.527 µs) : 0, 665
Telemetry [baseline] (7.216 ms) : 0, 7216
Telemetry [candidate] (7.238 ms) : 0, 7238
section iast
BytebuddyAgent [baseline] (775.627 ms) : 0, 775627
BytebuddyAgent [candidate] (777.583 ms) : 0, 777583
GlobalTracer [baseline] (287.046 ms) : 0, 287046
GlobalTracer [candidate] (287.496 ms) : 0, 287496
AppSec [baseline] (53.881 ms) : 0, 53881
AppSec [candidate] (53.822 ms) : 0, 53822
Remote Config [baseline] (583.562 µs) : 0, 584
Remote Config [candidate] (585.717 µs) : 0, 586
Telemetry [baseline] (6.551 ms) : 0, 6551
Telemetry [candidate] (6.599 ms) : 0, 6599
IAST [baseline] (19.741 ms) : 0, 19741
IAST [candidate] (20.667 ms) : 0, 20667
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (766.969 ms) : 0, 766969
BytebuddyAgent [candidate] (767.643 ms) : 0, 767643
GlobalTracer [baseline] (286.058 ms) : 0, 286058
GlobalTracer [candidate] (286.125 ms) : 0, 286125
AppSec [baseline] (53.726 ms) : 0, 53726
AppSec [candidate] (53.063 ms) : 0, 53063
Remote Config [baseline] (578.631 µs) : 0, 579
Remote Config [candidate] (578.694 µs) : 0, 579
Telemetry [baseline] (6.446 ms) : 0, 6446
Telemetry [candidate] (7.153 ms) : 0, 7153
IAST [baseline] (19.182 ms) : 0, 19182
IAST [candidate] (18.656 ms) : 0, 18656

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.28.0-SNAPSHOT~2553c52c54, baseline=1.28.0-SNAPSHOT~17330cb335

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.053 s) : 0, 1053077
Total [baseline] (9.444 s) : 0, 9443781
Agent [candidate] (1.062 s) : 0, 1062197
Total [candidate] (9.381 s) : 0, 9380859
section appsec
Agent [baseline] (1.154 s) : 0, 1154108
Total [baseline] (9.516 s) : 0, 9516080
Agent [candidate] (1.159 s) : 0, 1158691
Total [candidate] (9.517 s) : 0, 9516819
section iast
Agent [baseline] (1.175 s) : 0, 1175126
Total [baseline] (9.672 s) : 0, 9672111
Agent [candidate] (1.189 s) : 0, 1189237
Total [candidate] (9.672 s) : 0, 9672131
section profiling
Agent [baseline] (1.404 s) : 0, 1404211
Total [baseline] (9.707 s) : 0, 9707493
Agent [candidate] (1.406 s) : 0, 1405733
Total [candidate] (9.714 s) : 0, 9714239

• baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.053 s	-
Agent	appsec	1.154 s	101.031 ms (9.6%)
Agent	iast	1.175 s	122.049 ms (11.6%)
Agent	profiling	1.404 s	351.134 ms (33.3%)
Total	tracing	9.444 s	-
Total	appsec	9.516 s	72.299 ms (0.8%)
Total	iast	9.672 s	228.33 ms (2.4%)
Total	profiling	9.707 s	263.712 ms (2.8%)

• candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.062 s	-
Agent	appsec	1.159 s	96.493 ms (9.1%)
Agent	iast	1.189 s	127.04 ms (12.0%)
Agent	profiling	1.406 s	343.536 ms (32.3%)
Total	tracing	9.381 s	-
Total	appsec	9.517 s	135.96 ms (1.4%)
Total	iast	9.672 s	291.272 ms (3.1%)
Total	profiling	9.714 s	333.38 ms (3.6%)

gantt
    title petclinic - break down per module: candidate=1.28.0-SNAPSHOT~2553c52c54, baseline=1.28.0-SNAPSHOT~17330cb335

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (664.991 ms) : 0, 664991
BytebuddyAgent [candidate] (670.875 ms) : 0, 670875
GlobalTracer [baseline] (295.561 ms) : 0, 295561
GlobalTracer [candidate] (298.157 ms) : 0, 298157
AppSec [baseline] (50.492 ms) : 0, 50492
AppSec [candidate] (50.772 ms) : 0, 50772
Remote Config [baseline] (655.403 µs) : 0, 655
Remote Config [candidate] (680.016 µs) : 0, 680
Telemetry [baseline] (7.189 ms) : 0, 7189
Telemetry [candidate] (7.28 ms) : 0, 7280
section appsec
BytebuddyAgent [baseline] (667.024 ms) : 0, 667024
BytebuddyAgent [candidate] (670.068 ms) : 0, 670068
GlobalTracer [baseline] (296.121 ms) : 0, 296121
GlobalTracer [candidate] (297.638 ms) : 0, 297638
AppSec [baseline] (149.058 ms) : 0, 149058
AppSec [candidate] (148.922 ms) : 0, 148922
Remote Config [baseline] (649.114 µs) : 0, 649
Remote Config [candidate] (652.422 µs) : 0, 652
Telemetry [baseline] (6.904 ms) : 0, 6904
Telemetry [candidate] (6.926 ms) : 0, 6926
section iast
BytebuddyAgent [baseline] (773.702 ms) : 0, 773702
BytebuddyAgent [candidate] (783.351 ms) : 0, 783351
GlobalTracer [baseline] (286.286 ms) : 0, 286286
GlobalTracer [candidate] (289.658 ms) : 0, 289658
AppSec [baseline] (52.361 ms) : 0, 52361
AppSec [candidate] (53.807 ms) : 0, 53807
Remote Config [baseline] (624.811 µs) : 0, 625
Remote Config [candidate] (605.658 µs) : 0, 606
Telemetry [baseline] (6.581 ms) : 0, 6581
Telemetry [candidate] (7.338 ms) : 0, 7338
IAST [baseline] (21.371 ms) : 0, 21371
IAST [candidate] (19.918 ms) : 0, 19918
section profiling
BytebuddyAgent [baseline] (663.835 ms) : 0, 663835
BytebuddyAgent [candidate] (664.559 ms) : 0, 664559
GlobalTracer [baseline] (376.711 ms) : 0, 376711
GlobalTracer [candidate] (377.244 ms) : 0, 377244
AppSec [baseline] (51.078 ms) : 0, 51078
AppSec [candidate] (51.048 ms) : 0, 51048
Remote Config [baseline] (995.11 µs) : 0, 995
Remote Config [candidate] (995.29 µs) : 0, 995
Telemetry [baseline] (7.24 ms) : 0, 7240
Telemetry [candidate] (7.326 ms) : 0, 7326
ProfilingAgent [baseline] (249.898 ms) : 0, 249898
ProfilingAgent [candidate] (250.198 ms) : 0, 250198
Profiling [baseline] (249.924 ms) : 0, 249924
Profiling [candidate] (250.223 ms) : 0, 250223

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-01-16T10:13:34	2024-01-16T10:30:07
git_branch	master	ban/run-with-verbose-spans-in-smoke-tests
git_commit_date	1705397650	1705398472
git_commit_sha	17330cb335	2553c52c54
release_version	1.28.0-SNAPSHOT~17330cb335	1.28.0-SNAPSHOT~2553c52c54
start_time	2024-01-16T10:13:21	2024-01-16T10:29:54

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1705401263	1705401263
ci_job_id	409159184	409159184
ci_pipeline_id	26507174	26507174
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 12 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.28.0-SNAPSHOT~2553c52c54, baseline=1.28.0-SNAPSHOT~17330cb335
    dateFormat X
    axisFormat %s
section baseline
no_agent (367.395 µs) : 347, 387
.   : milestone, 367,
iast (477.769 µs) : 457, 498
.   : milestone, 478,
iast_FULL (546.837 µs) : 527, 567
.   : milestone, 547,
iast_INACTIVE (455.296 µs) : 434, 476
.   : milestone, 455,
iast_TELEMETRY_OFF (483.802 µs) : 462, 505
.   : milestone, 484,
tracing (446.063 µs) : 425, 467
.   : milestone, 446,
section candidate
no_agent (374.758 µs) : 354, 396
.   : milestone, 375,
iast (480.092 µs) : 459, 501
.   : milestone, 480,
iast_FULL (540.965 µs) : 520, 562
.   : milestone, 541,
iast_INACTIVE (454.203 µs) : 434, 475
.   : milestone, 454,
iast_TELEMETRY_OFF (475.127 µs) : 454, 496
.   : milestone, 475,
tracing (444.762 µs) : 424, 465
.   : milestone, 445,

• baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	367.395 µs [347.347 µs, 387.443 µs]	-
iast	477.769 µs [457.494 µs, 498.045 µs]	110.374 µs (30.0%)
iast_FULL	546.837 µs [526.564 µs, 567.11 µs]	179.442 µs (48.8%)
iast_INACTIVE	455.296 µs [434.324 µs, 476.268 µs]	87.901 µs (23.9%)
iast_TELEMETRY_OFF	483.802 µs [462.183 µs, 505.422 µs]	116.407 µs (31.7%)
tracing	446.063 µs [425.264 µs, 466.862 µs]	78.668 µs (21.4%)

• candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	374.758 µs [353.86 µs, 395.657 µs]	-
iast	480.092 µs [459.358 µs, 500.827 µs]	105.334 µs (28.1%)
iast_FULL	540.965 µs [520.415 µs, 561.516 µs]	166.207 µs (44.4%)
iast_INACTIVE	454.203 µs [433.619 µs, 474.788 µs]	79.445 µs (21.2%)
iast_TELEMETRY_OFF	475.127 µs [454.422 µs, 495.832 µs]	100.369 µs (26.8%)
tracing	444.762 µs [424.292 µs, 465.232 µs]	70.004 µs (18.7%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.28.0-SNAPSHOT~2553c52c54, baseline=1.28.0-SNAPSHOT~17330cb335
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.365 ms) : 1346, 1384
.   : milestone, 1365,
appsec (1.76 ms) : 1734, 1786
.   : milestone, 1760,
iast (1.534 ms) : 1509, 1559
.   : milestone, 1534,
profiling (1.536 ms) : 1511, 1562
.   : milestone, 1536,
tracing (1.497 ms) : 1473, 1522
.   : milestone, 1497,
section candidate
no_agent (1.361 ms) : 1342, 1380
.   : milestone, 1361,
appsec (1.765 ms) : 1740, 1791
.   : milestone, 1765,
iast (1.534 ms) : 1509, 1558
.   : milestone, 1534,
profiling (1.587 ms) : 1560, 1614
.   : milestone, 1587,
tracing (1.499 ms) : 1474, 1525
.   : milestone, 1499,

• baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.365 ms [1.346 ms, 1.384 ms]	-
appsec	1.76 ms [1.734 ms, 1.786 ms]	394.42 µs (28.9%)
iast	1.534 ms [1.509 ms, 1.559 ms]	168.94 µs (12.4%)
profiling	1.536 ms [1.511 ms, 1.562 ms]	171.086 µs (12.5%)
tracing	1.497 ms [1.473 ms, 1.522 ms]	132.066 µs (9.7%)

• candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.361 ms [1.342 ms, 1.38 ms]	-
appsec	1.765 ms [1.74 ms, 1.791 ms]	404.683 µs (29.7%)
iast	1.534 ms [1.509 ms, 1.558 ms]	173.053 µs (12.7%)
profiling	1.587 ms [1.56 ms, 1.614 ms]	226.201 µs (16.6%)
tracing	1.499 ms [1.474 ms, 1.525 ms]	138.761 µs (10.2%)