dd-trace-java
dd-trace-java copied to clipboard
add recipe for handling flaky tests
What Does This Do
This change adds some documentation explaining how to handle flaky tests when they occur on pull requests. It also removes a few outdated references to CircleCI.
Motivation
The motivation is to make it easier for new contributors to get up to speed on the project.
Jira ticket: LANGPLAT-352
Hi! 👋 Thanks for your pull request! 🎉
To help us review it, please make sure to:
- Remove the tag from the pull request title
If you need help, please check our contributing guidelines.
Benchmarks
Startup
Parameters
| Baseline | Candidate | |
|---|---|---|
| baseline_or_candidate | baseline | candidate |
| git_branch | master | emmett.butler/flaky-guidance |
| git_commit_date | 1750427857 | 1750447681 |
| git_commit_sha | 60c30c276e | f8ec06c202 |
| release_version | 1.51.0-SNAPSHOT~60c30c276e | 1.51.0-SNAPSHOT~f8ec06c202 |
See matching parameters
| Baseline | Candidate | |
|---|---|---|
| application | insecure-bank | insecure-bank |
| ci_job_date | 1750449582 | 1750449582 |
| ci_job_id | 992078701 | 992078701 |
| ci_pipeline_id | 68361068 | 68361068 |
| cpu_model | Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| kernel_version | Linux runner-clrnfvyp-project-304-concurrent-0-4979j6gk 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux | Linux runner-clrnfvyp-project-304-concurrent-0-4979j6gk 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| module | Agent | Agent |
| parent | None | None |
Summary
Found 0 performance improvements and 0 performance regressions! Performance is the same for 43 metrics, 10 unstable metrics.
Startup time reports for petclinic
gantt
title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.026 s) : 0, 1025897
Total [baseline] (10.504 s) : 0, 10503768
Agent [candidate] (1.025 s) : 0, 1025436
Total [candidate] (10.643 s) : 0, 10642590
section appsec
Agent [baseline] (1.188 s) : 0, 1187607
Total [baseline] (10.833 s) : 0, 10832560
Agent [candidate] (1.185 s) : 0, 1185209
Total [candidate] (10.773 s) : 0, 10772905
section iast
Agent [baseline] (1.151 s) : 0, 1151015
Total [baseline] (10.848 s) : 0, 10847997
Agent [candidate] (1.161 s) : 0, 1160964
Total [candidate] (10.864 s) : 0, 10863621
section profiling
Agent [baseline] (1.268 s) : 0, 1268441
Total [baseline] (10.952 s) : 0, 10951651
Agent [candidate] (1.273 s) : 0, 1272572
Total [candidate] (10.991 s) : 0, 10990852
- baseline results
| Module | Variant | Duration | Δ tracing |
|---|---|---|---|
| Agent | tracing | 1.026 s | - |
| Agent | appsec | 1.188 s | 161.71 ms (15.8%) |
| Agent | iast | 1.151 s | 125.118 ms (12.2%) |
| Agent | profiling | 1.268 s | 242.543 ms (23.6%) |
| Total | tracing | 10.504 s | - |
| Total | appsec | 10.833 s | 328.792 ms (3.1%) |
| Total | iast | 10.848 s | 344.229 ms (3.3%) |
| Total | profiling | 10.952 s | 447.883 ms (4.3%) |
- candidate results
| Module | Variant | Duration | Δ tracing |
|---|---|---|---|
| Agent | tracing | 1.025 s | - |
| Agent | appsec | 1.185 s | 159.773 ms (15.6%) |
| Agent | iast | 1.161 s | 135.528 ms (13.2%) |
| Agent | profiling | 1.273 s | 247.136 ms (24.1%) |
| Total | tracing | 10.643 s | - |
| Total | appsec | 10.773 s | 130.315 ms (1.2%) |
| Total | iast | 10.864 s | 221.031 ms (2.1%) |
| Total | profiling | 10.991 s | 348.263 ms (3.3%) |
gantt
title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (686.68 ms) : 0, 686680
BytebuddyAgent [candidate] (685.804 ms) : 0, 685804
GlobalTracer [baseline] (242.344 ms) : 0, 242344
GlobalTracer [candidate] (242.107 ms) : 0, 242107
AppSec [baseline] (61.238 ms) : 0, 61238
AppSec [candidate] (58.359 ms) : 0, 58359
Debugger [baseline] (6.097 ms) : 0, 6097
Debugger [candidate] (7.007 ms) : 0, 7007
Remote Config [baseline] (645.145 µs) : 0, 645
Remote Config [candidate] (669.307 µs) : 0, 669
Telemetry [baseline] (8.003 ms) : 0, 8003
Telemetry [candidate] (10.641 ms) : 0, 10641
section appsec
BytebuddyAgent [baseline] (714.341 ms) : 0, 714341
BytebuddyAgent [candidate] (713.031 ms) : 0, 713031
GlobalTracer [baseline] (236.799 ms) : 0, 236799
GlobalTracer [candidate] (236.308 ms) : 0, 236308
IAST [baseline] (22.015 ms) : 0, 22015
IAST [candidate] (22.129 ms) : 0, 22129
AppSec [baseline] (179.877 ms) : 0, 179877
AppSec [candidate] (179.283 ms) : 0, 179283
Debugger [baseline] (5.809 ms) : 0, 5809
Debugger [candidate] (5.77 ms) : 0, 5770
Remote Config [baseline] (604.759 µs) : 0, 605
Remote Config [candidate] (606.005 µs) : 0, 606
Telemetry [baseline] (7.265 ms) : 0, 7265
Telemetry [candidate] (7.216 ms) : 0, 7216
section iast
BytebuddyAgent [baseline] (804.712 ms) : 0, 804712
BytebuddyAgent [candidate] (812.268 ms) : 0, 812268
GlobalTracer [baseline] (231.61 ms) : 0, 231610
GlobalTracer [candidate] (233.392 ms) : 0, 233392
IAST [baseline] (25.285 ms) : 0, 25285
IAST [candidate] (27.029 ms) : 0, 27029
AppSec [baseline] (54.034 ms) : 0, 54034
AppSec [candidate] (52.633 ms) : 0, 52633
Debugger [baseline] (5.927 ms) : 0, 5927
Debugger [candidate] (5.999 ms) : 0, 5999
Remote Config [baseline] (601.135 µs) : 0, 601
Remote Config [candidate] (629.193 µs) : 0, 629
Telemetry [baseline] (7.972 ms) : 0, 7972
Telemetry [candidate] (8.044 ms) : 0, 8044
section profiling
BytebuddyAgent [baseline] (676.044 ms) : 0, 676044
BytebuddyAgent [candidate] (677.589 ms) : 0, 677589
GlobalTracer [baseline] (360.592 ms) : 0, 360592
GlobalTracer [candidate] (361.396 ms) : 0, 361396
AppSec [baseline] (62.258 ms) : 0, 62258
AppSec [candidate] (62.303 ms) : 0, 62303
Debugger [baseline] (6.09 ms) : 0, 6090
Debugger [candidate] (6.097 ms) : 0, 6097
Remote Config [baseline] (705.341 µs) : 0, 705
Remote Config [candidate] (706.053 µs) : 0, 706
Telemetry [baseline] (8.06 ms) : 0, 8060
Telemetry [candidate] (8.135 ms) : 0, 8135
ProfilingAgent [baseline] (106.325 ms) : 0, 106325
ProfilingAgent [candidate] (107.586 ms) : 0, 107586
Profiling [baseline] (106.35 ms) : 0, 106350
Profiling [candidate] (107.612 ms) : 0, 107612
Startup time reports for insecure-bank
gantt
title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.03 s) : 0, 1030421
Total [baseline] (8.552 s) : 0, 8551769
Agent [candidate] (1.025 s) : 0, 1025058
Total [candidate] (8.568 s) : 0, 8567647
section iast
Agent [baseline] (1.148 s) : 0, 1147624
Total [baseline] (9.204 s) : 0, 9203944
Agent [candidate] (1.152 s) : 0, 1152074
Total [candidate] (9.219 s) : 0, 9219466
- baseline results
| Module | Variant | Duration | Δ tracing |
|---|---|---|---|
| Agent | tracing | 1.03 s | - |
| Agent | iast | 1.148 s | 117.203 ms (11.4%) |
| Total | tracing | 8.552 s | - |
| Total | iast | 9.204 s | 652.176 ms (7.6%) |
- candidate results
| Module | Variant | Duration | Δ tracing |
|---|---|---|---|
| Agent | tracing | 1.025 s | - |
| Agent | iast | 1.152 s | 127.015 ms (12.4%) |
| Total | tracing | 8.568 s | - |
| Total | iast | 9.219 s | 651.819 ms (7.6%) |
gantt
title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (691.091 ms) : 0, 691091
BytebuddyAgent [candidate] (685.648 ms) : 0, 685648
GlobalTracer [baseline] (242.007 ms) : 0, 242007
GlobalTracer [candidate] (242.091 ms) : 0, 242091
AppSec [baseline] (55.663 ms) : 0, 55663
AppSec [candidate] (58.542 ms) : 0, 58542
Debugger [baseline] (7.053 ms) : 0, 7053
Debugger [candidate] (7.611 ms) : 0, 7611
Remote Config [baseline] (697.43 µs) : 0, 697
Remote Config [candidate] (657.787 µs) : 0, 658
Telemetry [baseline] (12.886 ms) : 0, 12886
Telemetry [candidate] (9.628 ms) : 0, 9628
section iast
BytebuddyAgent [baseline] (802.759 ms) : 0, 802759
BytebuddyAgent [candidate] (805.568 ms) : 0, 805568
GlobalTracer [baseline] (231.144 ms) : 0, 231144
GlobalTracer [candidate] (232.076 ms) : 0, 232076
IAST [baseline] (27.331 ms) : 0, 27331
IAST [candidate] (26.119 ms) : 0, 26119
AppSec [baseline] (51.178 ms) : 0, 51178
AppSec [candidate] (53.023 ms) : 0, 53023
Debugger [baseline] (5.835 ms) : 0, 5835
Debugger [candidate] (5.932 ms) : 0, 5932
Remote Config [baseline] (602.001 µs) : 0, 602
Remote Config [candidate] (597.377 µs) : 0, 597
Telemetry [baseline] (7.973 ms) : 0, 7973
Telemetry [candidate] (7.96 ms) : 0, 7960
Load
Parameters
| Baseline | Candidate | |
|---|---|---|
| baseline_or_candidate | baseline | candidate |
| git_branch | master | emmett.butler/flaky-guidance |
| git_commit_date | 1750427857 | 1750447681 |
| git_commit_sha | 60c30c276e | f8ec06c202 |
| release_version | 1.51.0-SNAPSHOT~60c30c276e | 1.51.0-SNAPSHOT~f8ec06c202 |
See matching parameters
| Baseline | Candidate | |
|---|---|---|
| application | insecure-bank | insecure-bank |
| ci_job_date | 1750449252 | 1750449252 |
| ci_job_id | 992078702 | 992078702 |
| ci_pipeline_id | 68361068 | 68361068 |
| cpu_model | Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| kernel_version | Linux runner-bziamzy-project-304-concurrent-0-kviyhrps 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux | Linux runner-bziamzy-project-304-concurrent-0-kviyhrps 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
Summary
Found 3 performance improvements and 1 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.
| scenario | Δ mean http_req_duration | Δ mean throughput | candidate mean http_req_duration | candidate mean throughput | baseline mean http_req_duration | baseline mean throughput |
|---|---|---|---|---|---|---|
| scenario:load:insecure-bank:iast:high_load | better [-560.820µs; -235.932µs] or [-5.950%; -2.503%] |
unstable [-48.791op/s; +91.354op/s] or [-9.906%; +18.547%] |
9.027ms | 513.844op/s | 9.425ms | 492.562op/s |
| scenario:load:insecure-bank:tracing:high_load | better [-605.283µs; -349.509µs] or [-7.449%; -4.301%] |
unstable [-46.962op/s; +116.524op/s] or [-8.232%; +20.426%] |
7.648ms | 605.250op/s | 8.125ms | 570.469op/s |
| scenario:load:petclinic:appsec:high_load | worse [+1.484ms; +2.357ms] or [+3.231%; +5.131%] |
unstable [-10.528op/s; +2.353op/s] or [-10.339%; +2.311%] |
47.869ms | 97.737op/s | 45.948ms | 101.825op/s |
| scenario:load:petclinic:iast:high_load | better [-2.365ms; -1.548ms] or [-5.258%; -3.443%] |
unstable [-2.544op/s; +11.969op/s] or [-2.446%; +11.509%] |
43.020ms | 108.713op/s | 44.977ms | 104.000op/s |
Request duration reports for petclinic
gantt
title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e
dateFormat X
axisFormat %s
section baseline
no_agent (37.529 ms) : 37223, 37835
. : milestone, 37529,
appsec (45.948 ms) : 45558, 46338
. : milestone, 45948,
code_origins (44.91 ms) : 44527, 45293
. : milestone, 44910,
iast (44.977 ms) : 44581, 45373
. : milestone, 44977,
profiling (48.746 ms) : 48309, 49182
. : milestone, 48746,
tracing (42.558 ms) : 42226, 42891
. : milestone, 42558,
section candidate
no_agent (36.583 ms) : 36293, 36873
. : milestone, 36583,
appsec (47.869 ms) : 47448, 48290
. : milestone, 47869,
code_origins (44.016 ms) : 43649, 44383
. : milestone, 44016,
iast (43.02 ms) : 42658, 43382
. : milestone, 43020,
profiling (48.052 ms) : 47584, 48521
. : milestone, 48052,
tracing (42.79 ms) : 42450, 43130
. : milestone, 42790,
- baseline results
| Variant | Request duration [CI 0.99] | Δ no_agent |
|---|---|---|
| no_agent | 37.529 ms [37.223 ms, 37.835 ms] | - |
| appsec | 45.948 ms [45.558 ms, 46.338 ms] | 8.419 ms (22.4%) |
| code_origins | 44.91 ms [44.527 ms, 45.293 ms] | 7.381 ms (19.7%) |
| iast | 44.977 ms [44.581 ms, 45.373 ms] | 7.448 ms (19.8%) |
| profiling | 48.746 ms [48.309 ms, 49.182 ms] | 11.217 ms (29.9%) |
| tracing | 42.558 ms [42.226 ms, 42.891 ms] | 5.03 ms (13.4%) |
- candidate results
| Variant | Request duration [CI 0.99] | Δ no_agent |
|---|---|---|
| no_agent | 36.583 ms [36.293 ms, 36.873 ms] | - |
| appsec | 47.869 ms [47.448 ms, 48.29 ms] | 11.286 ms (30.9%) |
| code_origins | 44.016 ms [43.649 ms, 44.383 ms] | 7.433 ms (20.3%) |
| iast | 43.02 ms [42.658 ms, 43.382 ms] | 6.437 ms (17.6%) |
| profiling | 48.052 ms [47.584 ms, 48.521 ms] | 11.469 ms (31.4%) |
| tracing | 42.79 ms [42.45 ms, 43.13 ms] | 6.207 ms (17.0%) |
Request duration reports for insecure-bank
gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e
dateFormat X
axisFormat %s
section baseline
no_agent (4.387 ms) : 4337, 4436
. : milestone, 4387,
iast (9.425 ms) : 9269, 9581
. : milestone, 9425,
iast_FULL (13.818 ms) : 13544, 14091
. : milestone, 13818,
iast_GLOBAL (9.898 ms) : 9726, 10070
. : milestone, 9898,
profiling (8.476 ms) : 8336, 8615
. : milestone, 8476,
tracing (8.125 ms) : 8000, 8251
. : milestone, 8125,
section candidate
no_agent (4.468 ms) : 4414, 4523
. : milestone, 4468,
iast (9.027 ms) : 8880, 9173
. : milestone, 9027,
iast_FULL (14.096 ms) : 13812, 14379
. : milestone, 14096,
iast_GLOBAL (10.207 ms) : 10025, 10389
. : milestone, 10207,
profiling (8.772 ms) : 8632, 8911
. : milestone, 8772,
tracing (7.648 ms) : 7536, 7759
. : milestone, 7648,
- baseline results
| Variant | Request duration [CI 0.99] | Δ no_agent |
|---|---|---|
| no_agent | 4.387 ms [4.337 ms, 4.436 ms] | - |
| iast | 9.425 ms [9.269 ms, 9.581 ms] | 5.038 ms (114.9%) |
| iast_FULL | 13.818 ms [13.544 ms, 14.091 ms] | 9.431 ms (215.0%) |
| iast_GLOBAL | 9.898 ms [9.726 ms, 10.07 ms] | 5.511 ms (125.6%) |
| profiling | 8.476 ms [8.336 ms, 8.615 ms] | 4.089 ms (93.2%) |
| tracing | 8.125 ms [8.0 ms, 8.251 ms] | 3.739 ms (85.2%) |
- candidate results
| Variant | Request duration [CI 0.99] | Δ no_agent |
|---|---|---|
| no_agent | 4.468 ms [4.414 ms, 4.523 ms] | - |
| iast | 9.027 ms [8.88 ms, 9.173 ms] | 4.558 ms (102.0%) |
| iast_FULL | 14.096 ms [13.812 ms, 14.379 ms] | 9.628 ms (215.5%) |
| iast_GLOBAL | 10.207 ms [10.025 ms, 10.389 ms] | 5.739 ms (128.4%) |
| profiling | 8.772 ms [8.632 ms, 8.911 ms] | 4.304 ms (96.3%) |
| tracing | 7.648 ms [7.536 ms, 7.759 ms] | 3.18 ms (71.2%) |
Dacapo
Parameters
| Baseline | Candidate | |
|---|---|---|
| baseline_or_candidate | baseline | candidate |
| git_branch | master | emmett.butler/flaky-guidance |
| git_commit_date | 1750427857 | 1750447681 |
| git_commit_sha | 60c30c276e | f8ec06c202 |
| release_version | 1.51.0-SNAPSHOT~60c30c276e | 1.51.0-SNAPSHOT~f8ec06c202 |
See matching parameters
| Baseline | Candidate | |
|---|---|---|
| application | biojava | biojava |
| ci_job_date | 1750449763 | 1750449763 |
| ci_job_id | 992078703 | 992078703 |
| ci_pipeline_id | 68361068 | 68361068 |
| cpu_model | Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| kernel_version | Linux runner-bziamzy-project-304-concurrent-1-fv58j1gk 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux | Linux runner-bziamzy-project-304-concurrent-1-fv58j1gk 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
Summary
Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.
Execution time for biojava
gantt
title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e
dateFormat X
axisFormat %s
section baseline
no_agent (15.59 s) : 15590000, 15590000
. : milestone, 15590000,
appsec (15.101 s) : 15101000, 15101000
. : milestone, 15101000,
iast (18.597 s) : 18597000, 18597000
. : milestone, 18597000,
iast_GLOBAL (18.075 s) : 18075000, 18075000
. : milestone, 18075000,
profiling (15.369 s) : 15369000, 15369000
. : milestone, 15369000,
tracing (15.156 s) : 15156000, 15156000
. : milestone, 15156000,
section candidate
no_agent (14.768 s) : 14768000, 14768000
. : milestone, 14768000,
appsec (14.961 s) : 14961000, 14961000
. : milestone, 14961000,
iast (18.938 s) : 18938000, 18938000
. : milestone, 18938000,
iast_GLOBAL (17.98 s) : 17980000, 17980000
. : milestone, 17980000,
profiling (15.309 s) : 15309000, 15309000
. : milestone, 15309000,
tracing (14.932 s) : 14932000, 14932000
. : milestone, 14932000,
- baseline results
| Variant | Execution Time [CI 0.99] | Δ no_agent |
|---|---|---|
| no_agent | 15.59 s [15.59 s, 15.59 s] | - |
| appsec | 15.101 s [15.101 s, 15.101 s] | -489.0 ms (-3.1%) |
| iast | 18.597 s [18.597 s, 18.597 s] | 3.007 s (19.3%) |
| iast_GLOBAL | 18.075 s [18.075 s, 18.075 s] | 2.485 s (15.9%) |
| profiling | 15.369 s [15.369 s, 15.369 s] | -221.0 ms (-1.4%) |
| tracing | 15.156 s [15.156 s, 15.156 s] | -434.0 ms (-2.8%) |
- candidate results
| Variant | Execution Time [CI 0.99] | Δ no_agent |
|---|---|---|
| no_agent | 14.768 s [14.768 s, 14.768 s] | - |
| appsec | 14.961 s [14.961 s, 14.961 s] | 193.0 ms (1.3%) |
| iast | 18.938 s [18.938 s, 18.938 s] | 4.17 s (28.2%) |
| iast_GLOBAL | 17.98 s [17.98 s, 17.98 s] | 3.212 s (21.7%) |
| profiling | 15.309 s [15.309 s, 15.309 s] | 541.0 ms (3.7%) |
| tracing | 14.932 s [14.932 s, 14.932 s] | 164.0 ms (1.1%) |
Execution time for tomcat
gantt
title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e
dateFormat X
axisFormat %s
section baseline
no_agent (1.475 ms) : 1463, 1486
. : milestone, 1475,
appsec (2.396 ms) : 2347, 2444
. : milestone, 2396,
iast (2.194 ms) : 2132, 2255
. : milestone, 2194,
iast_GLOBAL (2.222 ms) : 2160, 2283
. : milestone, 2222,
profiling (2.048 ms) : 1998, 2099
. : milestone, 2048,
tracing (2.009 ms) : 1961, 2056
. : milestone, 2009,
section candidate
no_agent (1.475 ms) : 1464, 1487
. : milestone, 1475,
appsec (2.393 ms) : 2345, 2442
. : milestone, 2393,
iast (2.192 ms) : 2130, 2253
. : milestone, 2192,
iast_GLOBAL (2.227 ms) : 2165, 2289
. : milestone, 2227,
profiling (2.035 ms) : 1986, 2085
. : milestone, 2035,
tracing (1.996 ms) : 1949, 2043
. : milestone, 1996,
- baseline results
| Variant | Execution Time [CI 0.99] | Δ no_agent |
|---|---|---|
| no_agent | 1.475 ms [1.463 ms, 1.486 ms] | - |
| appsec | 2.396 ms [2.347 ms, 2.444 ms] | 921.382 µs (62.5%) |
| iast | 2.194 ms [2.132 ms, 2.255 ms] | 719.104 µs (48.8%) |
| iast_GLOBAL | 2.222 ms [2.16 ms, 2.283 ms] | 747.106 µs (50.7%) |
| profiling | 2.048 ms [1.998 ms, 2.099 ms] | 573.892 µs (38.9%) |
| tracing | 2.009 ms [1.961 ms, 2.056 ms] | 534.218 µs (36.2%) |
- candidate results
| Variant | Execution Time [CI 0.99] | Δ no_agent |
|---|---|---|
| no_agent | 1.475 ms [1.464 ms, 1.487 ms] | - |
| appsec | 2.393 ms [2.345 ms, 2.442 ms] | 918.427 µs (62.3%) |
| iast | 2.192 ms [2.13 ms, 2.253 ms] | 716.564 µs (48.6%) |
| iast_GLOBAL | 2.227 ms [2.165 ms, 2.289 ms] | 751.884 µs (51.0%) |
| profiling | 2.035 ms [1.986 ms, 2.085 ms] | 560.378 µs (38.0%) |
| tracing | 1.996 ms [1.949 ms, 2.043 ms] | 521.097 µs (35.3%) |
I also wonder if we can share links to our confluence docs about how to use / leverage our GitLab CI?
@PerfectSlayer that's a great idea. I'll open another pull request to add those.
Just realized, but #9088 has a lot of overlapping changes
Closing PR to preview review reminder. Feel free to re-open it if needed!