trafficstars
What Does This Do
Remove details tag for spark.stage and spark.sql spans
Motivation
This field is very heavy, since it contains the full stacktrace that triggered the spark action, and not used within data jobs monitoring
It also has a limited usefulness since most stacktraces only contains internal spark code, not written by the user
Benchmarks
Startup
Parameters
|
Baseline |
Candidate |
| baseline_or_candidate |
baseline |
candidate |
| git_branch |
master |
paul.laffon/remove-spark-details |
| git_commit_date |
1701683052 |
1701682900 |
| git_commit_sha |
3e81c006b5 |
573c6a7ba7 |
| release_version |
1.25.0-SNAPSHOT~3e81c006b5 |
1.25.0-SNAPSHOT~573c6a7ba7 |
See matching parameters
|
Baseline |
Candidate |
| application |
insecure-bank |
insecure-bank |
| ci_job_date |
1701685543 |
1701685543 |
| ci_job_id |
379786136 |
379786136 |
| ci_pipeline_id |
24084861 |
24084861 |
| cpu_model |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| module |
Agent |
Agent |
| parent |
None |
None |
| variant |
iast |
iast |
Summary
Found 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 10 unstable metrics.
Startup time reports for insecure-bank
gantt
title insecure-bank - global startup overhead: candidate=1.25.0-SNAPSHOT~573c6a7ba7, baseline=1.25.0-SNAPSHOT~3e81c006b5
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.042 s) : 0, 1042319
Total [baseline] (8.717 s) : 0, 8717491
Agent [candidate] (1.051 s) : 0, 1050878
Total [candidate] (8.778 s) : 0, 8778310
section iast
Agent [baseline] (1.16 s) : 0, 1159544
Total [baseline] (9.238 s) : 0, 9237619
Agent [candidate] (1.161 s) : 0, 1160787
Total [candidate] (9.28 s) : 0, 9280060
section iast_TELEMETRY_OFF
Agent [baseline] (1.155 s) : 0, 1154825
Total [baseline] (9.287 s) : 0, 9286908
Agent [candidate] (1.152 s) : 0, 1152005
Total [candidate] (9.247 s) : 0, 9246946
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.042 s |
- |
| Agent |
iast |
1.16 s |
117.225 ms (11.2%) |
| Agent |
iast_TELEMETRY_OFF |
1.155 s |
112.506 ms (10.8%) |
| Total |
tracing |
8.717 s |
- |
| Total |
iast |
9.238 s |
520.127 ms (6.0%) |
| Total |
iast_TELEMETRY_OFF |
9.287 s |
569.416 ms (6.5%) |
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.051 s |
- |
| Agent |
iast |
1.161 s |
109.909 ms (10.5%) |
| Agent |
iast_TELEMETRY_OFF |
1.152 s |
101.127 ms (9.6%) |
| Total |
tracing |
8.778 s |
- |
| Total |
iast |
9.28 s |
501.75 ms (5.7%) |
| Total |
iast_TELEMETRY_OFF |
9.247 s |
468.636 ms (5.3%) |
gantt
title insecure-bank - break down per module: candidate=1.25.0-SNAPSHOT~573c6a7ba7, baseline=1.25.0-SNAPSHOT~3e81c006b5
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (644.709 ms) : 0, 644709
BytebuddyAgent [candidate] (650.301 ms) : 0, 650301
GlobalTracer [baseline] (306.843 ms) : 0, 306843
GlobalTracer [candidate] (309.136 ms) : 0, 309136
AppSec [baseline] (48.723 ms) : 0, 48723
AppSec [candidate] (49.067 ms) : 0, 49067
Remote Config [baseline] (671.825 µs) : 0, 672
Remote Config [candidate] (668.528 µs) : 0, 669
Telemetry [baseline] (7.142 ms) : 0, 7142
Telemetry [candidate] (7.116 ms) : 0, 7116
section iast
BytebuddyAgent [baseline] (766.209 ms) : 0, 766209
BytebuddyAgent [candidate] (766.338 ms) : 0, 766338
GlobalTracer [baseline] (284.588 ms) : 0, 284588
GlobalTracer [candidate] (285.019 ms) : 0, 285019
AppSec [baseline] (46.408 ms) : 0, 46408
AppSec [candidate] (46.473 ms) : 0, 46473
Remote Config [baseline] (595.027 µs) : 0, 595
Remote Config [candidate] (586.396 µs) : 0, 586
Telemetry [baseline] (6.602 ms) : 0, 6602
Telemetry [candidate] (7.31 ms) : 0, 7310
IAST [baseline] (20.748 ms) : 0, 20748
IAST [candidate] (20.773 ms) : 0, 20773
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (760.395 ms) : 0, 760395
BytebuddyAgent [candidate] (758.77 ms) : 0, 758770
GlobalTracer [baseline] (285.885 ms) : 0, 285885
GlobalTracer [candidate] (285.214 ms) : 0, 285214
AppSec [baseline] (49.078 ms) : 0, 49078
AppSec [candidate] (46.977 ms) : 0, 46977
Remote Config [baseline] (1.277 ms) : 0, 1277
Remote Config [candidate] (2.104 ms) : 0, 2104
Telemetry [baseline] (7.137 ms) : 0, 7137
Telemetry [candidate] (7.324 ms) : 0, 7324
IAST [baseline] (16.653 ms) : 0, 16653
IAST [candidate] (17.44 ms) : 0, 17440
Startup time reports for petclinic
gantt
title petclinic - global startup overhead: candidate=1.25.0-SNAPSHOT~573c6a7ba7, baseline=1.25.0-SNAPSHOT~3e81c006b5
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.052 s) : 0, 1052028
Total [baseline] (9.339 s) : 0, 9338539
Agent [candidate] (1.044 s) : 0, 1044234
Total [candidate] (9.337 s) : 0, 9337464
section appsec
Agent [baseline] (1.134 s) : 0, 1133700
Total [baseline] (9.496 s) : 0, 9495601
Agent [candidate] (1.131 s) : 0, 1131284
Total [candidate] (9.451 s) : 0, 9450551
section iast
Agent [baseline] (1.169 s) : 0, 1168827
Total [baseline] (9.692 s) : 0, 9692107
Agent [candidate] (1.171 s) : 0, 1170571
Total [candidate] (9.66 s) : 0, 9659626
section profiling
Agent [baseline] (1.23 s) : 0, 1230242
Total [baseline] (9.613 s) : 0, 9613285
Agent [candidate] (1.235 s) : 0, 1234614
Total [candidate] (9.66 s) : 0, 9659882
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.052 s |
- |
| Agent |
appsec |
1.134 s |
81.672 ms (7.8%) |
| Agent |
iast |
1.169 s |
116.799 ms (11.1%) |
| Agent |
profiling |
1.23 s |
178.214 ms (16.9%) |
| Total |
tracing |
9.339 s |
- |
| Total |
appsec |
9.496 s |
157.062 ms (1.7%) |
| Total |
iast |
9.692 s |
353.568 ms (3.8%) |
| Total |
profiling |
9.613 s |
274.746 ms (2.9%) |
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.044 s |
- |
| Agent |
appsec |
1.131 s |
87.05 ms (8.3%) |
| Agent |
iast |
1.171 s |
126.337 ms (12.1%) |
| Agent |
profiling |
1.235 s |
190.379 ms (18.2%) |
| Total |
tracing |
9.337 s |
- |
| Total |
appsec |
9.451 s |
113.086 ms (1.2%) |
| Total |
iast |
9.66 s |
322.162 ms (3.5%) |
| Total |
profiling |
9.66 s |
322.417 ms (3.5%) |
gantt
title petclinic - break down per module: candidate=1.25.0-SNAPSHOT~573c6a7ba7, baseline=1.25.0-SNAPSHOT~3e81c006b5
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (650.738 ms) : 0, 650738
BytebuddyAgent [candidate] (646.037 ms) : 0, 646037
GlobalTracer [baseline] (309.075 ms) : 0, 309075
GlobalTracer [candidate] (307.091 ms) : 0, 307091
AppSec [baseline] (49.837 ms) : 0, 49837
AppSec [candidate] (49.135 ms) : 0, 49135
Remote Config [baseline] (681.48 µs) : 0, 681
Remote Config [candidate] (662.53 µs) : 0, 663
Telemetry [baseline] (7.115 ms) : 0, 7115
Telemetry [candidate] (7.077 ms) : 0, 7077
section appsec
BytebuddyAgent [baseline] (645.731 ms) : 0, 645731
BytebuddyAgent [candidate] (644.709 ms) : 0, 644709
GlobalTracer [baseline] (306.124 ms) : 0, 306124
GlobalTracer [candidate] (306.904 ms) : 0, 306904
AppSec [baseline] (137.369 ms) : 0, 137369
AppSec [candidate] (136.758 ms) : 0, 136758
Remote Config [baseline] (649.243 µs) : 0, 649
Remote Config [candidate] (646.06 µs) : 0, 646
Telemetry [baseline] (9.476 ms) : 0, 9476
Telemetry [candidate] (8.02 ms) : 0, 8020
section iast
BytebuddyAgent [baseline] (772.375 ms) : 0, 772375
BytebuddyAgent [candidate] (773.679 ms) : 0, 773679
GlobalTracer [baseline] (287.041 ms) : 0, 287041
GlobalTracer [candidate] (287.603 ms) : 0, 287603
AppSec [baseline] (46.654 ms) : 0, 46654
AppSec [candidate] (46.654 ms) : 0, 46654
Remote Config [baseline] (615.093 µs) : 0, 615
Remote Config [candidate] (630.359 µs) : 0, 630
Telemetry [baseline] (8.808 ms) : 0, 8808
Telemetry [candidate] (6.611 ms) : 0, 6611
IAST [baseline] (18.732 ms) : 0, 18732
IAST [candidate] (20.767 ms) : 0, 20767
section profiling
BytebuddyAgent [baseline] (654.579 ms) : 0, 654579
BytebuddyAgent [candidate] (657.31 ms) : 0, 657310
GlobalTracer [baseline] (376.543 ms) : 0, 376543
GlobalTracer [candidate] (377.133 ms) : 0, 377133
AppSec [baseline] (48.335 ms) : 0, 48335
AppSec [candidate] (48.499 ms) : 0, 48499
Remote Config [baseline] (694.876 µs) : 0, 695
Remote Config [candidate] (678.576 µs) : 0, 679
Telemetry [baseline] (7.402 ms) : 0, 7402
Telemetry [candidate] (7.422 ms) : 0, 7422
ProfilingAgent [baseline] (88.628 ms) : 0, 88628
ProfilingAgent [candidate] (89.376 ms) : 0, 89376
Profiling [baseline] (88.653 ms) : 0, 88653
Profiling [candidate] (89.401 ms) : 0, 89401
Load
Parameters
|
Baseline |
Candidate |
| baseline_or_candidate |
baseline |
candidate |
| end_time |
2023-12-04T10:04:53 |
2023-12-04T10:21:29 |
| git_branch |
master |
paul.laffon/remove-spark-details |
| git_commit_date |
1701683052 |
1701682900 |
| git_commit_sha |
3e81c006b5 |
573c6a7ba7 |
| release_version |
1.25.0-SNAPSHOT~3e81c006b5 |
1.25.0-SNAPSHOT~573c6a7ba7 |
| start_time |
2023-12-04T10:04:40 |
2023-12-04T10:21:16 |
See matching parameters
|
Baseline |
Candidate |
| application |
insecure-bank |
insecure-bank |
| ci_job_date |
1701685543 |
1701685543 |
| ci_job_id |
379786136 |
379786136 |
| ci_pipeline_id |
24084861 |
24084861 |
| cpu_model |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| variant |
iast |
iast |
Summary
Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 13 unstable metrics.
Request duration reports for insecure-bank
gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.25.0-SNAPSHOT~573c6a7ba7, baseline=1.25.0-SNAPSHOT~3e81c006b5
dateFormat X
axisFormat %s
section baseline
no_agent (373.597 µs) : 353, 394
. : milestone, 374,
iast (474.91 µs) : 454, 496
. : milestone, 475,
iast_FULL (538.204 µs) : 518, 559
. : milestone, 538,
iast_INACTIVE (449.665 µs) : 428, 471
. : milestone, 450,
iast_TELEMETRY_OFF (469.491 µs) : 448, 491
. : milestone, 469,
tracing (446.891 µs) : 427, 467
. : milestone, 447,
section candidate
no_agent (372.229 µs) : 351, 393
. : milestone, 372,
iast (485.645 µs) : 464, 507
. : milestone, 486,
iast_FULL (535.341 µs) : 515, 556
. : milestone, 535,
iast_INACTIVE (450.589 µs) : 430, 472
. : milestone, 451,
iast_TELEMETRY_OFF (468.095 µs) : 448, 489
. : milestone, 468,
tracing (450.332 µs) : 429, 471
. : milestone, 450,
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
373.597 µs [353.342 µs, 393.851 µs] |
- |
| iast |
474.91 µs [454.243 µs, 495.576 µs] |
101.313 µs (27.1%) |
| iast_FULL |
538.204 µs [517.792 µs, 558.617 µs] |
164.607 µs (44.1%) |
| iast_INACTIVE |
449.665 µs [428.248 µs, 471.082 µs] |
76.068 µs (20.4%) |
| iast_TELEMETRY_OFF |
469.491 µs [448.31 µs, 490.672 µs] |
95.894 µs (25.7%) |
| tracing |
446.891 µs [426.5 µs, 467.282 µs] |
73.294 µs (19.6%) |
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
372.229 µs [351.425 µs, 393.034 µs] |
- |
| iast |
485.645 µs [464.138 µs, 507.151 µs] |
113.415 µs (30.5%) |
| iast_FULL |
535.341 µs [514.792 µs, 555.889 µs] |
163.111 µs (43.8%) |
| iast_INACTIVE |
450.589 µs [429.611 µs, 471.568 µs] |
78.36 µs (21.1%) |
| iast_TELEMETRY_OFF |
468.095 µs [447.683 µs, 488.506 µs] |
95.865 µs (25.8%) |
| tracing |
450.332 µs [429.491 µs, 471.173 µs] |
78.102 µs (21.0%) |
Request duration reports for petclinic
gantt
title petclinic - request duration [CI 0.99] : candidate=1.25.0-SNAPSHOT~573c6a7ba7, baseline=1.25.0-SNAPSHOT~3e81c006b5
dateFormat X
axisFormat %s
section baseline
no_agent (1.369 ms) : 1350, 1389
. : milestone, 1369,
appsec (1.775 ms) : 1750, 1801
. : milestone, 1775,
iast (1.528 ms) : 1503, 1552
. : milestone, 1528,
profiling (1.532 ms) : 1507, 1556
. : milestone, 1532,
tracing (1.513 ms) : 1488, 1538
. : milestone, 1513,
section candidate
no_agent (1.37 ms) : 1351, 1389
. : milestone, 1370,
appsec (1.771 ms) : 1746, 1797
. : milestone, 1771,
iast (1.51 ms) : 1486, 1535
. : milestone, 1510,
profiling (1.563 ms) : 1537, 1590
. : milestone, 1563,
tracing (1.486 ms) : 1461, 1511
. : milestone, 1486,
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
1.369 ms [1.35 ms, 1.389 ms] |
- |
| appsec |
1.775 ms [1.75 ms, 1.801 ms] |
405.995 µs (29.6%) |
| iast |
1.528 ms [1.503 ms, 1.552 ms] |
158.153 µs (11.5%) |
| profiling |
1.532 ms [1.507 ms, 1.556 ms] |
162.181 µs (11.8%) |
| tracing |
1.513 ms [1.488 ms, 1.538 ms] |
143.707 µs (10.5%) |
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
1.37 ms [1.351 ms, 1.389 ms] |
- |
| appsec |
1.771 ms [1.746 ms, 1.797 ms] |
401.075 µs (29.3%) |
| iast |
1.51 ms [1.486 ms, 1.535 ms] |
140.029 µs (10.2%) |
| profiling |
1.563 ms [1.537 ms, 1.59 ms] |
193.321 µs (14.1%) |
| tracing |
1.486 ms [1.461 ms, 1.511 ms] |
115.881 µs (8.5%) |
dd-trace-java copied to clipboard
DataDog
![pr-commenter[bot] avatar](https://avatars.githubusercontent.com/u/365230?v=4 "Published by a pub.dev verified publisher"){kind=small}