trafficstars
What Does This Do
Adds support for dependency collection with telemetry on native image.
Dependency collection is done during the native image build (analysis stage), and embedded in the final binary within resource files. These are later read at runtime to send them through telemetry.
Motivation
Additional Notes
Depends on:
- https://github.com/DataDog/dd-trace-java/pull/6286 for the initial telemetry support
To do:
- [ ] Check if the scenario of multiple analysis runs with multiple dependencies files is possible and works.
Benchmarks
Startup
Parameters
|
Baseline |
Candidate |
| baseline_or_candidate |
baseline |
candidate |
| git_branch |
master |
smola/graalvm-telemetry-deps |
| git_commit_date |
1701362494 |
1701369930 |
| git_commit_sha |
451af09813 |
307f544d18 |
| release_version |
1.25.0-SNAPSHOT~451af09813 |
1.25.0-SNAPSHOT~307f544d18 |
See matching parameters
|
Baseline |
Candidate |
| application |
insecure-bank |
insecure-bank |
| ci_job_date |
1701372466 |
1701372466 |
| ci_job_id |
378715270 |
378715270 |
| ci_pipeline_id |
24000290 |
24000290 |
| cpu_model |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| module |
Agent |
Agent |
| parent |
None |
None |
| variant |
iast |
iast |
Summary
Found 0 performance improvements and 2 performance regressions! Performance is the same for 42 metrics, 10 unstable metrics.
| scenario |
Δ mean execution_time |
candidate mean execution_time |
baseline mean execution_time |
| scenario:startup:insecure-bank:tracing:Telemetry |
worse
[+156.061µs; +504.807µs] or [+2.131%; +6.892%] |
7.655ms |
7.325ms |
| scenario:startup:petclinic:tracing:Telemetry |
worse
[+197.494µs; +529.893µs] or [+2.721%; +7.301%] |
7.622ms |
7.258ms |
Startup time reports for petclinic
gantt
title petclinic - global startup overhead: candidate=1.25.0-SNAPSHOT~307f544d18, baseline=1.25.0-SNAPSHOT~451af09813
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.041 s) : 0, 1041260
Total [baseline] (9.406 s) : 0, 9405892
Agent [candidate] (1.046 s) : 0, 1046225
Total [candidate] (9.42 s) : 0, 9420128
section appsec
Agent [baseline] (1.123 s) : 0, 1122778
Total [baseline] (9.51 s) : 0, 9510050
Agent [candidate] (1.129 s) : 0, 1128818
Total [candidate] (9.423 s) : 0, 9422534
section iast
Agent [baseline] (1.153 s) : 0, 1153277
Total [baseline] (9.57 s) : 0, 9570258
Agent [candidate] (1.151 s) : 0, 1150791
Total [candidate] (9.561 s) : 0, 9561093
section profiling
Agent [baseline] (1.242 s) : 0, 1242228
Total [baseline] (9.725 s) : 0, 9725006
Agent [candidate] (1.223 s) : 0, 1223135
Total [candidate] (9.644 s) : 0, 9644304
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.041 s |
- |
| Agent |
appsec |
1.123 s |
81.518 ms (7.8%) |
| Agent |
iast |
1.153 s |
112.017 ms (10.8%) |
| Agent |
profiling |
1.242 s |
200.968 ms (19.3%) |
| Total |
tracing |
9.406 s |
- |
| Total |
appsec |
9.51 s |
104.158 ms (1.1%) |
| Total |
iast |
9.57 s |
164.366 ms (1.7%) |
| Total |
profiling |
9.725 s |
319.115 ms (3.4%) |
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.046 s |
- |
| Agent |
appsec |
1.129 s |
82.593 ms (7.9%) |
| Agent |
iast |
1.151 s |
104.566 ms (10.0%) |
| Agent |
profiling |
1.223 s |
176.91 ms (16.9%) |
| Total |
tracing |
9.42 s |
- |
| Total |
appsec |
9.423 s |
2.406 ms (0.0%) |
| Total |
iast |
9.561 s |
140.965 ms (1.5%) |
| Total |
profiling |
9.644 s |
224.176 ms (2.4%) |
gantt
title petclinic - break down per module: candidate=1.25.0-SNAPSHOT~307f544d18, baseline=1.25.0-SNAPSHOT~451af09813
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (649.779 ms) : 0, 649779
BytebuddyAgent [candidate] (652.581 ms) : 0, 652581
GlobalTracer [baseline] (300.183 ms) : 0, 300183
GlobalTracer [candidate] (301.569 ms) : 0, 301569
AppSec [baseline] (48.806 ms) : 0, 48806
AppSec [candidate] (49.079 ms) : 0, 49079
Remote Config [baseline] (689.144 µs) : 0, 689
Remote Config [candidate] (699.004 µs) : 0, 699
Telemetry [baseline] (7.258 ms) : 0, 7258
Telemetry [candidate] (7.622 ms) : 0, 7622
section appsec
BytebuddyAgent [baseline] (645.363 ms) : 0, 645363
BytebuddyAgent [candidate] (648.921 ms) : 0, 648921
GlobalTracer [baseline] (298.798 ms) : 0, 298798
GlobalTracer [candidate] (300.368 ms) : 0, 300368
AppSec [baseline] (136.284 ms) : 0, 136284
AppSec [candidate] (137.242 ms) : 0, 137242
Remote Config [baseline] (651.217 µs) : 0, 651
Remote Config [candidate] (647.253 µs) : 0, 647
Telemetry [baseline] (7.409 ms) : 0, 7409
Telemetry [candidate] (7.154 ms) : 0, 7154
section iast
BytebuddyAgent [baseline] (765.256 ms) : 0, 765256
BytebuddyAgent [candidate] (763.87 ms) : 0, 763870
GlobalTracer [baseline] (278.743 ms) : 0, 278743
GlobalTracer [candidate] (278.197 ms) : 0, 278197
AppSec [baseline] (46.482 ms) : 0, 46482
AppSec [candidate] (46.507 ms) : 0, 46507
Remote Config [baseline] (651.753 µs) : 0, 652
Remote Config [candidate] (1.317 ms) : 0, 1317
Telemetry [baseline] (9.37 ms) : 0, 9370
Telemetry [candidate] (7.535 ms) : 0, 7535
IAST [baseline] (18.551 ms) : 0, 18551
IAST [candidate] (19.168 ms) : 0, 19168
section profiling
ProfilingAgent [baseline] (89.482 ms) : 0, 89482
ProfilingAgent [candidate] (88.406 ms) : 0, 88406
BytebuddyAgent [baseline] (665.549 ms) : 0, 665549
BytebuddyAgent [candidate] (654.726 ms) : 0, 654726
GlobalTracer [baseline] (374.326 ms) : 0, 374326
GlobalTracer [candidate] (368.751 ms) : 0, 368751
AppSec [baseline] (49.808 ms) : 0, 49808
AppSec [candidate] (49.068 ms) : 0, 49068
Remote Config [baseline] (721.575 µs) : 0, 722
Remote Config [candidate] (708.64 µs) : 0, 709
Telemetry [baseline] (7.511 ms) : 0, 7511
Telemetry [candidate] (7.562 ms) : 0, 7562
Profiling [baseline] (89.505 ms) : 0, 89505
Profiling [candidate] (88.429 ms) : 0, 88429
Startup time reports for insecure-bank
gantt
title insecure-bank - global startup overhead: candidate=1.25.0-SNAPSHOT~307f544d18, baseline=1.25.0-SNAPSHOT~451af09813
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.038 s) : 0, 1037619
Total [baseline] (8.71 s) : 0, 8710008
Agent [candidate] (1.043 s) : 0, 1042881
Total [candidate] (8.737 s) : 0, 8737041
section iast
Agent [baseline] (1.152 s) : 0, 1151712
Total [baseline] (9.218 s) : 0, 9218020
Agent [candidate] (1.163 s) : 0, 1162892
Total [candidate] (9.226 s) : 0, 9225691
section iast_TELEMETRY_OFF
Agent [baseline] (1.145 s) : 0, 1145422
Total [baseline] (9.245 s) : 0, 9244632
Agent [candidate] (1.153 s) : 0, 1152744
Total [candidate] (9.239 s) : 0, 9239190
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.038 s |
- |
| Agent |
iast |
1.152 s |
114.093 ms (11.0%) |
| Agent |
iast_TELEMETRY_OFF |
1.145 s |
107.804 ms (10.4%) |
| Total |
tracing |
8.71 s |
- |
| Total |
iast |
9.218 s |
508.012 ms (5.8%) |
| Total |
iast_TELEMETRY_OFF |
9.245 s |
534.623 ms (6.1%) |
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.043 s |
- |
| Agent |
iast |
1.163 s |
120.011 ms (11.5%) |
| Agent |
iast_TELEMETRY_OFF |
1.153 s |
109.864 ms (10.5%) |
| Total |
tracing |
8.737 s |
- |
| Total |
iast |
9.226 s |
488.65 ms (5.6%) |
| Total |
iast_TELEMETRY_OFF |
9.239 s |
502.149 ms (5.7%) |
gantt
title insecure-bank - break down per module: candidate=1.25.0-SNAPSHOT~307f544d18, baseline=1.25.0-SNAPSHOT~451af09813
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (647.512 ms) : 0, 647512
BytebuddyAgent [candidate] (650.707 ms) : 0, 650707
GlobalTracer [baseline] (299.158 ms) : 0, 299158
GlobalTracer [candidate] (300.228 ms) : 0, 300228
AppSec [baseline] (48.5 ms) : 0, 48500
AppSec [candidate] (49.008 ms) : 0, 49008
Remote Config [baseline] (688.391 µs) : 0, 688
Remote Config [candidate] (691.136 µs) : 0, 691
Telemetry [baseline] (7.325 ms) : 0, 7325
Telemetry [candidate] (7.655 ms) : 0, 7655
section iast
BytebuddyAgent [baseline] (764.438 ms) : 0, 764438
BytebuddyAgent [candidate] (772.26 ms) : 0, 772260
GlobalTracer [baseline] (277.943 ms) : 0, 277943
GlobalTracer [candidate] (280.461 ms) : 0, 280461
AppSec [baseline] (48.154 ms) : 0, 48154
AppSec [candidate] (48.909 ms) : 0, 48909
IAST [baseline] (18.376 ms) : 0, 18376
IAST [candidate] (17.127 ms) : 0, 17127
Remote Config [baseline] (569.637 µs) : 0, 570
Remote Config [candidate] (574.102 µs) : 0, 574
Telemetry [baseline] (7.921 ms) : 0, 7921
Telemetry [candidate] (8.917 ms) : 0, 8917
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (758.999 ms) : 0, 758999
BytebuddyAgent [candidate] (764.033 ms) : 0, 764033
GlobalTracer [baseline] (278.888 ms) : 0, 278888
GlobalTracer [candidate] (280.959 ms) : 0, 280959
AppSec [baseline] (51.009 ms) : 0, 51009
AppSec [candidate] (52.279 ms) : 0, 52279
IAST [baseline] (14.546 ms) : 0, 14546
IAST [candidate] (13.854 ms) : 0, 13854
Remote Config [baseline] (570.796 µs) : 0, 571
Remote Config [candidate] (561.845 µs) : 0, 562
Telemetry [baseline] (7.04 ms) : 0, 7040
Telemetry [candidate] (6.644 ms) : 0, 6644
Load
Parameters
|
Baseline |
Candidate |
| baseline_or_candidate |
baseline |
candidate |
| end_time |
2023-11-30T19:06:56 |
2023-11-30T19:23:30 |
| git_branch |
master |
smola/graalvm-telemetry-deps |
| git_commit_date |
1701362494 |
1701369930 |
| git_commit_sha |
451af09813 |
307f544d18 |
| release_version |
1.25.0-SNAPSHOT~451af09813 |
1.25.0-SNAPSHOT~307f544d18 |
| start_time |
2023-11-30T19:06:43 |
2023-11-30T19:23:17 |
See matching parameters
|
Baseline |
Candidate |
| application |
insecure-bank |
insecure-bank |
| ci_job_date |
1701372466 |
1701372466 |
| ci_job_id |
378715270 |
378715270 |
| ci_pipeline_id |
24000290 |
24000290 |
| cpu_model |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| variant |
iast |
iast |
Summary
Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 12 unstable metrics.
Request duration reports for petclinic
gantt
title petclinic - request duration [CI 0.99] : candidate=1.25.0-SNAPSHOT~307f544d18, baseline=1.25.0-SNAPSHOT~451af09813
dateFormat X
axisFormat %s
section baseline
no_agent (1.348 ms) : 1329, 1367
. : milestone, 1348,
appsec (1.74 ms) : 1716, 1765
. : milestone, 1740,
iast (1.518 ms) : 1493, 1542
. : milestone, 1518,
profiling (1.51 ms) : 1485, 1535
. : milestone, 1510,
tracing (1.5 ms) : 1474, 1525
. : milestone, 1500,
section candidate
no_agent (1.353 ms) : 1333, 1373
. : milestone, 1353,
appsec (1.788 ms) : 1762, 1813
. : milestone, 1788,
iast (1.552 ms) : 1527, 1576
. : milestone, 1552,
profiling (1.552 ms) : 1525, 1579
. : milestone, 1552,
tracing (1.52 ms) : 1495, 1546
. : milestone, 1520,
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
1.348 ms [1.329 ms, 1.367 ms] |
- |
| appsec |
1.74 ms [1.716 ms, 1.765 ms] |
392.183 µs (29.1%) |
| iast |
1.518 ms [1.493 ms, 1.542 ms] |
169.712 µs (12.6%) |
| profiling |
1.51 ms [1.485 ms, 1.535 ms] |
162.36 µs (12.0%) |
| tracing |
1.5 ms [1.474 ms, 1.525 ms] |
151.76 µs (11.3%) |
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
1.353 ms [1.333 ms, 1.373 ms] |
- |
| appsec |
1.788 ms [1.762 ms, 1.813 ms] |
434.425 µs (32.1%) |
| iast |
1.552 ms [1.527 ms, 1.576 ms] |
198.756 µs (14.7%) |
| profiling |
1.552 ms [1.525 ms, 1.579 ms] |
198.71 µs (14.7%) |
| tracing |
1.52 ms [1.495 ms, 1.546 ms] |
167.373 µs (12.4%) |
Request duration reports for insecure-bank
gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.25.0-SNAPSHOT~307f544d18, baseline=1.25.0-SNAPSHOT~451af09813
dateFormat X
axisFormat %s
section baseline
no_agent (370.886 µs) : 351, 391
. : milestone, 371,
iast (475.716 µs) : 455, 497
. : milestone, 476,
iast_FULL (539.986 µs) : 519, 560
. : milestone, 540,
iast_INACTIVE (445.675 µs) : 425, 467
. : milestone, 446,
iast_TELEMETRY_OFF (472.73 µs) : 452, 493
. : milestone, 473,
tracing (448.051 µs) : 427, 469
. : milestone, 448,
section candidate
no_agent (365.914 µs) : 346, 385
. : milestone, 366,
iast (472.523 µs) : 452, 493
. : milestone, 473,
iast_FULL (536.628 µs) : 516, 557
. : milestone, 537,
iast_INACTIVE (448.837 µs) : 429, 469
. : milestone, 449,
iast_TELEMETRY_OFF (487.729 µs) : 467, 508
. : milestone, 488,
tracing (438.3 µs) : 418, 458
. : milestone, 438,
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
370.886 µs [350.74 µs, 391.033 µs] |
- |
| iast |
475.716 µs [454.558 µs, 496.875 µs] |
104.83 µs (28.3%) |
| iast_FULL |
539.986 µs [519.499 µs, 560.473 µs] |
169.1 µs (45.6%) |
| iast_INACTIVE |
445.675 µs [424.79 µs, 466.559 µs] |
74.788 µs (20.2%) |
| iast_TELEMETRY_OFF |
472.73 µs [452.12 µs, 493.339 µs] |
101.843 µs (27.5%) |
| tracing |
448.051 µs [426.839 µs, 469.264 µs] |
77.165 µs (20.8%) |
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
365.914 µs [346.375 µs, 385.453 µs] |
- |
| iast |
472.523 µs [452.129 µs, 492.918 µs] |
106.609 µs (29.1%) |
| iast_FULL |
536.628 µs [516.068 µs, 557.187 µs] |
170.714 µs (46.7%) |
| iast_INACTIVE |
448.837 µs [428.631 µs, 469.044 µs] |
82.923 µs (22.7%) |
| iast_TELEMETRY_OFF |
487.729 µs [467.12 µs, 508.338 µs] |
121.816 µs (33.3%) |
| tracing |
438.3 µs [418.302 µs, 458.297 µs] |
72.386 µs (19.8%) |
dd-trace-java copied to clipboard
DataDog
![pr-commenter[bot] avatar](https://avatars.githubusercontent.com/u/365230?v=4 "Published by a pub.dev verified publisher"){kind=small}