trafficstars
What Does This Do
Introduces InstrumenterModule::order() to declare ordering between modules. Modules with higher order values are applied after those with lower values. This is used to sort modules when discovering them via META-INF/services
The sorted list is used to create a build-time index of known modules and their Instrumenters. This index will be used to iterate over modules at runtime and provide a stable id for each expected Instrumenter
Additional Notes
We need stable Instrumenter ids to build the "known-types" index, which uses a trie to match against known class-names and return the matched Instrumenter ids. This new index lets us capture the expected sequence of modules and their Instrumenters. We to capture both because with the change to use modules the same Instrumenter type could appear under different modules, but constructed with different arguments. Recording the module surrounding the Instrumenter helps disambiguate between these different constructions.
Jira ticket: AIT-9441
Benchmarks
Startup
Parameters
|
Baseline |
Candidate |
| baseline_or_candidate |
baseline |
candidate |
| git_branch |
master |
mcculls/generate-instrumenter-index |
| git_commit_date |
1708348048 |
1708349577 |
| git_commit_sha |
b7f3d8e478 |
8e746069d2 |
| release_version |
1.31.0-SNAPSHOT~b7f3d8e478 |
1.31.0-SNAPSHOT~8e746069d2 |
See matching parameters
|
Baseline |
Candidate |
| application |
insecure-bank |
insecure-bank |
| ci_job_date |
1708352525 |
1708352525 |
| ci_job_id |
437277935 |
437277935 |
| ci_pipeline_id |
28513602 |
28513602 |
| cpu_model |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| module |
Agent |
Agent |
| parent |
None |
None |
| variant |
iast |
iast |
Summary
Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 8 unstable metrics.
Startup time reports for petclinic
gantt
title petclinic - global startup overhead: candidate=1.31.0-SNAPSHOT~8e746069d2, baseline=1.31.0-SNAPSHOT~b7f3d8e478
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.083 s) : 0, 1083305
Total [baseline] (9.179 s) : 0, 9179433
Agent [candidate] (1.082 s) : 0, 1081686
Total [candidate] (9.194 s) : 0, 9193924
section appsec
Agent [baseline] (1.176 s) : 0, 1176178
Total [baseline] (9.305 s) : 0, 9305125
Agent [candidate] (1.197 s) : 0, 1196654
Total [candidate] (9.387 s) : 0, 9386949
section iast
Agent [baseline] (1.221 s) : 0, 1220572
Total [baseline] (9.442 s) : 0, 9442356
Agent [candidate] (1.212 s) : 0, 1212189
Total [candidate] (9.397 s) : 0, 9397298
section profiling
Agent [baseline] (1.292 s) : 0, 1291847
Total [baseline] (9.507 s) : 0, 9506711
Agent [candidate] (1.291 s) : 0, 1290535
Total [candidate] (9.372 s) : 0, 9371598
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.083 s |
- |
| Agent |
appsec |
1.176 s |
92.873 ms (8.6%) |
| Agent |
iast |
1.221 s |
137.267 ms (12.7%) |
| Agent |
profiling |
1.292 s |
208.542 ms (19.3%) |
| Total |
tracing |
9.179 s |
- |
| Total |
appsec |
9.305 s |
125.692 ms (1.4%) |
| Total |
iast |
9.442 s |
262.923 ms (2.9%) |
| Total |
profiling |
9.507 s |
327.278 ms (3.6%) |
| Module |
Variant |
Duration |
Δ tracing |
| Agent |
tracing |
1.082 s |
- |
| Agent |
appsec |
1.197 s |
114.968 ms (10.6%) |
| Agent |
iast |
1.212 s |
130.502 ms (12.1%) |
| Agent |
profiling |
1.291 s |
208.849 ms (19.3%) |
| Total |
tracing |
9.194 s |
- |
| Total |
appsec |
9.387 s |
193.024 ms (2.1%) |
| Total |
iast |
9.397 s |
203.373 ms (2.2%) |
| Total |
profiling |
9.372 s |
177.673 ms (1.9%) |
gantt
title petclinic - break down per module: candidate=1.31.0-SNAPSHOT~8e746069d2, baseline=1.31.0-SNAPSHOT~b7f3d8e478
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (690.528 ms) : 0, 690528
BytebuddyAgent [candidate] (688.454 ms) : 0, 688454
GlobalTracer [baseline] (298.672 ms) : 0, 298672
GlobalTracer [candidate] (298.788 ms) : 0, 298788
AppSec [baseline] (51.199 ms) : 0, 51199
AppSec [candidate] (51.513 ms) : 0, 51513
Remote Config [baseline] (710.636 µs) : 0, 711
Remote Config [candidate] (713.607 µs) : 0, 714
Telemetry [baseline] (7.795 ms) : 0, 7795
Telemetry [candidate] (7.815 ms) : 0, 7815
section appsec
BytebuddyAgent [baseline] (685.75 ms) : 0, 685750
BytebuddyAgent [candidate] (699.765 ms) : 0, 699765
GlobalTracer [baseline] (297.365 ms) : 0, 297365
GlobalTracer [candidate] (302.103 ms) : 0, 302103
AppSec [baseline] (151.356 ms) : 0, 151356
AppSec [candidate] (152.114 ms) : 0, 152114
Remote Config [baseline] (639.905 µs) : 0, 640
Remote Config [candidate] (642.843 µs) : 0, 643
Telemetry [baseline] (6.943 ms) : 0, 6943
Telemetry [candidate] (6.962 ms) : 0, 6962
section iast
BytebuddyAgent [baseline] (808.614 ms) : 0, 808614
BytebuddyAgent [candidate] (802.621 ms) : 0, 802621
GlobalTracer [baseline] (292.743 ms) : 0, 292743
GlobalTracer [candidate] (290.961 ms) : 0, 290961
AppSec [baseline] (54.2 ms) : 0, 54200
AppSec [candidate] (56.085 ms) : 0, 56085
Remote Config [baseline] (628.205 µs) : 0, 628
Remote Config [candidate] (608.57 µs) : 0, 609
Telemetry [baseline] (8.096 ms) : 0, 8096
Telemetry [candidate] (7.37 ms) : 0, 7370
IAST [baseline] (21.425 ms) : 0, 21425
IAST [candidate] (19.951 ms) : 0, 19951
section profiling
BytebuddyAgent [baseline] (677.145 ms) : 0, 677145
BytebuddyAgent [candidate] (677.983 ms) : 0, 677983
GlobalTracer [baseline] (380.77 ms) : 0, 380770
GlobalTracer [candidate] (380.098 ms) : 0, 380098
AppSec [baseline] (53.257 ms) : 0, 53257
AppSec [candidate] (53.134 ms) : 0, 53134
Remote Config [baseline] (784.909 µs) : 0, 785
Remote Config [candidate] (811.736 µs) : 0, 812
Telemetry [baseline] (8.961 ms) : 0, 8961
Telemetry [candidate] (9.642 ms) : 0, 9642
ProfilingAgent [baseline] (115.251 ms) : 0, 115251
ProfilingAgent [candidate] (113.073 ms) : 0, 113073
Profiling [baseline] (115.275 ms) : 0, 115275
Profiling [candidate] (113.098 ms) : 0, 113098
Load
Parameters
|
Baseline |
Candidate |
| baseline_or_candidate |
baseline |
candidate |
| end_time |
2024-02-19T13:58:27 |
2024-02-19T14:17:05 |
| git_branch |
master |
mcculls/generate-instrumenter-index |
| git_commit_date |
1708348048 |
1708349577 |
| git_commit_sha |
b7f3d8e478 |
8e746069d2 |
| release_version |
1.31.0-SNAPSHOT~b7f3d8e478 |
1.31.0-SNAPSHOT~8e746069d2 |
| start_time |
2024-02-19T13:58:13 |
2024-02-19T14:16:51 |
See matching parameters
|
Baseline |
Candidate |
| application |
insecure-bank |
insecure-bank |
| ci_job_date |
1708352525 |
1708352525 |
| ci_job_id |
437277935 |
437277935 |
| ci_pipeline_id |
28513602 |
28513602 |
| cpu_model |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| variant |
iast |
iast |
Summary
Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 14 unstable metrics.
Request duration reports for petclinic
gantt
title petclinic - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~8e746069d2, baseline=1.31.0-SNAPSHOT~b7f3d8e478
dateFormat X
axisFormat %s
section baseline
no_agent (1.347 ms) : 1328, 1366
. : milestone, 1347,
appsec (1.792 ms) : 1768, 1816
. : milestone, 1792,
iast (1.513 ms) : 1490, 1537
. : milestone, 1513,
profiling (1.525 ms) : 1501, 1548
. : milestone, 1525,
tracing (1.526 ms) : 1503, 1550
. : milestone, 1526,
section candidate
no_agent (1.342 ms) : 1323, 1361
. : milestone, 1342,
appsec (1.782 ms) : 1758, 1806
. : milestone, 1782,
iast (1.525 ms) : 1502, 1549
. : milestone, 1525,
profiling (1.541 ms) : 1517, 1566
. : milestone, 1541,
tracing (1.509 ms) : 1486, 1533
. : milestone, 1509,
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
1.347 ms [1.328 ms, 1.366 ms] |
- |
| appsec |
1.792 ms [1.768 ms, 1.816 ms] |
444.845 µs (33.0%) |
| iast |
1.513 ms [1.49 ms, 1.537 ms] |
166.327 µs (12.3%) |
| profiling |
1.525 ms [1.501 ms, 1.548 ms] |
177.47 µs (13.2%) |
| tracing |
1.526 ms [1.503 ms, 1.55 ms] |
179.108 µs (13.3%) |
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
1.342 ms [1.323 ms, 1.361 ms] |
- |
| appsec |
1.782 ms [1.758 ms, 1.806 ms] |
440.109 µs (32.8%) |
| iast |
1.525 ms [1.502 ms, 1.549 ms] |
183.302 µs (13.7%) |
| profiling |
1.541 ms [1.517 ms, 1.566 ms] |
199.488 µs (14.9%) |
| tracing |
1.509 ms [1.486 ms, 1.533 ms] |
167.58 µs (12.5%) |
Request duration reports for insecure-bank
gantt
title insecure-bank - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~8e746069d2, baseline=1.31.0-SNAPSHOT~b7f3d8e478
dateFormat X
axisFormat %s
section baseline
no_agent (368.5 µs) : 349, 388
. : milestone, 369,
iast (482.628 µs) : 462, 504
. : milestone, 483,
iast_FULL (545.599 µs) : 525, 567
. : milestone, 546,
iast_GLOBAL (505.501 µs) : 484, 527
. : milestone, 506,
iast_HARDCODED_SECRET_DISABLED (479.995 µs) : 460, 500
. : milestone, 480,
iast_INACTIVE (450.737 µs) : 430, 472
. : milestone, 451,
iast_TELEMETRY_OFF (476.72 µs) : 456, 498
. : milestone, 477,
tracing (444.504 µs) : 424, 465
. : milestone, 445,
section candidate
no_agent (372.956 µs) : 353, 393
. : milestone, 373,
iast (475.53 µs) : 455, 496
. : milestone, 476,
iast_FULL (542.802 µs) : 522, 563
. : milestone, 543,
iast_GLOBAL (504.137 µs) : 484, 524
. : milestone, 504,
iast_HARDCODED_SECRET_DISABLED (477.781 µs) : 457, 498
. : milestone, 478,
iast_INACTIVE (448.525 µs) : 428, 469
. : milestone, 449,
iast_TELEMETRY_OFF (473.611 µs) : 453, 494
. : milestone, 474,
tracing (451.036 µs) : 430, 472
. : milestone, 451,
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
368.5 µs [348.694 µs, 388.306 µs] |
- |
| iast |
482.628 µs [461.574 µs, 503.681 µs] |
114.127 µs (31.0%) |
| iast_FULL |
545.599 µs [524.533 µs, 566.666 µs] |
177.099 µs (48.1%) |
| iast_GLOBAL |
505.501 µs [483.957 µs, 527.046 µs] |
137.001 µs (37.2%) |
| iast_HARDCODED_SECRET_DISABLED |
479.995 µs [459.651 µs, 500.339 µs] |
111.495 µs (30.3%) |
| iast_INACTIVE |
450.737 µs [429.9 µs, 471.574 µs] |
82.237 µs (22.3%) |
| iast_TELEMETRY_OFF |
476.72 µs [455.933 µs, 497.507 µs] |
108.22 µs (29.4%) |
| tracing |
444.504 µs [424.457 µs, 464.55 µs] |
76.003 µs (20.6%) |
| Variant |
Request duration [CI 0.99] |
Δ no_agent |
| no_agent |
372.956 µs [352.732 µs, 393.179 µs] |
- |
| iast |
475.53 µs [454.952 µs, 496.108 µs] |
102.574 µs (27.5%) |
| iast_FULL |
542.802 µs [522.268 µs, 563.336 µs] |
169.847 µs (45.5%) |
| iast_GLOBAL |
504.137 µs [483.953 µs, 524.32 µs] |
131.181 µs (35.2%) |
| iast_HARDCODED_SECRET_DISABLED |
477.781 µs [457.242 µs, 498.32 µs] |
104.825 µs (28.1%) |
| iast_INACTIVE |
448.525 µs [427.909 µs, 469.14 µs] |
75.569 µs (20.3%) |
| iast_TELEMETRY_OFF |
473.611 µs [453.056 µs, 494.167 µs] |
100.656 µs (27.0%) |
| tracing |
451.036 µs [429.798 µs, 472.273 µs] |
78.08 µs (20.9%) |
Kafka / producer-benchmark
Parameters
|
Baseline |
Candidate |
| baseline_or_candidate |
baseline |
candidate |
| git_branch |
mcculls/rename-instrumenter-group-to-module |
mcculls/generate-instrumenter-index |
| git_commit_date |
1707677899 |
1707679289 |
| git_commit_sha |
ef6278e1c0eb5262fe0ad48bfbded71d7abac127 |
c36f59fabc1ee46734a9b109215e5461dc13f065 |
See matching parameters
|
Baseline |
Candidate |
| ci_job_date |
1707681183 |
1707681183 |
| ci_job_id |
431200937 |
431200937 |
| ci_pipeline_id |
28093300 |
28093300 |
| cpu_model |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| jdkVersion |
11.0.21 |
11.0.21 |
| jmhVersion |
1.36 |
1.36 |
| jvm |
/usr/lib/jvm/java-11-openjdk-amd64/bin/java |
/usr/lib/jvm/java-11-openjdk-amd64/bin/java |
| jvmArgs |
-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant |
-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant |
| vmName |
OpenJDK 64-Bit Server VM |
OpenJDK 64-Bit Server VM |
| vmVersion |
11.0.21+9-post-Ubuntu-0ubuntu122.04 |
11.0.21+9-post-Ubuntu-0ubuntu122.04 |
Summary
Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.
See unchanged results
| scenario |
Δ mean throughput |
| scenario:not-instrumented/KafkaProduceBenchmark.benchProduce |
same |
| scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce |
same |
| scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce |
same |
Kafka / consumer-benchmark
Parameters
|
Baseline |
Candidate |
| baseline_or_candidate |
baseline |
candidate |
| git_branch |
mcculls/rename-instrumenter-group-to-module |
mcculls/generate-instrumenter-index |
| git_commit_date |
1707677899 |
1707679289 |
| git_commit_sha |
ef6278e1c0eb5262fe0ad48bfbded71d7abac127 |
c36f59fabc1ee46734a9b109215e5461dc13f065 |
See matching parameters
|
Baseline |
Candidate |
| ci_job_date |
1707681327 |
1707681327 |
| ci_job_id |
431200938 |
431200938 |
| ci_pipeline_id |
28093300 |
28093300 |
| cpu_model |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |
| jdkVersion |
11.0.21 |
11.0.21 |
| jmhVersion |
1.36 |
1.36 |
| jvm |
/usr/lib/jvm/java-11-openjdk-amd64/bin/java |
/usr/lib/jvm/java-11-openjdk-amd64/bin/java |
| jvmArgs |
-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant |
-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant |
| vmName |
OpenJDK 64-Bit Server VM |
OpenJDK 64-Bit Server VM |
| vmVersion |
11.0.21+9-post-Ubuntu-0ubuntu122.04 |
11.0.21+9-post-Ubuntu-0ubuntu122.04 |
Summary
Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.
See unchanged results
| scenario |
Δ mean throughput |
| scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume |
same |
| scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume |
same |
| scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume |
same |
dd-trace-java copied to clipboard
DataDog
![pr-commenter[bot] avatar](https://avatars.githubusercontent.com/u/365230?v=4 "Published by a pub.dev verified publisher"){kind=small}