dd-trace-java icon indicating copy to clipboard operation
dd-trace-java copied to clipboard

Published 20 hours ago • verified publisher icon DataDog

🍒 9019 - Fix NPE in akka-http and pekko-http integrations

Open mcculls opened this issue 5 months ago • 1 comments

Backport #9019 to release/v1.50.x

mcculls avatar Jun 27 '25 14:06 mcculls

Benchmarks

Startup

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master stuart.mcculloch/backport-pr-9019
git_commit_date 1751033772 1751034047
git_commit_sha 640fc88be4 e319c2b580
release_version 1.51.0-SNAPSHOT~640fc88be4 1.51.0-SNAPSHOT~e319c2b580
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1751035580 1751035580
ci_job_id 1002547602 1002547602
ci_pipeline_id 68995237 68995237
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-tcrivaq7-project-304-concurrent-5-52dlx4pl 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-tcrivaq7-project-304-concurrent-5-52dlx4pl 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 2 performance regressions! Performance is the same for 10 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:no_agent:high_load worse
[+88.573µs; +196.782µs] or [+2.154%; +4.786%]		 unstable
[-163.760op/s; +89.510op/s] or [-14.684%; +8.026%]		 4.255ms 1078.125op/s 4.112ms 1115.250op/s
scenario:load:petclinic:profiling:high_load worse
[+1.597ms; +2.491ms] or [+3.536%; +5.514%]		 unstable
[-13.214op/s; +1.666op/s] or [-12.594%; +1.588%]		 47.213ms 99.150op/s 45.169ms 104.924op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~e319c2b580, baseline=1.51.0-SNAPSHOT~640fc88be4
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.112 ms) : 4062, 4161
.   : milestone, 4112,
iast (9.207 ms) : 9057, 9357
.   : milestone, 9207,
iast_FULL (13.413 ms) : 13148, 13677
.   : milestone, 13413,
iast_GLOBAL (10.056 ms) : 9865, 10246
.   : milestone, 10056,
profiling (8.915 ms) : 8763, 9068
.   : milestone, 8915,
tracing (7.663 ms) : 7547, 7780
.   : milestone, 7663,
section candidate
no_agent (4.255 ms) : 4204, 4306
.   : milestone, 4255,
iast (8.873 ms) : 8722, 9024
.   : milestone, 8873,
iast_FULL (13.971 ms) : 13692, 14250
.   : milestone, 13971,
iast_GLOBAL (10.327 ms) : 10143, 10511
.   : milestone, 10327,
profiling (8.608 ms) : 8469, 8747
.   : milestone, 8608,
tracing (7.553 ms) : 7445, 7660
.   : milestone, 7553,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.112 ms [4.062 ms, 4.161 ms] -
iast 9.207 ms [9.057 ms, 9.357 ms] 5.095 ms (123.9%)
iast_FULL 13.413 ms [13.148 ms, 13.677 ms] 9.301 ms (226.2%)
iast_GLOBAL 10.056 ms [9.865 ms, 10.246 ms] 5.944 ms (144.5%)
profiling 8.915 ms [8.763 ms, 9.068 ms] 4.803 ms (116.8%)
tracing 7.663 ms [7.547 ms, 7.78 ms] 3.552 ms (86.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.255 ms [4.204 ms, 4.306 ms] -
iast 8.873 ms [8.722 ms, 9.024 ms] 4.618 ms (108.5%)
iast_FULL 13.971 ms [13.692 ms, 14.25 ms] 9.716 ms (228.4%)
iast_GLOBAL 10.327 ms [10.143 ms, 10.511 ms] 6.073 ms (142.7%)
profiling 8.608 ms [8.469 ms, 8.747 ms] 4.353 ms (102.3%)
tracing 7.553 ms [7.445 ms, 7.66 ms] 3.298 ms (77.5%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~e319c2b580, baseline=1.51.0-SNAPSHOT~640fc88be4
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.804 ms) : 36507, 37102
.   : milestone, 36804,
appsec (47.019 ms) : 46598, 47441
.   : milestone, 47019,
code_origins (45.687 ms) : 45287, 46086
.   : milestone, 45687,
iast (42.611 ms) : 42247, 42975
.   : milestone, 42611,
profiling (45.169 ms) : 44779, 45560
.   : milestone, 45169,
tracing (44.046 ms) : 43667, 44424
.   : milestone, 44046,
section candidate
no_agent (36.934 ms) : 36638, 37230
.   : milestone, 36934,
appsec (48.31 ms) : 47894, 48726
.   : milestone, 48310,
code_origins (45.765 ms) : 45403, 46126
.   : milestone, 45765,
iast (42.786 ms) : 42416, 43157
.   : milestone, 42786,
profiling (47.213 ms) : 46775, 47652
.   : milestone, 47213,
tracing (43.014 ms) : 42643, 43386
.   : milestone, 43014,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.804 ms [36.507 ms, 37.102 ms] -
appsec 47.019 ms [46.598 ms, 47.441 ms] 10.215 ms (27.8%)
code_origins 45.687 ms [45.287 ms, 46.086 ms] 8.882 ms (24.1%)
iast 42.611 ms [42.247 ms, 42.975 ms] 5.807 ms (15.8%)
profiling 45.169 ms [44.779 ms, 45.56 ms] 8.365 ms (22.7%)
tracing 44.046 ms [43.667 ms, 44.424 ms] 7.241 ms (19.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.934 ms [36.638 ms, 37.23 ms] -
appsec 48.31 ms [47.894 ms, 48.726 ms] 11.376 ms (30.8%)
code_origins 45.765 ms [45.403 ms, 46.126 ms] 8.831 ms (23.9%)
iast 42.786 ms [42.416 ms, 43.157 ms] 5.852 ms (15.8%)
profiling 47.213 ms [46.775 ms, 47.652 ms] 10.279 ms (27.8%)
tracing 43.014 ms [42.643 ms, 43.386 ms] 6.08 ms (16.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master stuart.mcculloch/backport-pr-9019
git_commit_date 1751033772 1751034047
git_commit_sha 640fc88be4 e319c2b580
release_version 1.51.0-SNAPSHOT~640fc88be4 1.51.0-SNAPSHOT~e319c2b580
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1751036086 1751036086
ci_job_id 1002547605 1002547605
ci_pipeline_id 68995237 68995237
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-bpyswchx-project-304-concurrent-2-zmi04276 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-bpyswchx-project-304-concurrent-2-zmi04276 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~e319c2b580, baseline=1.51.0-SNAPSHOT~640fc88be4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.481 ms) : 1469, 1493
.   : milestone, 1481,
appsec (2.421 ms) : 2371, 2470
.   : milestone, 2421,
iast (2.206 ms) : 2144, 2269
.   : milestone, 2206,
iast_GLOBAL (2.248 ms) : 2185, 2311
.   : milestone, 2248,
profiling (2.061 ms) : 2010, 2112
.   : milestone, 2061,
tracing (2.024 ms) : 1976, 2072
.   : milestone, 2024,
section candidate
no_agent (1.481 ms) : 1469, 1492
.   : milestone, 1481,
appsec (2.424 ms) : 2375, 2474
.   : milestone, 2424,
iast (2.203 ms) : 2141, 2265
.   : milestone, 2203,
iast_GLOBAL (2.247 ms) : 2185, 2310
.   : milestone, 2247,
profiling (2.063 ms) : 2012, 2113
.   : milestone, 2063,
tracing (2.022 ms) : 1974, 2070
.   : milestone, 2022,
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.481 ms [1.469 ms, 1.493 ms] -
appsec 2.421 ms [2.371 ms, 2.47 ms] 939.664 µs (63.5%)
iast 2.206 ms [2.144 ms, 2.269 ms] 725.253 µs (49.0%)
iast_GLOBAL 2.248 ms [2.185 ms, 2.311 ms] 767.329 µs (51.8%)
profiling 2.061 ms [2.01 ms, 2.112 ms] 580.474 µs (39.2%)
tracing 2.024 ms [1.976 ms, 2.072 ms] 542.718 µs (36.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.481 ms [1.469 ms, 1.492 ms] -
appsec 2.424 ms [2.375 ms, 2.474 ms] 943.6 µs (63.7%)
iast 2.203 ms [2.141 ms, 2.265 ms] 722.407 µs (48.8%)
iast_GLOBAL 2.247 ms [2.185 ms, 2.31 ms] 766.379 µs (51.8%)
profiling 2.063 ms [2.012 ms, 2.113 ms] 581.837 µs (39.3%)
tracing 2.022 ms [1.974 ms, 2.07 ms] 541.108 µs (36.5%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~e319c2b580, baseline=1.51.0-SNAPSHOT~640fc88be4
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.993 s) : 14993000, 14993000
.   : milestone, 14993000,
appsec (15.0 s) : 15000000, 15000000
.   : milestone, 15000000,
iast (18.629 s) : 18629000, 18629000
.   : milestone, 18629000,
iast_GLOBAL (17.736 s) : 17736000, 17736000
.   : milestone, 17736000,
profiling (15.161 s) : 15161000, 15161000
.   : milestone, 15161000,
tracing (14.769 s) : 14769000, 14769000
.   : milestone, 14769000,
section candidate
no_agent (15.479 s) : 15479000, 15479000
.   : milestone, 15479000,
appsec (14.803 s) : 14803000, 14803000
.   : milestone, 14803000,
iast (19.035 s) : 19035000, 19035000
.   : milestone, 19035000,
iast_GLOBAL (17.942 s) : 17942000, 17942000
.   : milestone, 17942000,
profiling (15.347 s) : 15347000, 15347000
.   : milestone, 15347000,
tracing (14.845 s) : 14845000, 14845000
.   : milestone, 14845000,
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.993 s [14.993 s, 14.993 s] -
appsec 15.0 s [15.0 s, 15.0 s] 7.0 ms (0.0%)
iast 18.629 s [18.629 s, 18.629 s] 3.636 s (24.3%)
iast_GLOBAL 17.736 s [17.736 s, 17.736 s] 2.743 s (18.3%)
profiling 15.161 s [15.161 s, 15.161 s] 168.0 ms (1.1%)
tracing 14.769 s [14.769 s, 14.769 s] -224.0 ms (-1.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.479 s [15.479 s, 15.479 s] -
appsec 14.803 s [14.803 s, 14.803 s] -676.0 ms (-4.4%)
iast 19.035 s [19.035 s, 19.035 s] 3.556 s (23.0%)
iast_GLOBAL 17.942 s [17.942 s, 17.942 s] 2.463 s (15.9%)
profiling 15.347 s [15.347 s, 15.347 s] -132.0 ms (-0.9%)
tracing 14.845 s [14.845 s, 14.845 s] -634.0 ms (-4.1%)

pr-commenter[bot] avatar Jun 27 '25 14:06 pr-commenter[bot]