Enable Data Streams without adding context into Kafka headers

[piochelepiotr]

What Does This Do

This PR allows customers to enable a limited version of Data Streams monitoring, but without passing context through headers. This can be useful is really high throughput pipelines with small message size, in which the header size would hugely impact the total throughput through Kafka.

Motivation

Additional Notes

Jira ticket: [PROJ-IDENT]

[pr-commenter[bot]]

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	piotr-wolski/allow-data-streams-without-context-propagation
git_commit_date	1712563128	1712690200
git_commit_sha	b773b0fba6	dd3a29e79c
release_version	1.33.0-SNAPSHOT~b773b0fba6	1.33.0-SNAPSHOT~dd3a29e79c

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1712693923	1712693923
ci_job_id	482350839	482350839
ci_pipeline_id	31754772	31754772
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 47 metrics, 15 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:petclinic:profiling:Remote Config	worse [+21.916µs; +79.701µs] or [+3.120%; +11.346%]	753.257µs	702.448µs

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.33.0-SNAPSHOT~dd3a29e79c, baseline=1.33.0-SNAPSHOT~b773b0fba6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.08 s) : 0, 1080118
Total [baseline] (8.564 s) : 0, 8564188
Agent [candidate] (1.075 s) : 0, 1075268
Total [candidate] (8.572 s) : 0, 8572468
section iast
Agent [baseline] (1.214 s) : 0, 1213780
Total [baseline] (9.045 s) : 0, 9044590
Agent [candidate] (1.197 s) : 0, 1196701
Total [candidate] (9.047 s) : 0, 9046605
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.213 s) : 0, 1212945
Total [baseline] (9.006 s) : 0, 9005950
Agent [candidate] (1.201 s) : 0, 1201305
Total [candidate] (9.03 s) : 0, 9029681
section iast_TELEMETRY_OFF
Agent [baseline] (1.194 s) : 0, 1194232
Total [baseline] (9.004 s) : 0, 9003972
Agent [candidate] (1.214 s) : 0, 1213936
Total [candidate] (9.039 s) : 0, 9039462

• baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.08 s	-
Agent	iast	1.214 s	133.661 ms (12.4%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.213 s	132.827 ms (12.3%)
Agent	iast_TELEMETRY_OFF	1.194 s	114.114 ms (10.6%)
Total	tracing	8.564 s	-
Total	iast	9.045 s	480.402 ms (5.6%)
Total	iast_HARDCODED_SECRET_DISABLED	9.006 s	441.762 ms (5.2%)
Total	iast_TELEMETRY_OFF	9.004 s	439.784 ms (5.1%)

• candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.075 s	-
Agent	iast	1.197 s	121.433 ms (11.3%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.201 s	126.037 ms (11.7%)
Agent	iast_TELEMETRY_OFF	1.214 s	138.668 ms (12.9%)
Total	tracing	8.572 s	-
Total	iast	9.047 s	474.137 ms (5.5%)
Total	iast_HARDCODED_SECRET_DISABLED	9.03 s	457.213 ms (5.3%)
Total	iast_TELEMETRY_OFF	9.039 s	466.994 ms (5.4%)

gantt
    title insecure-bank - break down per module: candidate=1.33.0-SNAPSHOT~dd3a29e79c, baseline=1.33.0-SNAPSHOT~b773b0fba6

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (676.993 ms) : 0, 676993
BytebuddyAgent [candidate] (672.769 ms) : 0, 672769
GlobalTracer [baseline] (310.476 ms) : 0, 310476
GlobalTracer [candidate] (310.301 ms) : 0, 310301
AppSec [baseline] (49.859 ms) : 0, 49859
AppSec [candidate] (49.506 ms) : 0, 49506
Remote Config [baseline] (677.2 µs) : 0, 677
Remote Config [candidate] (664.583 µs) : 0, 665
Telemetry [baseline] (7.627 ms) : 0, 7627
Telemetry [candidate] (7.706 ms) : 0, 7706
section iast
BytebuddyAgent [baseline] (805.304 ms) : 0, 805304
BytebuddyAgent [candidate] (792.183 ms) : 0, 792183
GlobalTracer [baseline] (291.2 ms) : 0, 291200
GlobalTracer [candidate] (288.147 ms) : 0, 288147
AppSec [baseline] (49.133 ms) : 0, 49133
AppSec [candidate] (48.848 ms) : 0, 48848
IAST [baseline] (24.654 ms) : 0, 24654
IAST [candidate] (25.326 ms) : 0, 25326
Remote Config [baseline] (579.144 µs) : 0, 579
Remote Config [candidate] (602.606 µs) : 0, 603
Telemetry [baseline] (8.128 ms) : 0, 8128
Telemetry [candidate] (7.42 ms) : 0, 7420
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (804.84 ms) : 0, 804840
BytebuddyAgent [candidate] (794.322 ms) : 0, 794322
GlobalTracer [baseline] (291.003 ms) : 0, 291003
GlobalTracer [candidate] (290.394 ms) : 0, 290394
AppSec [baseline] (50.581 ms) : 0, 50581
AppSec [candidate] (51.044 ms) : 0, 51044
IAST [baseline] (24.504 ms) : 0, 24504
IAST [candidate] (23.068 ms) : 0, 23068
Remote Config [baseline] (602.343 µs) : 0, 602
Remote Config [candidate] (586.279 µs) : 0, 586
Telemetry [baseline] (6.646 ms) : 0, 6646
Telemetry [candidate] (7.51 ms) : 0, 7510
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (790.931 ms) : 0, 790931
BytebuddyAgent [candidate] (803.656 ms) : 0, 803656
GlobalTracer [baseline] (287.503 ms) : 0, 287503
GlobalTracer [candidate] (293.049 ms) : 0, 293049
AppSec [baseline] (48.239 ms) : 0, 48239
AppSec [candidate] (49.306 ms) : 0, 49306
IAST [baseline] (24.708 ms) : 0, 24708
IAST [candidate] (25.065 ms) : 0, 25065
Remote Config [baseline] (577.444 µs) : 0, 577
Remote Config [candidate] (588.763 µs) : 0, 589
Telemetry [baseline] (7.984 ms) : 0, 7984
Telemetry [candidate] (7.375 ms) : 0, 7375

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.33.0-SNAPSHOT~dd3a29e79c, baseline=1.33.0-SNAPSHOT~b773b0fba6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.073 s) : 0, 1072707
Total [baseline] (10.448 s) : 0, 10448382
Agent [candidate] (1.08 s) : 0, 1080036
Total [candidate] (10.469 s) : 0, 10469385
section appsec
Agent [baseline] (1.203 s) : 0, 1202854
Total [baseline] (10.493 s) : 0, 10493250
Agent [candidate] (1.195 s) : 0, 1195322
Total [candidate] (10.529 s) : 0, 10528857
section iast
Agent [baseline] (1.205 s) : 0, 1205464
Total [baseline] (10.74 s) : 0, 10739739
Agent [candidate] (1.194 s) : 0, 1194450
Total [candidate] (10.767 s) : 0, 10766557
section profiling
Agent [baseline] (1.268 s) : 0, 1268125
Total [baseline] (10.596 s) : 0, 10596127
Agent [candidate] (1.287 s) : 0, 1286648
Total [candidate] (10.666 s) : 0, 10666457

• baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.073 s	-
Agent	appsec	1.203 s	130.148 ms (12.1%)
Agent	iast	1.205 s	132.757 ms (12.4%)
Agent	profiling	1.268 s	195.418 ms (18.2%)
Total	tracing	10.448 s	-
Total	appsec	10.493 s	44.867 ms (0.4%)
Total	iast	10.74 s	291.357 ms (2.8%)
Total	profiling	10.596 s	147.745 ms (1.4%)

• candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.08 s	-
Agent	appsec	1.195 s	115.286 ms (10.7%)
Agent	iast	1.194 s	114.414 ms (10.6%)
Agent	profiling	1.287 s	206.612 ms (19.1%)
Total	tracing	10.469 s	-
Total	appsec	10.529 s	59.471 ms (0.6%)
Total	iast	10.767 s	297.171 ms (2.8%)
Total	profiling	10.666 s	197.071 ms (1.9%)

gantt
    title petclinic - break down per module: candidate=1.33.0-SNAPSHOT~dd3a29e79c, baseline=1.33.0-SNAPSHOT~b773b0fba6

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (672.123 ms) : 0, 672123
BytebuddyAgent [candidate] (676.412 ms) : 0, 676412
GlobalTracer [baseline] (308.303 ms) : 0, 308303
GlobalTracer [candidate] (311.337 ms) : 0, 311337
AppSec [baseline] (49.728 ms) : 0, 49728
AppSec [candidate] (49.556 ms) : 0, 49556
Remote Config [baseline] (666.833 µs) : 0, 667
Remote Config [candidate] (665.072 µs) : 0, 665
Telemetry [baseline] (7.628 ms) : 0, 7628
Telemetry [candidate] (7.587 ms) : 0, 7587
section appsec
BytebuddyAgent [baseline] (698.456 ms) : 0, 698456
BytebuddyAgent [candidate] (694.309 ms) : 0, 694309
GlobalTracer [baseline] (292.842 ms) : 0, 292842
GlobalTracer [candidate] (291.175 ms) : 0, 291175
AppSec [baseline] (149.526 ms) : 0, 149526
AppSec [candidate] (149.296 ms) : 0, 149296
IAST [baseline] (18.843 ms) : 0, 18843
IAST [candidate] (18.835 ms) : 0, 18835
Remote Config [baseline] (602.874 µs) : 0, 603
Remote Config [candidate] (603.125 µs) : 0, 603
Telemetry [baseline] (8.004 ms) : 0, 8004
Telemetry [candidate] (6.781 ms) : 0, 6781
section iast
BytebuddyAgent [baseline] (799.23 ms) : 0, 799230
BytebuddyAgent [candidate] (791.682 ms) : 0, 791682
GlobalTracer [baseline] (289.609 ms) : 0, 289609
GlobalTracer [candidate] (287.336 ms) : 0, 287336
AppSec [baseline] (49.095 ms) : 0, 49095
AppSec [candidate] (51.324 ms) : 0, 51324
IAST [baseline] (25.135 ms) : 0, 25135
IAST [candidate] (22.029 ms) : 0, 22029
Remote Config [baseline] (581.84 µs) : 0, 582
Remote Config [candidate] (565.792 µs) : 0, 566
Telemetry [baseline] (7.282 ms) : 0, 7282
Telemetry [candidate] (7.28 ms) : 0, 7280
section profiling
BytebuddyAgent [baseline] (677.052 ms) : 0, 677052
BytebuddyAgent [candidate] (687.125 ms) : 0, 687125
GlobalTracer [baseline] (380.463 ms) : 0, 380463
GlobalTracer [candidate] (385.767 ms) : 0, 385767
AppSec [baseline] (50.185 ms) : 0, 50185
AppSec [candidate] (50.973 ms) : 0, 50973
Remote Config [baseline] (702.448 µs) : 0, 702
Remote Config [candidate] (753.257 µs) : 0, 753
Telemetry [baseline] (7.386 ms) : 0, 7386
Telemetry [candidate] (7.553 ms) : 0, 7553
ProfilingAgent [baseline] (96.286 ms) : 0, 96286
ProfilingAgent [candidate] (97.352 ms) : 0, 97352
Profiling [baseline] (96.31 ms) : 0, 96310
Profiling [candidate] (97.376 ms) : 0, 97376

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-04-09T19:50:58	2024-04-09T20:12:59
git_branch	master	piotr-wolski/allow-data-streams-without-context-propagation
git_commit_date	1712563128	1712690200
git_commit_sha	b773b0fba6	dd3a29e79c
release_version	1.33.0-SNAPSHOT~b773b0fba6	1.33.0-SNAPSHOT~dd3a29e79c
start_time	2024-04-09T19:50:44	2024-04-09T20:12:46

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1712693923	1712693923
ci_job_id	482350839	482350839
ci_pipeline_id	31754772	31754772
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.33.0-SNAPSHOT~dd3a29e79c, baseline=1.33.0-SNAPSHOT~b773b0fba6
    dateFormat X
    axisFormat %s
section baseline
no_agent (368.781 µs) : 349, 389
.   : milestone, 369,
iast (481.414 µs) : 459, 504
.   : milestone, 481,
iast_FULL (539.301 µs) : 518, 560
.   : milestone, 539,
iast_GLOBAL (500.675 µs) : 478, 523
.   : milestone, 501,
iast_HARDCODED_SECRET_DISABLED (479.803 µs) : 459, 501
.   : milestone, 480,
iast_INACTIVE (452.16 µs) : 431, 474
.   : milestone, 452,
iast_TELEMETRY_OFF (471.377 µs) : 450, 493
.   : milestone, 471,
tracing (450.338 µs) : 430, 471
.   : milestone, 450,
section candidate
no_agent (366.063 µs) : 347, 385
.   : milestone, 366,
iast (476.665 µs) : 455, 498
.   : milestone, 477,
iast_FULL (541.937 µs) : 521, 563
.   : milestone, 542,
iast_GLOBAL (500.113 µs) : 479, 521
.   : milestone, 500,
iast_HARDCODED_SECRET_DISABLED (475.447 µs) : 454, 497
.   : milestone, 475,
iast_INACTIVE (453.633 µs) : 433, 475
.   : milestone, 454,
iast_TELEMETRY_OFF (469.172 µs) : 448, 490
.   : milestone, 469,
tracing (451.003 µs) : 430, 472
.   : milestone, 451,

• baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	368.781 µs [348.56 µs, 389.003 µs]	-
iast	481.414 µs [459.149 µs, 503.678 µs]	112.632 µs (30.5%)
iast_FULL	539.301 µs [518.251 µs, 560.351 µs]	170.52 µs (46.2%)
iast_GLOBAL	500.675 µs [478.409 µs, 522.941 µs]	131.894 µs (35.8%)
iast_HARDCODED_SECRET_DISABLED	479.803 µs [458.814 µs, 500.792 µs]	111.022 µs (30.1%)
iast_INACTIVE	452.16 µs [430.665 µs, 473.655 µs]	83.379 µs (22.6%)
iast_TELEMETRY_OFF	471.377 µs [450.047 µs, 492.707 µs]	102.596 µs (27.8%)
tracing	450.338 µs [430.114 µs, 470.563 µs]	81.557 µs (22.1%)

• candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	366.063 µs [346.678 µs, 385.448 µs]	-
iast	476.665 µs [455.216 µs, 498.114 µs]	110.602 µs (30.2%)
iast_FULL	541.937 µs [520.823 µs, 563.052 µs]	175.874 µs (48.0%)
iast_GLOBAL	500.113 µs [479.268 µs, 520.959 µs]	134.05 µs (36.6%)
iast_HARDCODED_SECRET_DISABLED	475.447 µs [454.294 µs, 496.601 µs]	109.385 µs (29.9%)
iast_INACTIVE	453.633 µs [432.518 µs, 474.747 µs]	87.57 µs (23.9%)
iast_TELEMETRY_OFF	469.172 µs [448.484 µs, 489.86 µs]	103.109 µs (28.2%)
tracing	451.003 µs [430.468 µs, 471.538 µs]	84.94 µs (23.2%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.33.0-SNAPSHOT~dd3a29e79c, baseline=1.33.0-SNAPSHOT~b773b0fba6
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.341 ms) : 1321, 1362
.   : milestone, 1341,
appsec (1.731 ms) : 1707, 1754
.   : milestone, 1731,
appsec_no_iast (1.729 ms) : 1705, 1752
.   : milestone, 1729,
iast (1.479 ms) : 1456, 1502
.   : milestone, 1479,
profiling (1.519 ms) : 1493, 1545
.   : milestone, 1519,
tracing (1.484 ms) : 1459, 1509
.   : milestone, 1484,
section candidate
no_agent (1.343 ms) : 1323, 1362
.   : milestone, 1343,
appsec (1.708 ms) : 1683, 1733
.   : milestone, 1708,
appsec_no_iast (1.735 ms) : 1711, 1758
.   : milestone, 1735,
iast (1.505 ms) : 1484, 1527
.   : milestone, 1505,
profiling (1.496 ms) : 1471, 1520
.   : milestone, 1496,
tracing (1.474 ms) : 1449, 1498
.   : milestone, 1474,

• baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.341 ms [1.321 ms, 1.362 ms]	-
appsec	1.731 ms [1.707 ms, 1.754 ms]	389.427 µs (29.0%)
appsec_no_iast	1.729 ms [1.705 ms, 1.752 ms]	387.157 µs (28.9%)
iast	1.479 ms [1.456 ms, 1.502 ms]	137.745 µs (10.3%)
profiling	1.519 ms [1.493 ms, 1.545 ms]	177.314 µs (13.2%)
tracing	1.484 ms [1.459 ms, 1.509 ms]	142.574 µs (10.6%)

• candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.343 ms [1.323 ms, 1.362 ms]	-
appsec	1.708 ms [1.683 ms, 1.733 ms]	365.253 µs (27.2%)
appsec_no_iast	1.735 ms [1.711 ms, 1.758 ms]	391.921 µs (29.2%)
iast	1.505 ms [1.484 ms, 1.527 ms]	162.781 µs (12.1%)
profiling	1.496 ms [1.471 ms, 1.52 ms]	153.096 µs (11.4%)
tracing	1.474 ms [1.449 ms, 1.498 ms]	131.058 µs (9.8%)

[pr-commenter[bot]]

Kafka / producer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	piotr-wolski/allow-data-streams-without-context-propagation
git_commit_date	1712235284	1712690200
git_commit_sha	734e3c59980f8b62181f48e8eaf72b2aeb1cde51	dd3a29e79c050065c75dc6e244bf99a119897ef7

See matching parameters

	Baseline	Candidate
ci_job_date	1712691851	1712691851
ci_job_id	482350840	482350840
ci_pipeline_id	31754772	31754772
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.21	11.0.21
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.21+9-post-Ubuntu-0ubuntu122.04	11.0.21+9-post-Ubuntu-0ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce	same

[pr-commenter[bot]]

Kafka / consumer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	piotr-wolski/allow-data-streams-without-context-propagation
git_commit_date	1712235284	1712690200
git_commit_sha	734e3c59980f8b62181f48e8eaf72b2aeb1cde51	dd3a29e79c050065c75dc6e244bf99a119897ef7

See matching parameters

	Baseline	Candidate
ci_job_date	1712691932	1712691932
ci_job_id	482350841	482350841
ci_pipeline_id	31754772	31754772
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.21	11.0.21
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.21+9-post-Ubuntu-0ubuntu122.04	11.0.21+9-post-Ubuntu-0ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume	unsure [-11706.867op/s; -1094.522op/s] or [-3.721%; -0.348%]

[nayeem-kamal]

LGTM. @ygree 's suggestions are good for improving the readability of the changes.