vector
vector copied to clipboard
vectordotdev
feat(kafka): make consumer group instance id configurable
This will allow us to specify member ids and run using static group membership, and furtuer mitigate rebalances.
Deploy Preview for vrl-playground canceled.
| Name | Link |
|---|---|
| Latest commit | c05d13fa269a93861459400df9f275b55f8a1839 |
| Latest deploy log | https://app.netlify.com/sites/vrl-playground/deploys/63c0464b84bea800080b04e7 |
![netlify[bot] avatar](https://avatars.githubusercontent.com/in/13473?v=4 "Published by a pub.dev verified publisher"){kind=small}
Deploy Preview for vector-project ready!
| Name | Link |
|---|---|
| Latest commit | c05d13fa269a93861459400df9f275b55f8a1839 |
| Latest deploy log | https://app.netlify.com/sites/vector-project/deploys/63c0464bc301eb000917565e |
| Deploy Preview | https://deploy-preview-15869--vector-project.netlify.app/reports/lighthouse |
| Preview on mobile | Toggle QR Code...Use your smartphone camera to open QR code link. |
To edit notification comments on pull requests, go to your Netlify site settings.
@aholmberg as far as I see, the group instance id could be already provided to the Kafka via librdkafka options in Vector config. Could you please explain, why did you decide to move it to a separate option?
Could you please explain, why did you decide to move it to a separate option?
Honestly I was just focused on making it configurable having the same ergonomics as group name and it didn't occur to me. I thought it would be nice for folks to be aware of the option without having to go down two levels of libraries to learn of its existence. It looks to me like all of these could be sent is as librdkafkaoptions. Is there a heuristic for what gets a vector config vs what is left to the reader?
I don't mind closing this if it's not wanted. No biggie.
Is there a heuristic for what gets a vector config vs what is left to the reader?
I think one test could be: is Vector setting the librdkafka option explicitly and do we want it to be configurable. In this case, the answer seems to be yes which means we should add a top-level option for it.
Regression Test Results
Run ID: 48bf5b07-e0a9-45ae-bf4c-f4e7b67a5841
Baseline: 135fe237b8564570d9198694c0a2bfb307081a6d
Comparison: b28c3a7fd1b6031ecd14ad8a369b5d47b589f1c8
Total vector CPUs: 7
Explanation
A regression test is an integrated performance test for vector in
a repeatable rig, with varying configuration for vector. What
follows is a statistical summary of a brief vector run for each
configuration across SHAs given above. The goal of these tests are to determine,
quickly, if vector performance is changed and to what degree by a
pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a
statistically significant change in their bytes_written_per_cpu_second performance
between baseline and comparison SHAs, with 90.0%
confidence OR have been detected as newly erratic. Negative values mean that
baseline is faster, positive comparison. Results that do not exhibit more than a
±5% change in mean bytes_written_per_cpu_second are
discarded. An experiment is erratic if its coefficient of variation is greater
than 0.1. The abbreviated table will be
omitted if no interesting changes are observed.
No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.
Fine details of change detection per experiment.
| experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| otlp_http_to_blackhole | 60.45KiB/CPU-s | 3.97 | 100.00% | 1.49MiB/CPU-s | 128.48KiB/CPU-s | 1.66KiB/CPU-s | 0.0 | 0.084386 | 1.55MiB/CPU-s | 115.68KiB/CPU-s | 1.49KiB/CPU-s | 0.0 | 0.073074 | False | False |
| socket_to_socket_blackhole | 492.25KiB/CPU-s | 3.67 | 100.00% | 13.1MiB/CPU-s | 515.66KiB/CPU-s | 6.65KiB/CPU-s | 0.0 | 0.038445 | 13.58MiB/CPU-s | 338.53KiB/CPU-s | 4.37KiB/CPU-s | 0.0 | 0.024346 | False | False |
| syslog_humio_logs | 317.66KiB/CPU-s | 3.5 | 100.00% | 8.87MiB/CPU-s | 380.57KiB/CPU-s | 4.91KiB/CPU-s | 0.0 | 0.041894 | 9.18MiB/CPU-s | 134.15KiB/CPU-s | 1.73KiB/CPU-s | 0.0 | 0.014268 | False | False |
| datadog_agent_remap_blackhole_acks | 958.76KiB/CPU-s | 3.07 | 100.00% | 30.5MiB/CPU-s | 1.97MiB/CPU-s | 26.0KiB/CPU-s | 0.0 | 0.064463 | 31.44MiB/CPU-s | 1.33MiB/CPU-s | 17.58KiB/CPU-s | 0.0 | 0.04227 | False | False |
| otlp_grpc_to_blackhole | 26.23KiB/CPU-s | 2.52 | 100.00% | 1.02MiB/CPU-s | 48.22KiB/CPU-s | 637.4B/CPU-s | 0.0 | 0.046305 | 1.04MiB/CPU-s | 55.12KiB/CPU-s | 728.18B/CPU-s | 0.0 | 0.051631 | False | False |
| http_to_http_acks | 103.13KiB/CPU-s | 1.93 | 95.44% | 5.21MiB/CPU-s | 2.73MiB/CPU-s | 36.07KiB/CPU-s | 0.0 | 0.52374 | 5.31MiB/CPU-s | 2.79MiB/CPU-s | 36.86KiB/CPU-s | 0.0 | 0.52506 | True | False |
| syslog_regex_logs2metric_ddmetrics | 53.21KiB/CPU-s | 1.44 | 100.00% | 3.6MiB/CPU-s | 500.28KiB/CPU-s | 6.46KiB/CPU-s | 0.0 | 0.135801 | 3.65MiB/CPU-s | 489.0KiB/CPU-s | 6.31KiB/CPU-s | 0.0 | 0.130848 | True | False |
| datadog_agent_remap_datadog_logs_acks | 373.0KiB/CPU-s | 1.1 | 100.00% | 33.25MiB/CPU-s | 1.51MiB/CPU-s | 19.95KiB/CPU-s | 0.0 | 0.045394 | 33.62MiB/CPU-s | 1.4MiB/CPU-s | 18.46KiB/CPU-s | 0.0 | 0.041563 | False | False |
| datadog_agent_remap_datadog_logs | 346.23KiB/CPU-s | 1.0 | 100.00% | 33.7MiB/CPU-s | 1.43MiB/CPU-s | 18.89KiB/CPU-s | 0.0 | 0.042431 | 34.04MiB/CPU-s | 1.26MiB/CPU-s | 16.61KiB/CPU-s | 0.0 | 0.03695 | False | False |
| syslog_log2metric_splunk_hec_metrics | 49.06KiB/CPU-s | 0.52 | 100.00% | 9.3MiB/CPU-s | 308.35KiB/CPU-s | 3.98KiB/CPU-s | 0.0 | 0.032387 | 9.34MiB/CPU-s | 310.97KiB/CPU-s | 4.02KiB/CPU-s | 0.0 | 0.032495 | False | False |
| http_text_to_http_json | 78.76KiB/CPU-s | 0.31 | 100.00% | 24.92MiB/CPU-s | 663.54KiB/CPU-s | 8.56KiB/CPU-s | 0.0 | 0.026004 | 24.99MiB/CPU-s | 673.63KiB/CPU-s | 8.69KiB/CPU-s | 0.0 | 0.026318 | False | False |
| syslog_splunk_hec_logs | 8.62KiB/CPU-s | 0.09 | 96.11% | 8.96MiB/CPU-s | 210.48KiB/CPU-s | 2.72KiB/CPU-s | 0.0 | 0.022926 | 8.97MiB/CPU-s | 245.39KiB/CPU-s | 3.17KiB/CPU-s | 0.0 | 0.026703 | False | False |
| http_to_http_json | 8.31KiB/CPU-s | 0.06 | 96.17% | 13.61MiB/CPU-s | 228.11KiB/CPU-s | 2.94KiB/CPU-s | 0.0 | 0.016361 | 13.62MiB/CPU-s | 211.32KiB/CPU-s | 2.73KiB/CPU-s | 0.0 | 0.015148 | False | False |
| enterprise_http_to_http | 8.16KiB/CPU-s | 0.06 | 95.18% | 13.62MiB/CPU-s | 283.49KiB/CPU-s | 3.66KiB/CPU-s | 0.0 | 0.020331 | 13.62MiB/CPU-s | 148.61KiB/CPU-s | 1.92KiB/CPU-s | 0.0 | 0.010652 | False | False |
| splunk_hec_to_splunk_hec_logs_noack | 5.47KiB/CPU-s | 0.04 | 83.75% | 13.62MiB/CPU-s | 234.56KiB/CPU-s | 3.03KiB/CPU-s | 0.0 | 0.016817 | 13.62MiB/CPU-s | 192.84KiB/CPU-s | 2.49KiB/CPU-s | 0.0 | 0.013821 | False | False |
| splunk_hec_indexer_ack_blackhole | 2.4KiB/CPU-s | 0.02 | 40.77% | 13.62MiB/CPU-s | 252.27KiB/CPU-s | 3.25KiB/CPU-s | 0.0 | 0.018091 | 13.62MiB/CPU-s | 239.63KiB/CPU-s | 3.09KiB/CPU-s | 0.0 | 0.017182 | False | False |
| splunk_hec_to_splunk_hec_logs_acks | 3.29KiB/CPU-s | 0.02 | 40.03% | 13.61MiB/CPU-s | 352.35KiB/CPU-s | 4.55KiB/CPU-s | 0.0 | 0.025271 | 13.62MiB/CPU-s | 334.25KiB/CPU-s | 4.31KiB/CPU-s | 0.0 | 0.023967 | False | False |
| http_to_http_noack | -47.6B/CPU-s | -0.0 | 0.66% | 13.61MiB/CPU-s | 306.0KiB/CPU-s | 3.95KiB/CPU-s | 0.0 | 0.021948 | 13.61MiB/CPU-s | 311.23KiB/CPU-s | 4.02KiB/CPU-s | 0.0 | 0.022323 | False | False |
| fluent_elasticsearch | 1.2KiB/CPU-s | 0.0 | 97.39% | 45.41MiB/CPU-s | 29.89KiB/CPU-s | 390.88B/CPU-s | 0.0 | 0.000643 | 45.41MiB/CPU-s | 29.98KiB/CPU-s | 391.96B/CPU-s | 0.0 | 0.000645 | False | False |
| file_to_blackhole | -6.9KiB/CPU-s | -0.01 | 22.81% | 54.49MiB/CPU-s | 1.21MiB/CPU-s | 15.98KiB/CPU-s | 0.0 | 0.022204 | 54.48MiB/CPU-s | 1.34MiB/CPU-s | 17.63KiB/CPU-s | 0.0 | 0.024526 | False | False |
| syslog_log2metric_humio_metrics | -14.19KiB/CPU-s | -0.23 | 99.89% | 6.12MiB/CPU-s | 190.98KiB/CPU-s | 2.47KiB/CPU-s | 0.0 | 0.030451 | 6.11MiB/CPU-s | 277.04KiB/CPU-s | 3.58KiB/CPU-s | 0.0 | 0.044272 | False | False |
| datadog_agent_remap_blackhole | -635.56KiB/CPU-s | -2.0 | 100.00% | 30.98MiB/CPU-s | 1.43MiB/CPU-s | 18.91KiB/CPU-s | 0.0 | 0.046189 | 30.36MiB/CPU-s | 1.74MiB/CPU-s | 22.94KiB/CPU-s | 0.0 | 0.057156 | False | False |
| splunk_hec_route_s3 | -340.86KiB/CPU-s | -2.82 | 100.00% | 11.81MiB/CPU-s | 534.71KiB/CPU-s | 6.9KiB/CPU-s | 0.0 | 0.044205 | 11.48MiB/CPU-s | 711.76KiB/CPU-s | 9.18KiB/CPU-s | 0.0 | 0.060548 | False | False |
| syslog_loki | -448.07KiB/CPU-s | -5.0 | 100.00% | 8.75MiB/CPU-s | 336.56KiB/CPU-s | 4.34KiB/CPU-s | 0.0 | 0.037554 | 8.31MiB/CPU-s | 449.6KiB/CPU-s | 5.8KiB/CPU-s | 0.0 | 0.052807 | False | False |
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
@aholmberg I'll happily merge this in once we have the tests passing 😄
Sorry about that. I missed that it was tripped on format. Fixed and force-pushed.
Now it looks like some doc file needs to be updated. Looking for how to do that...
Regression Test Results
Run ID: 3870c56c-9fee-48fd-a390-70ba4960863e
Baseline: 69ede64bf2654256d708f3e6edf23e2b39e134a2
Comparison: ccbb5d2bed5f8f854643059352ba495ac8e860e9
Total vector CPUs: 7
Explanation
A regression test is an integrated performance test for vector in
a repeatable rig, with varying configuration for vector. What
follows is a statistical summary of a brief vector run for each
configuration across SHAs given above. The goal of these tests are to determine,
quickly, if vector performance is changed and to what degree by a
pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a
statistically significant change in their bytes_written_per_cpu_second performance
between baseline and comparison SHAs, with 90.0%
confidence OR have been detected as newly erratic. Negative values mean that
baseline is faster, positive comparison. Results that do not exhibit more than a
±5% change in mean bytes_written_per_cpu_second are
discarded. An experiment is erratic if its coefficient of variation is greater
than 0.1. The abbreviated table will be
omitted if no interesting changes are observed.
Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:
| experiment | Δ mean | Δ mean % | confidence |
|---|---|---|---|
| http_text_to_http_json | 1.5MiB/CPU-s | 6.29 | 100.00% |
Fine details of change detection per experiment.
| experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| http_text_to_http_json | 1.5MiB/CPU-s | 6.29 | 100.00% | 23.75MiB/CPU-s | 741.92KiB/CPU-s | 9.58KiB/CPU-s | 0.0 | 0.030503 | 25.25MiB/CPU-s | 565.82KiB/CPU-s | 7.3KiB/CPU-s | 0.0 | 0.021885 | False | False |
| splunk_hec_route_s3 | 384.75KiB/CPU-s | 3.35 | 100.00% | 11.21MiB/CPU-s | 770.0KiB/CPU-s | 9.93KiB/CPU-s | 0.0 | 0.067078 | 11.58MiB/CPU-s | 569.06KiB/CPU-s | 7.35KiB/CPU-s | 0.0 | 0.047965 | False | False |
| socket_to_socket_blackhole | 401.39KiB/CPU-s | 2.97 | 100.00% | 13.2MiB/CPU-s | 401.86KiB/CPU-s | 5.19KiB/CPU-s | 0.0 | 0.029719 | 13.6MiB/CPU-s | 310.99KiB/CPU-s | 4.02KiB/CPU-s | 0.0 | 0.022336 | False | False |
| datadog_agent_remap_blackhole_acks | 750.26KiB/CPU-s | 2.37 | 100.00% | 30.88MiB/CPU-s | 1.56MiB/CPU-s | 20.62KiB/CPU-s | 0.0 | 0.050495 | 31.61MiB/CPU-s | 953.25KiB/CPU-s | 12.31KiB/CPU-s | 0.0 | 0.029447 | False | False |
| otlp_grpc_to_blackhole | 22.06KiB/CPU-s | 2.12 | 100.00% | 1.02MiB/CPU-s | 44.96KiB/CPU-s | 594.28B/CPU-s | 0.0 | 0.04314 | 1.04MiB/CPU-s | 51.87KiB/CPU-s | 685.46B/CPU-s | 0.0 | 0.048744 | False | False |
| datadog_agent_remap_datadog_logs | 643.37KiB/CPU-s | 1.86 | 100.00% | 33.82MiB/CPU-s | 1.55MiB/CPU-s | 20.42KiB/CPU-s | 0.0 | 0.045716 | 34.45MiB/CPU-s | 1.11MiB/CPU-s | 14.68KiB/CPU-s | 0.0 | 0.03227 | False | False |
| syslog_humio_logs | 125.65KiB/CPU-s | 1.37 | 100.00% | 8.99MiB/CPU-s | 392.2KiB/CPU-s | 5.06KiB/CPU-s | 0.0 | 0.042612 | 9.11MiB/CPU-s | 257.47KiB/CPU-s | 3.32KiB/CPU-s | 0.0 | 0.027597 | False | False |
| datadog_agent_remap_blackhole | 335.15KiB/CPU-s | 1.07 | 100.00% | 30.6MiB/CPU-s | 1.9MiB/CPU-s | 25.12KiB/CPU-s | 0.0 | 0.062098 | 30.92MiB/CPU-s | 958.16KiB/CPU-s | 12.36KiB/CPU-s | 0.0 | 0.030257 | False | False |
| datadog_agent_remap_datadog_logs_acks | 338.5KiB/CPU-s | 1.0 | 100.00% | 33.2MiB/CPU-s | 1.41MiB/CPU-s | 18.66KiB/CPU-s | 0.0 | 0.042543 | 33.53MiB/CPU-s | 1.51MiB/CPU-s | 19.97KiB/CPU-s | 0.0 | 0.04505 | False | False |
| otlp_http_to_blackhole | 13.73KiB/CPU-s | 0.88 | 100.00% | 1.52MiB/CPU-s | 121.91KiB/CPU-s | 1.57KiB/CPU-s | 0.0 | 0.078552 | 1.53MiB/CPU-s | 127.38KiB/CPU-s | 1.64KiB/CPU-s | 0.0 | 0.081354 | False | False |
| syslog_splunk_hec_logs | 70.32KiB/CPU-s | 0.76 | 100.00% | 9.08MiB/CPU-s | 130.12KiB/CPU-s | 1.68KiB/CPU-s | 0.0 | 0.013986 | 9.15MiB/CPU-s | 124.69KiB/CPU-s | 1.61KiB/CPU-s | 0.0 | 0.013302 | False | False |
| enterprise_http_to_http | 10.1KiB/CPU-s | 0.07 | 93.66% | 13.61MiB/CPU-s | 354.81KiB/CPU-s | 4.58KiB/CPU-s | 0.0 | 0.025457 | 13.62MiB/CPU-s | 227.96KiB/CPU-s | 2.94KiB/CPU-s | 0.0 | 0.016344 | False | False |
| splunk_hec_to_splunk_hec_logs_noack | 5.06KiB/CPU-s | 0.04 | 80.94% | 13.62MiB/CPU-s | 230.92KiB/CPU-s | 2.98KiB/CPU-s | 0.0 | 0.016556 | 13.63MiB/CPU-s | 190.82KiB/CPU-s | 2.46KiB/CPU-s | 0.0 | 0.013675 | False | False |
| splunk_hec_to_splunk_hec_logs_acks | 841.42B/CPU-s | 0.01 | 10.71% | 13.62MiB/CPU-s | 334.02KiB/CPU-s | 4.31KiB/CPU-s | 0.0 | 0.023953 | 13.62MiB/CPU-s | 334.64KiB/CPU-s | 4.32KiB/CPU-s | 0.0 | 0.023996 | False | False |
| http_to_http_noack | 297.07B/CPU-s | 0.0 | 3.86% | 13.61MiB/CPU-s | 330.47KiB/CPU-s | 4.26KiB/CPU-s | 0.0 | 0.023707 | 13.61MiB/CPU-s | 325.8KiB/CPU-s | 4.2KiB/CPU-s | 0.0 | 0.023372 | False | False |
| http_to_http_json | 151.41B/CPU-s | 0.0 | 3.06% | 13.62MiB/CPU-s | 211.75KiB/CPU-s | 2.73KiB/CPU-s | 0.0 | 0.015178 | 13.62MiB/CPU-s | 210.15KiB/CPU-s | 2.71KiB/CPU-s | 0.0 | 0.015064 | False | False |
| fluent_elasticsearch | -541.35B/CPU-s | -0.0 | 66.96% | 45.41MiB/CPU-s | 29.79KiB/CPU-s | 389.39B/CPU-s | 0.0 | 0.000641 | 45.41MiB/CPU-s | 30.37KiB/CPU-s | 397.13B/CPU-s | 0.0 | 0.000653 | False | False |
| splunk_hec_indexer_ack_blackhole | -271.45B/CPU-s | -0.0 | 4.60% | 13.62MiB/CPU-s | 252.7KiB/CPU-s | 3.26KiB/CPU-s | 0.0 | 0.018121 | 13.62MiB/CPU-s | 250.92KiB/CPU-s | 3.24KiB/CPU-s | 0.0 | 0.017994 | False | False |
| file_to_blackhole | -17.2KiB/CPU-s | -0.03 | 54.98% | 54.49MiB/CPU-s | 1.06MiB/CPU-s | 14.0KiB/CPU-s | 0.0 | 0.019454 | 54.48MiB/CPU-s | 1.36MiB/CPU-s | 17.97KiB/CPU-s | 0.0 | 0.024995 | False | False |
| syslog_log2metric_humio_metrics | -17.8KiB/CPU-s | -0.28 | 100.00% | 6.21MiB/CPU-s | 83.49KiB/CPU-s | 1.08KiB/CPU-s | 0.0 | 0.013119 | 6.2MiB/CPU-s | 216.01KiB/CPU-s | 2.79KiB/CPU-s | 0.0 | 0.03404 | False | False |
| http_to_http_acks | -34.85KiB/CPU-s | -0.64 | 50.47% | 5.29MiB/CPU-s | 2.76MiB/CPU-s | 36.52KiB/CPU-s | 0.0 | 0.522576 | 5.25MiB/CPU-s | 2.71MiB/CPU-s | 35.75KiB/CPU-s | 0.0 | 0.515027 | True | False |
| syslog_regex_logs2metric_ddmetrics | -24.32KiB/CPU-s | -0.68 | 99.53% | 3.47MiB/CPU-s | 428.55KiB/CPU-s | 5.53KiB/CPU-s | 0.0 | 0.120466 | 3.45MiB/CPU-s | 511.23KiB/CPU-s | 6.6KiB/CPU-s | 0.0 | 0.144697 | True | False |
| syslog_log2metric_splunk_hec_metrics | -223.84KiB/CPU-s | -2.35 | 100.00% | 9.3MiB/CPU-s | 401.6KiB/CPU-s | 5.18KiB/CPU-s | 0.0 | 0.04217 | 9.08MiB/CPU-s | 424.81KiB/CPU-s | 5.49KiB/CPU-s | 0.0 | 0.045681 | False | False |
| syslog_loki | -257.84KiB/CPU-s | -2.91 | 100.00% | 8.65MiB/CPU-s | 458.3KiB/CPU-s | 5.91KiB/CPU-s | 0.0 | 0.05174 | 8.4MiB/CPU-s | 312.22KiB/CPU-s | 4.03KiB/CPU-s | 0.0 | 0.036306 | False | False |
@aholmberg you can run make generate-component-docs in the root of the project. You'll need some node dependencies, which are simple to install via brew, etc.
When run locally the docs are changing more than expected. I tried pushing just the kafka cue file change.
Regression Test Results
Run ID: 693f47d2-bae0-4f06-a1c0-16d869fa6360
Baseline: 69ede64bf2654256d708f3e6edf23e2b39e134a2
Comparison: c05d13fa269a93861459400df9f275b55f8a1839
Total vector CPUs: 7
Explanation
A regression test is an integrated performance test for vector in
a repeatable rig, with varying configuration for vector. What
follows is a statistical summary of a brief vector run for each
configuration across SHAs given above. The goal of these tests are to determine,
quickly, if vector performance is changed and to what degree by a
pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a
statistically significant change in their bytes_written_per_cpu_second performance
between baseline and comparison SHAs, with 90.0%
confidence OR have been detected as newly erratic. Negative values mean that
baseline is faster, positive comparison. Results that do not exhibit more than a
±5% change in mean bytes_written_per_cpu_second are
discarded. An experiment is erratic if its coefficient of variation is greater
than 0.1. The abbreviated table will be
omitted if no interesting changes are observed.
Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:
| experiment | Δ mean | Δ mean % | confidence |
|---|---|---|---|
| syslog_log2metric_splunk_hec_metrics | -1.04MiB/CPU-s | -11.26 | 100.00% |
Fine details of change detection per experiment.
| experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| syslog_log2metric_humio_metrics | 296.92KiB/CPU-s | 4.93 | 100.00% | 5.88MiB/CPU-s | 406.92KiB/CPU-s | 5.25KiB/CPU-s | 0.0 | 0.067626 | 6.17MiB/CPU-s | 357.23KiB/CPU-s | 4.61KiB/CPU-s | 0.0 | 0.056575 | False | False |
| http_text_to_http_json | 1.07MiB/CPU-s | 4.48 | 100.00% | 23.96MiB/CPU-s | 599.59KiB/CPU-s | 7.74KiB/CPU-s | 0.0 | 0.024436 | 25.03MiB/CPU-s | 728.68KiB/CPU-s | 9.4KiB/CPU-s | 0.0 | 0.028426 | False | False |
| syslog_humio_logs | 184.08KiB/CPU-s | 1.99 | 100.00% | 9.05MiB/CPU-s | 377.21KiB/CPU-s | 4.87KiB/CPU-s | 0.0 | 0.040697 | 9.23MiB/CPU-s | 96.51KiB/CPU-s | 1.25KiB/CPU-s | 0.0 | 0.01021 | False | False |
| otlp_http_to_blackhole | 29.27KiB/CPU-s | 1.9 | 100.00% | 1.51MiB/CPU-s | 123.32KiB/CPU-s | 1.59KiB/CPU-s | 0.0 | 0.079879 | 1.54MiB/CPU-s | 122.34KiB/CPU-s | 1.58KiB/CPU-s | 0.0 | 0.07777 | False | False |
| syslog_splunk_hec_logs | 139.91KiB/CPU-s | 1.51 | 100.00% | 9.02MiB/CPU-s | 248.68KiB/CPU-s | 3.21KiB/CPU-s | 0.0 | 0.026915 | 9.16MiB/CPU-s | 193.73KiB/CPU-s | 2.5KiB/CPU-s | 0.0 | 0.020655 | False | False |
| datadog_agent_remap_blackhole | 358.8KiB/CPU-s | 1.15 | 100.00% | 30.57MiB/CPU-s | 1.81MiB/CPU-s | 23.88KiB/CPU-s | 0.0 | 0.059066 | 30.92MiB/CPU-s | 1.42MiB/CPU-s | 18.78KiB/CPU-s | 0.0 | 0.045957 | False | False |
| otlp_grpc_to_blackhole | 11.24KiB/CPU-s | 1.08 | 100.00% | 1.02MiB/CPU-s | 53.91KiB/CPU-s | 712.69B/CPU-s | 0.0 | 0.051833 | 1.03MiB/CPU-s | 61.44KiB/CPU-s | 811.79B/CPU-s | 0.0 | 0.058443 | False | False |
| datadog_agent_remap_datadog_logs_acks | 174.98KiB/CPU-s | 0.51 | 100.00% | 33.55MiB/CPU-s | 1.31MiB/CPU-s | 17.25KiB/CPU-s | 0.0 | 0.038906 | 33.72MiB/CPU-s | 1.11MiB/CPU-s | 14.61KiB/CPU-s | 0.0 | 0.032777 | False | False |
| socket_to_socket_blackhole | 44.16KiB/CPU-s | 0.32 | 100.00% | 13.35MiB/CPU-s | 311.69KiB/CPU-s | 4.02KiB/CPU-s | 0.0 | 0.022806 | 13.39MiB/CPU-s | 521.91KiB/CPU-s | 6.74KiB/CPU-s | 0.0 | 0.038065 | False | False |
| splunk_hec_to_splunk_hec_logs_noack | 7.76KiB/CPU-s | 0.06 | 94.63% | 13.62MiB/CPU-s | 247.42KiB/CPU-s | 3.19KiB/CPU-s | 0.0 | 0.017742 | 13.63MiB/CPU-s | 189.41KiB/CPU-s | 2.45KiB/CPU-s | 0.0 | 0.013575 | False | False |
| file_to_blackhole | 2.81KiB/CPU-s | 0.01 | 10.21% | 54.49MiB/CPU-s | 1.16MiB/CPU-s | 15.33KiB/CPU-s | 0.0 | 0.021292 | 54.5MiB/CPU-s | 1.18MiB/CPU-s | 15.58KiB/CPU-s | 0.0 | 0.021647 | False | False |
| http_to_http_json | -230.39B/CPU-s | -0.0 | 4.64% | 13.62MiB/CPU-s | 211.1KiB/CPU-s | 2.72KiB/CPU-s | 0.0 | 0.015132 | 13.62MiB/CPU-s | 213.13KiB/CPU-s | 2.75KiB/CPU-s | 0.0 | 0.015278 | False | False |
| fluent_elasticsearch | 113.29B/CPU-s | 0.0 | 16.22% | 45.41MiB/CPU-s | 29.89KiB/CPU-s | 390.9B/CPU-s | 0.0 | 0.000643 | 45.41MiB/CPU-s | 29.99KiB/CPU-s | 391.97B/CPU-s | 0.0 | 0.000645 | False | False |
| splunk_hec_to_splunk_hec_logs_acks | 137.24B/CPU-s | 0.0 | 1.71% | 13.62MiB/CPU-s | 342.57KiB/CPU-s | 4.42KiB/CPU-s | 0.0 | 0.024569 | 13.62MiB/CPU-s | 344.53KiB/CPU-s | 4.44KiB/CPU-s | 0.0 | 0.024709 | False | False |
| http_to_http_noack | -2.37KiB/CPU-s | -0.02 | 31.74% | 13.61MiB/CPU-s | 305.35KiB/CPU-s | 3.94KiB/CPU-s | 0.0 | 0.021901 | 13.61MiB/CPU-s | 330.65KiB/CPU-s | 4.27KiB/CPU-s | 0.0 | 0.02372 | False | False |
| enterprise_http_to_http | -2.49KiB/CPU-s | -0.02 | 57.29% | 13.62MiB/CPU-s | 148.96KiB/CPU-s | 1.92KiB/CPU-s | 0.0 | 0.010676 | 13.62MiB/CPU-s | 192.3KiB/CPU-s | 2.48KiB/CPU-s | 0.0 | 0.013785 | False | False |
| splunk_hec_indexer_ack_blackhole | -2.43KiB/CPU-s | -0.02 | 40.52% | 13.62MiB/CPU-s | 247.69KiB/CPU-s | 3.2KiB/CPU-s | 0.0 | 0.017761 | 13.62MiB/CPU-s | 254.0KiB/CPU-s | 3.28KiB/CPU-s | 0.0 | 0.018217 | False | False |
| http_to_http_acks | -60.9KiB/CPU-s | -1.13 | 76.33% | 5.28MiB/CPU-s | 2.79MiB/CPU-s | 36.89KiB/CPU-s | 0.0 | 0.528762 | 5.22MiB/CPU-s | 2.71MiB/CPU-s | 35.88KiB/CPU-s | 0.0 | 0.52001 | True | False |
| splunk_hec_route_s3 | -189.04KiB/CPU-s | -1.57 | 100.00% | 11.76MiB/CPU-s | 579.79KiB/CPU-s | 7.48KiB/CPU-s | 0.0 | 0.048144 | 11.57MiB/CPU-s | 671.56KiB/CPU-s | 8.67KiB/CPU-s | 0.0 | 0.056654 | False | False |
| datadog_agent_remap_datadog_logs | -657.34KiB/CPU-s | -1.88 | 100.00% | 34.24MiB/CPU-s | 1.16MiB/CPU-s | 15.37KiB/CPU-s | 0.0 | 0.033989 | 33.59MiB/CPU-s | 1.53MiB/CPU-s | 20.25KiB/CPU-s | 0.0 | 0.045635 | False | False |
| datadog_agent_remap_blackhole_acks | -727.58KiB/CPU-s | -2.23 | 100.00% | 31.81MiB/CPU-s | 1.25MiB/CPU-s | 16.48KiB/CPU-s | 0.0 | 0.039193 | 31.1MiB/CPU-s | 1.57MiB/CPU-s | 20.81KiB/CPU-s | 0.0 | 0.050604 | False | False |
| syslog_regex_logs2metric_ddmetrics | -104.39KiB/CPU-s | -2.75 | 100.00% | 3.71MiB/CPU-s | 515.01KiB/CPU-s | 6.65KiB/CPU-s | 0.0 | 0.13564 | 3.61MiB/CPU-s | 424.33KiB/CPU-s | 5.48KiB/CPU-s | 0.0 | 0.114918 | True | False |
| syslog_loki | -321.45KiB/CPU-s | -3.56 | 100.00% | 8.81MiB/CPU-s | 297.38KiB/CPU-s | 3.84KiB/CPU-s | 0.0 | 0.032953 | 8.5MiB/CPU-s | 302.86KiB/CPU-s | 3.91KiB/CPU-s | 0.0 | 0.0348 | False | False |
| syslog_log2metric_splunk_hec_metrics | -1.04MiB/CPU-s | -11.26 | 100.00% | 9.28MiB/CPU-s | 355.96KiB/CPU-s | 4.6KiB/CPU-s | 0.0 | 0.037465 | 8.23MiB/CPU-s | 2.58MiB/CPU-s | 34.08KiB/CPU-s | 0.0 | 0.313063 | True | False |
Circling back to this PR, I realized I was incorrect that Vector is already setting the group.instance.id option per my suggested test:
I think one test could be: is Vector setting the librdkafka option explicitly and do we want it to be configurable. In this case, the answer seems to be yes which means we should add a top-level option for it.
It doesn't look like Vector is currently setting that option. Given that, I don't think we actually need this new option but can just let users configure it via librdkafka_options."group.instance.id" = .... I'll open a new PR to drop it ahead of the next release.
We could make it clearer which options are settable as librdkafka options though. I still don't think we want to mirror all of them on the kafka components themselves and so propose continuing with the test I defined to determine which should be raised up.