vectordotdev
docs(prometheus_remote_write source): Use generated docs for configuration
Deploy Preview for vector-project ready!
| Name | Link |
|---|---|
| Latest commit | 2fa1b43cbb4bd2f2c079e20d0e54cfe8075bc04d |
| Latest deploy log | https://app.netlify.com/sites/vector-project/deploys/63c935b8ee1f7d00082a1b05 |
| Deploy Preview | https://deploy-preview-15985--vector-project.netlify.app/reports/lighthouse |
| Preview on mobile | Toggle QR Code...Use your smartphone camera to open QR code link. |
To edit notification comments on pull requests, go to your Netlify site settings.
![netlify[bot] avatar](https://avatars.githubusercontent.com/in/13473?v=4 "Published by a pub.dev verified publisher"){kind=small}
Deploy Preview for vrl-playground canceled.
| Name | Link |
|---|---|
| Latest commit | 2fa1b43cbb4bd2f2c079e20d0e54cfe8075bc04d |
| Latest deploy log | https://app.netlify.com/sites/vrl-playground/deploys/63c935b8001548000830915b |
Regression Test Results
Run ID: 0ee66bda-5e52-496a-b2ad-188c94c2df88
Baseline: 24e89dd8180fa430fc6ff17ec535b2fdd8c768d7
Comparison: c392bb555c1b661bbeb0b1357681a1351a0a8772
Total vector CPUs: 7
Explanation
A regression test is an integrated performance test for vector in
a repeatable rig, with varying configuration for vector. What
follows is a statistical summary of a brief vector run for each
configuration across SHAs given above. The goal of these tests are to determine,
quickly, if vector performance is changed and to what degree by a
pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a
statistically significant change in their bytes_written_per_cpu_second performance
between baseline and comparison SHAs, with 90.0%
confidence OR have been detected as newly erratic. Negative values mean that
baseline is faster, positive comparison. Results that do not exhibit more than a
±5% change in mean bytes_written_per_cpu_second are
discarded. An experiment is erratic if its coefficient of variation is greater
than 0.1. The abbreviated table will be
omitted if no interesting changes are observed.
Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:
| experiment | Δ mean | Δ mean % | confidence |
|---|---|---|---|
| syslog_regex_logs2metric_ddmetrics | 197.24KiB/CPU-s | 5.31 | 100.00% |
Fine details of change detection per experiment.
| experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| syslog_regex_logs2metric_ddmetrics | 197.24KiB/CPU-s | 5.31 | 100.00% | 3.63MiB/CPU-s | 570.87KiB/CPU-s | 7.37KiB/CPU-s | 0.0 | 0.15359 | 3.82MiB/CPU-s | 468.31KiB/CPU-s | 6.05KiB/CPU-s | 0.0 | 0.119647 | True | False |
| syslog_loki | 281.48KiB/CPU-s | 3.25 | 100.00% | 8.45MiB/CPU-s | 238.82KiB/CPU-s | 3.08KiB/CPU-s | 0.0 | 0.02761 | 8.72MiB/CPU-s | 217.86KiB/CPU-s | 2.81KiB/CPU-s | 0.0 | 0.024393 | False | False |
| syslog_log2metric_splunk_hec_metrics | 145.16KiB/CPU-s | 1.52 | 100.00% | 9.31MiB/CPU-s | 426.57KiB/CPU-s | 5.51KiB/CPU-s | 0.0 | 0.044752 | 9.45MiB/CPU-s | 407.7KiB/CPU-s | 5.26KiB/CPU-s | 0.0 | 0.04213 | False | False |
| syslog_splunk_hec_logs | 106.41KiB/CPU-s | 1.18 | 100.00% | 8.83MiB/CPU-s | 281.52KiB/CPU-s | 3.63KiB/CPU-s | 0.0 | 0.031137 | 8.93MiB/CPU-s | 226.03KiB/CPU-s | 2.92KiB/CPU-s | 0.0 | 0.024709 | False | False |
| http_text_to_http_json | 255.4KiB/CPU-s | 1.05 | 100.00% | 23.75MiB/CPU-s | 513.56KiB/CPU-s | 6.63KiB/CPU-s | 0.0 | 0.021115 | 24.0MiB/CPU-s | 539.03KiB/CPU-s | 6.95KiB/CPU-s | 0.0 | 0.021932 | False | False |
| socket_to_socket_blackhole | 96.18KiB/CPU-s | 0.74 | 100.00% | 12.72MiB/CPU-s | 409.9KiB/CPU-s | 5.29KiB/CPU-s | 0.0 | 0.031472 | 12.81MiB/CPU-s | 407.27KiB/CPU-s | 5.26KiB/CPU-s | 0.0 | 0.031041 | False | False |
| http_to_http_acks | 27.85KiB/CPU-s | 0.52 | 39.99% | 5.22MiB/CPU-s | 2.76MiB/CPU-s | 38.51KiB/CPU-s | 0.0 | 0.529188 | 5.25MiB/CPU-s | 2.77MiB/CPU-s | 36.6KiB/CPU-s | 0.0 | 0.527365 | True | False |
| otlp_grpc_to_blackhole | 2.62KiB/CPU-s | 0.25 | 99.52% | 1.02MiB/CPU-s | 40.54KiB/CPU-s | 535.94B/CPU-s | 0.0 | 0.03872 | 1.02MiB/CPU-s | 59.38KiB/CPU-s | 784.43B/CPU-s | 0.0 | 0.056574 | False | False |
| syslog_log2metric_humio_metrics | 8.27KiB/CPU-s | 0.13 | 85.67% | 6.07MiB/CPU-s | 317.43KiB/CPU-s | 4.1KiB/CPU-s | 0.0 | 0.051091 | 6.07MiB/CPU-s | 301.39KiB/CPU-s | 3.89KiB/CPU-s | 0.0 | 0.048445 | False | False |
| enterprise_http_to_http | 8.38KiB/CPU-s | 0.06 | 95.65% | 13.62MiB/CPU-s | 285.16KiB/CPU-s | 3.68KiB/CPU-s | 0.0 | 0.020451 | 13.62MiB/CPU-s | 148.24KiB/CPU-s | 1.91KiB/CPU-s | 0.0 | 0.010625 | False | False |
| splunk_hec_to_splunk_hec_logs_noack | 5.12KiB/CPU-s | 0.04 | 81.68% | 13.62MiB/CPU-s | 227.62KiB/CPU-s | 2.94KiB/CPU-s | 0.0 | 0.016319 | 13.63MiB/CPU-s | 191.9KiB/CPU-s | 2.48KiB/CPU-s | 0.0 | 0.013753 | False | False |
| http_to_http_noack | -115.93B/CPU-s | -0.0 | 1.61% | 13.61MiB/CPU-s | 304.48KiB/CPU-s | 3.93KiB/CPU-s | 0.0 | 0.021839 | 13.61MiB/CPU-s | 308.79KiB/CPU-s | 3.98KiB/CPU-s | 0.0 | 0.022149 | False | False |
| fluent_elasticsearch | 163.88B/CPU-s | 0.0 | 23.19% | 45.41MiB/CPU-s | 29.68KiB/CPU-s | 388.18B/CPU-s | 0.0 | 0.000638 | 45.41MiB/CPU-s | 30.43KiB/CPU-s | 397.74B/CPU-s | 0.0 | 0.000654 | False | False |
| splunk_hec_indexer_ack_blackhole | -39.63B/CPU-s | -0.0 | 0.67% | 13.62MiB/CPU-s | 253.53KiB/CPU-s | 3.27KiB/CPU-s | 0.0 | 0.018181 | 13.62MiB/CPU-s | 253.85KiB/CPU-s | 3.27KiB/CPU-s | 0.0 | 0.018204 | False | False |
| splunk_hec_to_splunk_hec_logs_acks | -609.28B/CPU-s | -0.0 | 7.45% | 13.62MiB/CPU-s | 348.42KiB/CPU-s | 4.5KiB/CPU-s | 0.0 | 0.024988 | 13.61MiB/CPU-s | 349.35KiB/CPU-s | 4.51KiB/CPU-s | 0.0 | 0.025056 | False | False |
| http_to_http_json | -227.2B/CPU-s | -0.0 | 4.60% | 13.62MiB/CPU-s | 210.24KiB/CPU-s | 2.71KiB/CPU-s | 0.0 | 0.01507 | 13.62MiB/CPU-s | 211.73KiB/CPU-s | 2.73KiB/CPU-s | 0.0 | 0.015177 | False | False |
| file_to_blackhole | -9.55KiB/CPU-s | -0.02 | 35.89% | 54.5MiB/CPU-s | 1.07MiB/CPU-s | 14.16KiB/CPU-s | 0.0 | 0.019665 | 54.49MiB/CPU-s | 1.12MiB/CPU-s | 14.79KiB/CPU-s | 0.0 | 0.02057 | False | False |
| datadog_agent_remap_datadog_logs | -36.81KiB/CPU-s | -0.1 | 91.78% | 34.32MiB/CPU-s | 1.21MiB/CPU-s | 16.03KiB/CPU-s | 0.0 | 0.035359 | 34.29MiB/CPU-s | 1.05MiB/CPU-s | 13.84KiB/CPU-s | 0.0 | 0.030553 | False | False |
| otlp_http_to_blackhole | -2.91KiB/CPU-s | -0.19 | 80.60% | 1.51MiB/CPU-s | 116.72KiB/CPU-s | 1.51KiB/CPU-s | 0.0 | 0.075527 | 1.51MiB/CPU-s | 128.29KiB/CPU-s | 1.65KiB/CPU-s | 0.0 | 0.083169 | False | False |
| datadog_agent_remap_blackhole_acks | -217.85KiB/CPU-s | -0.66 | 100.00% | 32.01MiB/CPU-s | 350.81KiB/CPU-s | 4.53KiB/CPU-s | 0.0 | 0.010701 | 31.8MiB/CPU-s | 737.22KiB/CPU-s | 9.51KiB/CPU-s | 0.0 | 0.022639 | False | False |
| datadog_agent_remap_blackhole | -296.17KiB/CPU-s | -0.92 | 100.00% | 31.52MiB/CPU-s | 1.06MiB/CPU-s | 14.01KiB/CPU-s | 0.0 | 0.033637 | 31.23MiB/CPU-s | 1.64MiB/CPU-s | 21.65KiB/CPU-s | 0.0 | 0.05246 | False | False |
| syslog_humio_logs | -94.35KiB/CPU-s | -1.02 | 100.00% | 9.06MiB/CPU-s | 231.54KiB/CPU-s | 2.99KiB/CPU-s | 0.0 | 0.024947 | 8.97MiB/CPU-s | 314.78KiB/CPU-s | 4.06KiB/CPU-s | 0.0 | 0.034265 | False | False |
| splunk_hec_route_s3 | -179.83KiB/CPU-s | -1.48 | 100.00% | 11.85MiB/CPU-s | 534.39KiB/CPU-s | 6.89KiB/CPU-s | 0.0 | 0.044026 | 11.68MiB/CPU-s | 569.09KiB/CPU-s | 7.34KiB/CPU-s | 0.0 | 0.047589 | False | False |
| datadog_agent_remap_datadog_logs_acks | -958.58KiB/CPU-s | -2.8 | 100.00% | 33.44MiB/CPU-s | 1.3MiB/CPU-s | 17.16KiB/CPU-s | 0.0 | 0.038849 | 32.51MiB/CPU-s | 1.61MiB/CPU-s | 21.22KiB/CPU-s | 0.0 | 0.049409 | False | False |
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Regression Test Results
Run ID: debc1e30-3b30-4c97-a2e6-338f118e82ff
Baseline: 964919a17e57e0b57d446826425ce03144dad5e1
Comparison: b397581dccb9e46ad747514f0de8e91c1c5b78db
Total vector CPUs: 7
Explanation
A regression test is an integrated performance test for vector in
a repeatable rig, with varying configuration for vector. What
follows is a statistical summary of a brief vector run for each
configuration across SHAs given above. The goal of these tests are to determine,
quickly, if vector performance is changed and to what degree by a
pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a
statistically significant change in their bytes_written_per_cpu_second performance
between baseline and comparison SHAs, with 90.0%
confidence OR have been detected as newly erratic. Negative values mean that
baseline is faster, positive comparison. Results that do not exhibit more than a
±5% change in mean bytes_written_per_cpu_second are
discarded. An experiment is erratic if its coefficient of variation is greater
than 0.1. The abbreviated table will be
omitted if no interesting changes are observed.
No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.
Fine details of change detection per experiment.
| experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| syslog_log2metric_humio_metrics | 251.73KiB/CPU-s | 4.15 | 100.00% | 5.92MiB/CPU-s | 260.16KiB/CPU-s | 3.36KiB/CPU-s | 0.0 | 0.042907 | 6.17MiB/CPU-s | 163.33KiB/CPU-s | 2.11KiB/CPU-s | 0.0 | 0.025863 | False | False |
| syslog_log2metric_splunk_hec_metrics | 153.19KiB/CPU-s | 1.61 | 100.00% | 9.29MiB/CPU-s | 273.2KiB/CPU-s | 3.53KiB/CPU-s | 0.0 | 0.02872 | 9.44MiB/CPU-s | 161.24KiB/CPU-s | 2.08KiB/CPU-s | 0.0 | 0.016682 | False | False |
| http_to_http_noack | 8.26KiB/CPU-s | 0.06 | 82.83% | 13.61MiB/CPU-s | 373.4KiB/CPU-s | 4.82KiB/CPU-s | 0.0 | 0.026795 | 13.62MiB/CPU-s | 282.95KiB/CPU-s | 3.65KiB/CPU-s | 0.0 | 0.020292 | False | False |
| enterprise_http_to_http | 8.29KiB/CPU-s | 0.06 | 89.07% | 13.61MiB/CPU-s | 332.16KiB/CPU-s | 4.29KiB/CPU-s | 0.0 | 0.023829 | 13.62MiB/CPU-s | 225.26KiB/CPU-s | 2.91KiB/CPU-s | 0.0 | 0.01615 | False | False |
| splunk_hec_to_splunk_hec_logs_noack | 4.49KiB/CPU-s | 0.03 | 75.86% | 13.62MiB/CPU-s | 227.76KiB/CPU-s | 2.94KiB/CPU-s | 0.0 | 0.016328 | 13.63MiB/CPU-s | 190.22KiB/CPU-s | 2.46KiB/CPU-s | 0.0 | 0.013632 | False | False |
| splunk_hec_to_splunk_hec_logs_acks | 906.05B/CPU-s | 0.01 | 11.42% | 13.62MiB/CPU-s | 349.51KiB/CPU-s | 4.51KiB/CPU-s | 0.0 | 0.025062 | 13.62MiB/CPU-s | 325.0KiB/CPU-s | 4.19KiB/CPU-s | 0.0 | 0.023303 | False | False |
| http_to_http_json | 800.66B/CPU-s | 0.01 | 16.07% | 13.62MiB/CPU-s | 212.53KiB/CPU-s | 2.74KiB/CPU-s | 0.0 | 0.015235 | 13.62MiB/CPU-s | 210.02KiB/CPU-s | 2.71KiB/CPU-s | 0.0 | 0.015054 | False | False |
| splunk_hec_indexer_ack_blackhole | -403.96B/CPU-s | -0.0 | 7.06% | 13.62MiB/CPU-s | 246.34KiB/CPU-s | 3.18KiB/CPU-s | 0.0 | 0.017663 | 13.62MiB/CPU-s | 241.87KiB/CPU-s | 3.12KiB/CPU-s | 0.0 | 0.017343 | False | False |
| fluent_elasticsearch | -501.22B/CPU-s | -0.0 | 63.18% | 45.41MiB/CPU-s | 30.0KiB/CPU-s | 392.27B/CPU-s | 0.0 | 0.000645 | 45.41MiB/CPU-s | 30.26KiB/CPU-s | 395.39B/CPU-s | 0.0 | 0.000651 | False | False |
| file_to_blackhole | -14.28KiB/CPU-s | -0.03 | 50.49% | 54.49MiB/CPU-s | 1.08MiB/CPU-s | 14.28KiB/CPU-s | 0.0 | 0.019831 | 54.48MiB/CPU-s | 1.16MiB/CPU-s | 15.31KiB/CPU-s | 0.0 | 0.021285 | False | False |
| syslog_splunk_hec_logs | -5.56KiB/CPU-s | -0.06 | 94.35% | 9.06MiB/CPU-s | 130.87KiB/CPU-s | 1.69KiB/CPU-s | 0.0 | 0.014113 | 9.05MiB/CPU-s | 184.11KiB/CPU-s | 2.38KiB/CPU-s | 0.0 | 0.019866 | False | False |
| http_to_http_acks | -4.59KiB/CPU-s | -0.08 | 6.95% | 5.3MiB/CPU-s | 2.84MiB/CPU-s | 37.48KiB/CPU-s | 0.0 | 0.534787 | 5.3MiB/CPU-s | 2.79MiB/CPU-s | 36.85KiB/CPU-s | 0.0 | 0.526228 | True | False |
| splunk_hec_route_s3 | -15.15KiB/CPU-s | -0.13 | 86.54% | 11.7MiB/CPU-s | 584.26KiB/CPU-s | 7.54KiB/CPU-s | 0.0 | 0.048749 | 11.69MiB/CPU-s | 523.67KiB/CPU-s | 6.76KiB/CPU-s | 0.0 | 0.043749 | False | False |
| datadog_agent_remap_datadog_logs | -63.23KiB/CPU-s | -0.18 | 99.67% | 33.92MiB/CPU-s | 1.14MiB/CPU-s | 15.1KiB/CPU-s | 0.0 | 0.033699 | 33.86MiB/CPU-s | 1.16MiB/CPU-s | 15.31KiB/CPU-s | 0.0 | 0.034225 | False | False |
| datadog_agent_remap_datadog_logs_acks | -80.08KiB/CPU-s | -0.23 | 99.95% | 33.57MiB/CPU-s | 1.33MiB/CPU-s | 17.51KiB/CPU-s | 0.0 | 0.039488 | 33.49MiB/CPU-s | 1.13MiB/CPU-s | 14.99KiB/CPU-s | 0.0 | 0.033864 | False | False |
| syslog_regex_logs2metric_ddmetrics | -9.65KiB/CPU-s | -0.25 | 79.40% | 3.71MiB/CPU-s | 415.53KiB/CPU-s | 5.36KiB/CPU-s | 0.0 | 0.10927 | 3.7MiB/CPU-s | 419.93KiB/CPU-s | 5.42KiB/CPU-s | 0.0 | 0.110707 | True | False |
| syslog_humio_logs | -35.51KiB/CPU-s | -0.38 | 100.00% | 9.18MiB/CPU-s | 156.29KiB/CPU-s | 2.02KiB/CPU-s | 0.0 | 0.016616 | 9.15MiB/CPU-s | 125.58KiB/CPU-s | 1.62KiB/CPU-s | 0.0 | 0.013401 | False | False |
| datadog_agent_remap_blackhole | -153.31KiB/CPU-s | -0.48 | 100.00% | 30.91MiB/CPU-s | 1.18MiB/CPU-s | 15.62KiB/CPU-s | 0.0 | 0.038216 | 30.76MiB/CPU-s | 1.07MiB/CPU-s | 14.21KiB/CPU-s | 0.0 | 0.034946 | False | False |
| syslog_loki | -72.26KiB/CPU-s | -0.8 | 100.00% | 8.84MiB/CPU-s | 241.28KiB/CPU-s | 3.11KiB/CPU-s | 0.0 | 0.026666 | 8.76MiB/CPU-s | 252.24KiB/CPU-s | 3.26KiB/CPU-s | 0.0 | 0.028102 | False | False |
| otlp_grpc_to_blackhole | -16.49KiB/CPU-s | -1.53 | 100.00% | 1.06MiB/CPU-s | 38.54KiB/CPU-s | 509.45B/CPU-s | 0.0 | 0.035658 | 1.04MiB/CPU-s | 46.49KiB/CPU-s | 614.38B/CPU-s | 0.0 | 0.043676 | False | False |
| otlp_http_to_blackhole | -30.51KiB/CPU-s | -1.92 | 100.00% | 1.55MiB/CPU-s | 100.89KiB/CPU-s | 1.3KiB/CPU-s | 0.0 | 0.06344 | 1.52MiB/CPU-s | 116.42KiB/CPU-s | 1.5KiB/CPU-s | 0.0 | 0.074636 | False | False |
| socket_to_socket_blackhole | -286.88KiB/CPU-s | -2.09 | 100.00% | 13.43MiB/CPU-s | 142.22KiB/CPU-s | 1.84KiB/CPU-s | 0.0 | 0.010342 | 13.15MiB/CPU-s | 119.53KiB/CPU-s | 1.54KiB/CPU-s | 0.0 | 0.008877 | False | False |
| datadog_agent_remap_blackhole_acks | -859.82KiB/CPU-s | -2.64 | 100.00% | 31.85MiB/CPU-s | 835.04KiB/CPU-s | 10.78KiB/CPU-s | 0.0 | 0.025599 | 31.01MiB/CPU-s | 1.23MiB/CPU-s | 16.28KiB/CPU-s | 0.0 | 0.039695 | False | False |
| http_text_to_http_json | -683.3KiB/CPU-s | -2.73 | 100.00% | 24.41MiB/CPU-s | 559.36KiB/CPU-s | 7.22KiB/CPU-s | 0.0 | 0.022378 | 23.74MiB/CPU-s | 537.56KiB/CPU-s | 6.94KiB/CPU-s | 0.0 | 0.02211 | False | False |
Regression Test Results
Run ID: 19d2c150-4054-42b7-acc5-93745383c2d3
Baseline: 964919a17e57e0b57d446826425ce03144dad5e1
Comparison: 2fa1b43cbb4bd2f2c079e20d0e54cfe8075bc04d
Total vector CPUs: 7
Explanation
A regression test is an integrated performance test for vector in
a repeatable rig, with varying configuration for vector. What
follows is a statistical summary of a brief vector run for each
configuration across SHAs given above. The goal of these tests are to determine,
quickly, if vector performance is changed and to what degree by a
pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a
statistically significant change in their bytes_written_per_cpu_second performance
between baseline and comparison SHAs, with 90.0%
confidence OR have been detected as newly erratic. Negative values mean that
baseline is faster, positive comparison. Results that do not exhibit more than a
±5% change in mean bytes_written_per_cpu_second are
discarded. An experiment is erratic if its coefficient of variation is greater
than 0.1. The abbreviated table will be
omitted if no interesting changes are observed.
No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.
Fine details of change detection per experiment.
| experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| syslog_log2metric_humio_metrics | 176.81KiB/CPU-s | 2.91 | 100.00% | 5.94MiB/CPU-s | 268.75KiB/CPU-s | 3.47KiB/CPU-s | 0.0 | 0.044197 | 6.11MiB/CPU-s | 197.11KiB/CPU-s | 2.54KiB/CPU-s | 0.0 | 0.031499 | False | False |
| http_to_http_acks | 36.39KiB/CPU-s | 0.68 | 51.87% | 5.24MiB/CPU-s | 2.76MiB/CPU-s | 36.49KiB/CPU-s | 0.0 | 0.52672 | 5.28MiB/CPU-s | 2.77MiB/CPU-s | 36.59KiB/CPU-s | 0.0 | 0.524403 | True | False |
| syslog_splunk_hec_logs | 34.78KiB/CPU-s | 0.37 | 100.00% | 9.07MiB/CPU-s | 163.43KiB/CPU-s | 2.11KiB/CPU-s | 0.0 | 0.017596 | 9.1MiB/CPU-s | 172.07KiB/CPU-s | 2.22KiB/CPU-s | 0.0 | 0.018457 | False | False |
| syslog_humio_logs | 34.25KiB/CPU-s | 0.36 | 100.00% | 9.18MiB/CPU-s | 156.73KiB/CPU-s | 2.02KiB/CPU-s | 0.0 | 0.016666 | 9.22MiB/CPU-s | 104.02KiB/CPU-s | 1.34KiB/CPU-s | 0.0 | 0.011021 | False | False |
| http_to_http_json | 11.67KiB/CPU-s | 0.08 | 98.67% | 13.59MiB/CPU-s | 268.89KiB/CPU-s | 3.47KiB/CPU-s | 0.0 | 0.019322 | 13.6MiB/CPU-s | 247.13KiB/CPU-s | 3.19KiB/CPU-s | 0.0 | 0.017744 | False | False |
| http_to_http_noack | 10.26KiB/CPU-s | 0.07 | 94.55% | 13.61MiB/CPU-s | 348.19KiB/CPU-s | 4.49KiB/CPU-s | 0.0 | 0.024982 | 13.62MiB/CPU-s | 223.15KiB/CPU-s | 2.88KiB/CPU-s | 0.0 | 0.015999 | False | False |
| splunk_hec_to_splunk_hec_logs_noack | 3.32KiB/CPU-s | 0.02 | 62.60% | 13.62MiB/CPU-s | 216.82KiB/CPU-s | 2.8KiB/CPU-s | 0.0 | 0.015542 | 13.63MiB/CPU-s | 192.12KiB/CPU-s | 2.48KiB/CPU-s | 0.0 | 0.013769 | False | False |
| splunk_hec_to_splunk_hec_logs_acks | 857.75B/CPU-s | 0.01 | 10.06% | 13.61MiB/CPU-s | 360.67KiB/CPU-s | 4.65KiB/CPU-s | 0.0 | 0.025869 | 13.62MiB/CPU-s | 365.84KiB/CPU-s | 4.72KiB/CPU-s | 0.0 | 0.026238 | False | False |
| enterprise_http_to_http | 1.1KiB/CPU-s | 0.01 | 20.96% | 13.62MiB/CPU-s | 258.29KiB/CPU-s | 3.33KiB/CPU-s | 0.0 | 0.018517 | 13.62MiB/CPU-s | 191.78KiB/CPU-s | 2.48KiB/CPU-s | 0.0 | 0.013747 | False | False |
| file_to_blackhole | -1.35KiB/CPU-s | -0.0 | 5.20% | 54.5MiB/CPU-s | 1.13MiB/CPU-s | 14.93KiB/CPU-s | 0.0 | 0.020734 | 54.49MiB/CPU-s | 1.09MiB/CPU-s | 14.36KiB/CPU-s | 0.0 | 0.019958 | False | False |
| fluent_elasticsearch | -37.9B/CPU-s | -0.0 | 5.43% | 45.41MiB/CPU-s | 30.18KiB/CPU-s | 394.39B/CPU-s | 0.0 | 0.000649 | 45.41MiB/CPU-s | 30.03KiB/CPU-s | 392.33B/CPU-s | 0.0 | 0.000646 | False | False |
| splunk_hec_indexer_ack_blackhole | -1.28KiB/CPU-s | -0.01 | 21.76% | 13.62MiB/CPU-s | 251.79KiB/CPU-s | 3.25KiB/CPU-s | 0.0 | 0.018056 | 13.62MiB/CPU-s | 257.93KiB/CPU-s | 3.33KiB/CPU-s | 0.0 | 0.018497 | False | False |
| datadog_agent_remap_datadog_logs | -179.62KiB/CPU-s | -0.52 | 100.00% | 33.66MiB/CPU-s | 1.39MiB/CPU-s | 18.33KiB/CPU-s | 0.0 | 0.041236 | 33.48MiB/CPU-s | 1.47MiB/CPU-s | 19.41KiB/CPU-s | 0.0 | 0.043897 | False | False |
| syslog_log2metric_splunk_hec_metrics | -102.0KiB/CPU-s | -1.05 | 100.00% | 9.46MiB/CPU-s | 202.3KiB/CPU-s | 2.61KiB/CPU-s | 0.0 | 0.020874 | 9.36MiB/CPU-s | 227.68KiB/CPU-s | 2.94KiB/CPU-s | 0.0 | 0.023743 | False | False |
| datadog_agent_remap_blackhole_acks | -370.59KiB/CPU-s | -1.14 | 100.00% | 31.8MiB/CPU-s | 663.66KiB/CPU-s | 8.57KiB/CPU-s | 0.0 | 0.02038 | 31.44MiB/CPU-s | 692.72KiB/CPU-s | 8.95KiB/CPU-s | 0.0 | 0.021517 | False | False |
| datadog_agent_remap_blackhole | -371.24KiB/CPU-s | -1.17 | 100.00% | 30.91MiB/CPU-s | 1.42MiB/CPU-s | 18.81KiB/CPU-s | 0.0 | 0.046029 | 30.54MiB/CPU-s | 1.37MiB/CPU-s | 18.09KiB/CPU-s | 0.0 | 0.044809 | False | False |
| datadog_agent_remap_datadog_logs_acks | -462.39KiB/CPU-s | -1.33 | 100.00% | 34.06MiB/CPU-s | 914.72KiB/CPU-s | 11.8KiB/CPU-s | 0.0 | 0.026223 | 33.61MiB/CPU-s | 1006.27KiB/CPU-s | 12.99KiB/CPU-s | 0.0 | 0.029235 | False | False |
| socket_to_socket_blackhole | -202.27KiB/CPU-s | -1.47 | 100.00% | 13.45MiB/CPU-s | 132.44KiB/CPU-s | 1.71KiB/CPU-s | 0.0 | 0.009616 | 13.25MiB/CPU-s | 106.84KiB/CPU-s | 1.38KiB/CPU-s | 0.0 | 0.007873 | False | False |
| otlp_grpc_to_blackhole | -15.84KiB/CPU-s | -1.47 | 100.00% | 1.06MiB/CPU-s | 42.3KiB/CPU-s | 559.02B/CPU-s | 0.0 | 0.039144 | 1.04MiB/CPU-s | 52.98KiB/CPU-s | 699.93B/CPU-s | 0.0 | 0.049757 | False | False |
| splunk_hec_route_s3 | -212.6KiB/CPU-s | -1.78 | 100.00% | 11.69MiB/CPU-s | 642.62KiB/CPU-s | 8.29KiB/CPU-s | 0.0 | 0.053693 | 11.48MiB/CPU-s | 706.13KiB/CPU-s | 9.11KiB/CPU-s | 0.0 | 0.060067 | False | False |
| http_text_to_http_json | -523.9KiB/CPU-s | -2.11 | 100.00% | 24.23MiB/CPU-s | 609.21KiB/CPU-s | 7.86KiB/CPU-s | 0.0 | 0.024548 | 23.72MiB/CPU-s | 583.13KiB/CPU-s | 7.52KiB/CPU-s | 0.0 | 0.024005 | False | False |
| otlp_http_to_blackhole | -39.7KiB/CPU-s | -2.5 | 100.00% | 1.55MiB/CPU-s | 110.23KiB/CPU-s | 1.42KiB/CPU-s | 0.0 | 0.069404 | 1.51MiB/CPU-s | 124.84KiB/CPU-s | 1.61KiB/CPU-s | 0.0 | 0.08062 | False | False |
| syslog_loki | -234.96KiB/CPU-s | -2.62 | 100.00% | 8.76MiB/CPU-s | 274.77KiB/CPU-s | 3.55KiB/CPU-s | 0.0 | 0.03063 | 8.53MiB/CPU-s | 429.68KiB/CPU-s | 5.55KiB/CPU-s | 0.0 | 0.049185 | False | False |
| syslog_regex_logs2metric_ddmetrics | -127.86KiB/CPU-s | -3.41 | 100.00% | 3.67MiB/CPU-s | 395.7KiB/CPU-s | 5.11KiB/CPU-s | 0.0 | 0.105373 | 3.54MiB/CPU-s | 361.33KiB/CPU-s | 4.66KiB/CPU-s | 0.0 | 0.099612 | True | False |
Regression Test Results
Run ID: 9deda165-96b5-4435-bf9a-a3c8bda1b613
Baseline: 964919a17e57e0b57d446826425ce03144dad5e1
Comparison: 91a46a59c7eb1c40fbbd4be93031e45ae3cbf7fe
Total vector CPUs: 7
Explanation
A regression test is an integrated performance test for vector in
a repeatable rig, with varying configuration for vector. What
follows is a statistical summary of a brief vector run for each
configuration across SHAs given above. The goal of these tests are to determine,
quickly, if vector performance is changed and to what degree by a
pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a
statistically significant change in their bytes_written_per_cpu_second performance
between baseline and comparison SHAs, with 90.0%
confidence OR have been detected as newly erratic. Negative values mean that
baseline is faster, positive comparison. Results that do not exhibit more than a
±5% change in mean bytes_written_per_cpu_second are
discarded. An experiment is erratic if its coefficient of variation is greater
than 0.1. The abbreviated table will be
omitted if no interesting changes are observed.
Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:
| experiment | Δ mean | Δ mean % | confidence |
|---|---|---|---|
| splunk_hec_to_splunk_hec_logs_acks | 1.23MiB/CPU-s | 9.95 | 100.00% |
Fine details of change detection per experiment.
| experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| splunk_hec_to_splunk_hec_logs_acks | 1.23MiB/CPU-s | 9.95 | 100.00% | 12.38MiB/CPU-s | 3.72MiB/CPU-s | 49.1KiB/CPU-s | 0.0 | 0.300135 | 13.61MiB/CPU-s | 375.94KiB/CPU-s | 4.85KiB/CPU-s | 0.0 | 0.026963 | True | False |
| datadog_agent_remap_blackhole | 957.25KiB/CPU-s | 3.06 | 100.00% | 30.51MiB/CPU-s | 1.85MiB/CPU-s | 24.49KiB/CPU-s | 0.0 | 0.060724 | 31.45MiB/CPU-s | 860.21KiB/CPU-s | 11.1KiB/CPU-s | 0.0 | 0.026711 | False | False |
| datadog_agent_remap_datadog_logs | 543.02KiB/CPU-s | 1.58 | 100.00% | 33.52MiB/CPU-s | 1.63MiB/CPU-s | 21.5KiB/CPU-s | 0.0 | 0.048541 | 34.05MiB/CPU-s | 1.14MiB/CPU-s | 15.0KiB/CPU-s | 0.0 | 0.033349 | False | False |
| syslog_log2metric_splunk_hec_metrics | 102.28KiB/CPU-s | 1.08 | 100.00% | 9.25MiB/CPU-s | 298.55KiB/CPU-s | 3.85KiB/CPU-s | 0.0 | 0.031517 | 9.35MiB/CPU-s | 255.13KiB/CPU-s | 3.29KiB/CPU-s | 0.0 | 0.026645 | False | False |
| syslog_regex_logs2metric_ddmetrics | 39.89KiB/CPU-s | 1.07 | 100.00% | 3.65MiB/CPU-s | 411.26KiB/CPU-s | 5.31KiB/CPU-s | 0.0 | 0.109917 | 3.69MiB/CPU-s | 408.83KiB/CPU-s | 5.28KiB/CPU-s | 0.0 | 0.108114 | True | False |
| http_to_http_json | 36.83KiB/CPU-s | 0.26 | 100.00% | 13.59MiB/CPU-s | 266.06KiB/CPU-s | 3.43KiB/CPU-s | 0.0 | 0.019123 | 13.62MiB/CPU-s | 211.63KiB/CPU-s | 2.73KiB/CPU-s | 0.0 | 0.01517 | False | False |
| http_to_http_acks | 3.98KiB/CPU-s | 0.07 | 6.15% | 5.28MiB/CPU-s | 2.71MiB/CPU-s | 35.88KiB/CPU-s | 0.0 | 0.51455 | 5.28MiB/CPU-s | 2.8MiB/CPU-s | 36.99KiB/CPU-s | 0.0 | 0.53012 | True | False |
| splunk_hec_to_splunk_hec_logs_noack | 7.88KiB/CPU-s | 0.06 | 94.96% | 13.62MiB/CPU-s | 246.3KiB/CPU-s | 3.18KiB/CPU-s | 0.0 | 0.017661 | 13.63MiB/CPU-s | 191.29KiB/CPU-s | 2.47KiB/CPU-s | 0.0 | 0.013709 | False | False |
| splunk_hec_indexer_ack_blackhole | 2.2KiB/CPU-s | 0.02 | 37.81% | 13.62MiB/CPU-s | 249.14KiB/CPU-s | 3.21KiB/CPU-s | 0.0 | 0.017867 | 13.62MiB/CPU-s | 239.98KiB/CPU-s | 3.1KiB/CPU-s | 0.0 | 0.017207 | False | False |
| file_to_blackhole | 4.89KiB/CPU-s | 0.01 | 19.39% | 54.49MiB/CPU-s | 1.03MiB/CPU-s | 13.59KiB/CPU-s | 0.0 | 0.018886 | 54.49MiB/CPU-s | 1.1MiB/CPU-s | 14.54KiB/CPU-s | 0.0 | 0.020203 | False | False |
| fluent_elasticsearch | 105.24B/CPU-s | 0.0 | 15.16% | 45.41MiB/CPU-s | 29.53KiB/CPU-s | 385.74B/CPU-s | 0.0 | 0.000635 | 45.41MiB/CPU-s | 30.07KiB/CPU-s | 392.89B/CPU-s | 0.0 | 0.000647 | False | False |
| http_to_http_noack | 463.47B/CPU-s | 0.0 | 5.75% | 13.61MiB/CPU-s | 347.14KiB/CPU-s | 4.48KiB/CPU-s | 0.0 | 0.024906 | 13.61MiB/CPU-s | 341.22KiB/CPU-s | 4.4KiB/CPU-s | 0.0 | 0.024481 | False | False |
| enterprise_http_to_http | -386.51B/CPU-s | -0.0 | 8.24% | 13.62MiB/CPU-s | 191.91KiB/CPU-s | 2.48KiB/CPU-s | 0.0 | 0.013757 | 13.62MiB/CPU-s | 196.73KiB/CPU-s | 2.68KiB/CPU-s | 0.0 | 0.014103 | False | False |
| datadog_agent_remap_datadog_logs_acks | -210.3KiB/CPU-s | -0.62 | 100.00% | 32.99MiB/CPU-s | 1.62MiB/CPU-s | 21.45KiB/CPU-s | 0.0 | 0.04921 | 32.78MiB/CPU-s | 1.23MiB/CPU-s | 16.21KiB/CPU-s | 0.0 | 0.037418 | False | False |
| socket_to_socket_blackhole | -135.2KiB/CPU-s | -0.99 | 100.00% | 13.27MiB/CPU-s | 246.45KiB/CPU-s | 3.18KiB/CPU-s | 0.0 | 0.018134 | 13.14MiB/CPU-s | 144.68KiB/CPU-s | 1.87KiB/CPU-s | 0.0 | 0.010753 | False | False |
| http_text_to_http_json | -258.84KiB/CPU-s | -1.06 | 100.00% | 23.74MiB/CPU-s | 976.92KiB/CPU-s | 12.61KiB/CPU-s | 0.0 | 0.040188 | 23.48MiB/CPU-s | 685.5KiB/CPU-s | 8.84KiB/CPU-s | 0.0 | 0.028503 | False | False |
| otlp_grpc_to_blackhole | -11.81KiB/CPU-s | -1.1 | 100.00% | 1.05MiB/CPU-s | 38.62KiB/CPU-s | 510.49B/CPU-s | 0.0 | 0.036075 | 1.03MiB/CPU-s | 51.4KiB/CPU-s | 679.06B/CPU-s | 0.0 | 0.048551 | False | False |
| datadog_agent_remap_blackhole_acks | -360.06KiB/CPU-s | -1.12 | 100.00% | 31.49MiB/CPU-s | 741.75KiB/CPU-s | 9.58KiB/CPU-s | 0.0 | 0.022997 | 31.14MiB/CPU-s | 1.74MiB/CPU-s | 23.03KiB/CPU-s | 0.0 | 0.055934 | False | False |
| syslog_splunk_hec_logs | -137.25KiB/CPU-s | -1.5 | 100.00% | 8.97MiB/CPU-s | 233.17KiB/CPU-s | 3.01KiB/CPU-s | 0.0 | 0.025397 | 8.83MiB/CPU-s | 331.76KiB/CPU-s | 4.28KiB/CPU-s | 0.0 | 0.036684 | False | False |
| otlp_http_to_blackhole | -29.12KiB/CPU-s | -1.85 | 100.00% | 1.54MiB/CPU-s | 114.45KiB/CPU-s | 1.48KiB/CPU-s | 0.0 | 0.07253 | 1.51MiB/CPU-s | 126.24KiB/CPU-s | 1.63KiB/CPU-s | 0.0 | 0.081504 | False | False |
| syslog_log2metric_humio_metrics | -137.22KiB/CPU-s | -2.24 | 100.00% | 5.97MiB/CPU-s | 230.21KiB/CPU-s | 2.97KiB/CPU-s | 0.0 | 0.037641 | 5.84MiB/CPU-s | 368.1KiB/CPU-s | 4.75KiB/CPU-s | 0.0 | 0.061569 | False | False |
| syslog_loki | -208.45KiB/CPU-s | -2.36 | 100.00% | 8.63MiB/CPU-s | 308.52KiB/CPU-s | 3.98KiB/CPU-s | 0.0 | 0.034915 | 8.42MiB/CPU-s | 401.64KiB/CPU-s | 5.19KiB/CPU-s | 0.0 | 0.046551 | False | False |
| syslog_humio_logs | -257.12KiB/CPU-s | -2.75 | 100.00% | 9.14MiB/CPU-s | 152.53KiB/CPU-s | 1.97KiB/CPU-s | 0.0 | 0.016304 | 8.88MiB/CPU-s | 225.96KiB/CPU-s | 2.92KiB/CPU-s | 0.0 | 0.024835 | False | False |
| splunk_hec_route_s3 | -468.6KiB/CPU-s | -3.91 | 100.00% | 11.7MiB/CPU-s | 575.34KiB/CPU-s | 7.42KiB/CPU-s | 0.0 | 0.048018 | 11.24MiB/CPU-s | 735.19KiB/CPU-s | 9.49KiB/CPU-s | 0.0 | 0.063856 | False | False |