vector icon indicating copy to clipboard operation
vector copied to clipboard

docs(host_metrics source): autogen docs

Open davidhuie-dd opened this issue 2 years ago • 5 comments
trafficstars

Closes #15841

davidhuie-dd avatar Jan 13 '23 21:01 davidhuie-dd

CLA assistant check
All committers have signed the CLA.

bits-bot avatar Jan 13 '23 21:01 bits-bot

Deploy Preview for vrl-playground canceled.

Name Link
Latest commit 9a409591e9ee2182763e2cbea0b4c096ab90a56f
Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/63c87154f74fa000081dd42c

netlify[bot] avatar Jan 13 '23 21:01 netlify[bot]

Deploy Preview for vector-project ready!

Name Link
Latest commit 9a409591e9ee2182763e2cbea0b4c096ab90a56f
Latest deploy log https://app.netlify.com/sites/vector-project/deploys/63c8715418d1ad000997e8a7
Deploy Preview https://deploy-preview-15956--vector-project.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

netlify[bot] avatar Jan 13 '23 21:01 netlify[bot]

Regression Test Results

Run ID: 582dfd78-642d-43b7-aab9-b54871c2a571
Baseline: 6d05e13337bf8daacf47f0e289a521408fe079cc
Comparison: d715b942c836cdd8b13ca910d8b4d9e97dccd392
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
otlp_http_to_blackhole -154.99KiB/CPU-s -9.79 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_splunk_hec_logs 122.0KiB/CPU-s 1.35 100.00% 8.82MiB/CPU-s 390.45KiB/CPU-s 5.04KiB/CPU-s 0.0 0.043224 8.94MiB/CPU-s 215.39KiB/CPU-s 2.78KiB/CPU-s 0.0 0.023527 False False
syslog_log2metric_splunk_hec_metrics 90.52KiB/CPU-s 0.98 100.00% 9.01MiB/CPU-s 414.68KiB/CPU-s 5.35KiB/CPU-s 0.0 0.044943 9.1MiB/CPU-s 224.66KiB/CPU-s 2.9KiB/CPU-s 0.0 0.024112 False False
datadog_agent_remap_blackhole_acks 261.22KiB/CPU-s 0.83 100.00% 30.75MiB/CPU-s 1.34MiB/CPU-s 17.7KiB/CPU-s 0.0 0.043532 31.0MiB/CPU-s 1.65MiB/CPU-s 21.84KiB/CPU-s 0.0 0.053303 False False
datadog_agent_remap_blackhole 79.62KiB/CPU-s 0.25 99.97% 30.86MiB/CPU-s 958.24KiB/CPU-s 12.38KiB/CPU-s 0.0 0.030317 30.94MiB/CPU-s 1.39MiB/CPU-s 18.42KiB/CPU-s 0.0 0.045028 False False
http_to_http_json 30.43KiB/CPU-s 0.22 100.00% 13.59MiB/CPU-s 257.52KiB/CPU-s 3.32KiB/CPU-s 0.0 0.0185 13.62MiB/CPU-s 209.65KiB/CPU-s 2.7KiB/CPU-s 0.0 0.015028 False False
splunk_hec_to_splunk_hec_logs_noack 5.58KiB/CPU-s 0.04 84.44% 13.62MiB/CPU-s 237.81KiB/CPU-s 3.07KiB/CPU-s 0.0 0.01705 13.63MiB/CPU-s 189.98KiB/CPU-s 2.45KiB/CPU-s 0.0 0.013616 False False
enterprise_http_to_http 4.08KiB/CPU-s 0.03 76.41% 13.62MiB/CPU-s 221.63KiB/CPU-s 2.86KiB/CPU-s 0.0 0.01589 13.62MiB/CPU-s 147.67KiB/CPU-s 1.91KiB/CPU-s 0.0 0.010584 False False
splunk_hec_to_splunk_hec_logs_acks 1.92KiB/CPU-s 0.01 24.05% 13.62MiB/CPU-s 343.83KiB/CPU-s 4.44KiB/CPU-s 0.0 0.024658 13.62MiB/CPU-s 345.13KiB/CPU-s 4.45KiB/CPU-s 0.0 0.024748 False False
fluent_elasticsearch -357.91B/CPU-s -0.0 48.08% 45.41MiB/CPU-s 29.78KiB/CPU-s 389.43B/CPU-s 0.0 0.00064 45.41MiB/CPU-s 30.28KiB/CPU-s 395.83B/CPU-s 0.0 0.000651 False False
file_to_blackhole -8.07KiB/CPU-s -0.01 29.07% 54.49MiB/CPU-s 1.09MiB/CPU-s 14.34KiB/CPU-s 0.0 0.019925 54.49MiB/CPU-s 1.23MiB/CPU-s 16.21KiB/CPU-s 0.0 0.022538 False False
splunk_hec_indexer_ack_blackhole -737.18B/CPU-s -0.01 12.39% 13.62MiB/CPU-s 251.05KiB/CPU-s 3.24KiB/CPU-s 0.0 0.018004 13.62MiB/CPU-s 255.09KiB/CPU-s 3.29KiB/CPU-s 0.0 0.018294 False False
http_to_http_noack -2.06KiB/CPU-s -0.01 40.94% 13.62MiB/CPU-s 192.84KiB/CPU-s 2.49KiB/CPU-s 0.0 0.013824 13.62MiB/CPU-s 224.48KiB/CPU-s 2.9KiB/CPU-s 0.0 0.016094 False False
otlp_grpc_to_blackhole -826.03B/CPU-s -0.08 69.03% 1.05MiB/CPU-s 32.44KiB/CPU-s 428.85B/CPU-s 0.0 0.030168 1.05MiB/CPU-s 52.29KiB/CPU-s 690.8B/CPU-s 0.0 0.048664 False False
datadog_agent_remap_datadog_logs -153.06KiB/CPU-s -0.45 100.00% 33.46MiB/CPU-s 1.38MiB/CPU-s 18.2KiB/CPU-s 0.0 0.041172 33.31MiB/CPU-s 1.72MiB/CPU-s 22.72KiB/CPU-s 0.0 0.051627 False False
http_to_http_acks -27.73KiB/CPU-s -0.52 41.25% 5.24MiB/CPU-s 2.75MiB/CPU-s 36.37KiB/CPU-s 0.0 0.525643 5.21MiB/CPU-s 2.72MiB/CPU-s 35.94KiB/CPU-s 0.0 0.521991 True False
splunk_hec_route_s3 -97.55KiB/CPU-s -0.82 100.00% 11.64MiB/CPU-s 559.9KiB/CPU-s 7.22KiB/CPU-s 0.0 0.046966 11.55MiB/CPU-s 618.3KiB/CPU-s 7.98KiB/CPU-s 0.0 0.052293 False False
syslog_log2metric_humio_metrics -51.33KiB/CPU-s -0.84 100.00% 5.99MiB/CPU-s 286.54KiB/CPU-s 3.7KiB/CPU-s 0.0 0.046724 5.94MiB/CPU-s 282.29KiB/CPU-s 3.64KiB/CPU-s 0.0 0.04642 False False
datadog_agent_remap_datadog_logs_acks -295.24KiB/CPU-s -0.87 100.00% 33.31MiB/CPU-s 1.82MiB/CPU-s 24.05KiB/CPU-s 0.0 0.054626 33.02MiB/CPU-s 1.38MiB/CPU-s 18.25KiB/CPU-s 0.0 0.041796 False False
syslog_humio_logs -101.36KiB/CPU-s -1.09 100.00% 9.05MiB/CPU-s 356.75KiB/CPU-s 4.61KiB/CPU-s 0.0 0.038511 8.95MiB/CPU-s 242.28KiB/CPU-s 3.13KiB/CPU-s 0.0 0.026444 False False
syslog_regex_logs2metric_ddmetrics -40.69KiB/CPU-s -1.09 100.00% 3.63MiB/CPU-s 384.42KiB/CPU-s 4.96KiB/CPU-s 0.0 0.103418 3.59MiB/CPU-s 441.4KiB/CPU-s 5.7KiB/CPU-s 0.0 0.120061 True False
socket_to_socket_blackhole -217.79KiB/CPU-s -1.59 100.00% 13.41MiB/CPU-s 280.31KiB/CPU-s 3.62KiB/CPU-s 0.0 0.020418 13.19MiB/CPU-s 336.82KiB/CPU-s 4.35KiB/CPU-s 0.0 0.02493 False False
syslog_loki -172.9KiB/CPU-s -1.93 100.00% 8.73MiB/CPU-s 256.6KiB/CPU-s 3.31KiB/CPU-s 0.0 0.028713 8.56MiB/CPU-s 355.61KiB/CPU-s 4.59KiB/CPU-s 0.0 0.040578 False False
http_text_to_http_json -541.14KiB/CPU-s -2.13 100.00% 24.8MiB/CPU-s 566.22KiB/CPU-s 7.31KiB/CPU-s 0.0 0.02229 24.28MiB/CPU-s 825.99KiB/CPU-s 10.66KiB/CPU-s 0.0 0.033224 False False
otlp_http_to_blackhole -154.99KiB/CPU-s -9.79 100.00% 1.55MiB/CPU-s 107.19KiB/CPU-s 1.38KiB/CPU-s 0.0 0.067701 1.39MiB/CPU-s 436.22KiB/CPU-s 5.63KiB/CPU-s 0.0 0.305425 True False

github-actions[bot] avatar Jan 13 '23 22:01 github-actions[bot]

Regression Test Results

Run ID: 3fd53168-6a96-4495-9e73-2531ccc39fe4
Baseline: 39c7f015d7715d246e42b15eb66e6041999e944f
Comparison: b26d9d3d007f53d52a7e509ce404cf7214585d6e
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
syslog_log2metric_splunk_hec_metrics -1016.15KiB/CPU-s -10.67 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json 663.28KiB/CPU-s 2.73 100.00% 23.72MiB/CPU-s 950.02KiB/CPU-s 12.26KiB/CPU-s 0.0 0.039102 24.37MiB/CPU-s 712.48KiB/CPU-s 9.19KiB/CPU-s 0.0 0.028546 False False
datadog_agent_remap_blackhole_acks 440.8KiB/CPU-s 1.37 100.00% 31.43MiB/CPU-s 1.23MiB/CPU-s 16.22KiB/CPU-s 0.0 0.039031 31.86MiB/CPU-s 894.43KiB/CPU-s 11.55KiB/CPU-s 0.0 0.027412 False False
syslog_humio_logs 118.31KiB/CPU-s 1.29 100.00% 8.93MiB/CPU-s 290.07KiB/CPU-s 3.75KiB/CPU-s 0.0 0.031708 9.05MiB/CPU-s 245.01KiB/CPU-s 3.16KiB/CPU-s 0.0 0.026441 False False
socket_to_socket_blackhole 5.06KiB/CPU-s 0.04 52.89% 13.16MiB/CPU-s 519.19KiB/CPU-s 6.7KiB/CPU-s 0.0 0.038533 13.16MiB/CPU-s 160.93KiB/CPU-s 2.08KiB/CPU-s 0.0 0.011939 False False
splunk_hec_to_splunk_hec_logs_noack 3.16KiB/CPU-s 0.02 60.08% 13.62MiB/CPU-s 219.97KiB/CPU-s 2.84KiB/CPU-s 0.0 0.015769 13.62MiB/CPU-s 189.45KiB/CPU-s 2.45KiB/CPU-s 0.0 0.013578 False False
splunk_hec_to_splunk_hec_logs_acks 1.09KiB/CPU-s 0.01 13.49% 13.62MiB/CPU-s 344.81KiB/CPU-s 4.45KiB/CPU-s 0.0 0.024729 13.62MiB/CPU-s 359.26KiB/CPU-s 4.64KiB/CPU-s 0.0 0.025764 False False
fluent_elasticsearch 327.47B/CPU-s 0.0 44.58% 45.41MiB/CPU-s 30.04KiB/CPU-s 392.71B/CPU-s 0.0 0.000646 45.41MiB/CPU-s 29.83KiB/CPU-s 390.14B/CPU-s 0.0 0.000642 False False
enterprise_http_to_http 177.29B/CPU-s 0.0 3.94% 13.62MiB/CPU-s 191.49KiB/CPU-s 2.47KiB/CPU-s 0.0 0.013727 13.62MiB/CPU-s 192.34KiB/CPU-s 2.48KiB/CPU-s 0.0 0.013787 False False
splunk_hec_indexer_ack_blackhole -1.58KiB/CPU-s -0.01 26.69% 13.62MiB/CPU-s 252.23KiB/CPU-s 3.25KiB/CPU-s 0.0 0.018088 13.62MiB/CPU-s 256.34KiB/CPU-s 3.31KiB/CPU-s 0.0 0.018385 False False
file_to_blackhole -13.71KiB/CPU-s -0.02 47.92% 54.5MiB/CPU-s 1.18MiB/CPU-s 15.52KiB/CPU-s 0.0 0.021562 54.48MiB/CPU-s 1.11MiB/CPU-s 14.66KiB/CPU-s 0.0 0.020382 False False
http_to_http_noack -2.29KiB/CPU-s -0.02 31.09% 13.61MiB/CPU-s 302.38KiB/CPU-s 3.9KiB/CPU-s 0.0 0.021689 13.61MiB/CPU-s 325.36KiB/CPU-s 4.2KiB/CPU-s 0.0 0.02334 False False
datadog_agent_remap_datadog_logs -47.82KiB/CPU-s -0.14 93.85% 34.18MiB/CPU-s 1.59MiB/CPU-s 20.96KiB/CPU-s 0.0 0.046417 34.13MiB/CPU-s 1.11MiB/CPU-s 14.65KiB/CPU-s 0.0 0.032493 False False
datadog_agent_remap_datadog_logs_acks -110.37KiB/CPU-s -0.32 100.00% 34.17MiB/CPU-s 741.13KiB/CPU-s 9.57KiB/CPU-s 0.0 0.021178 34.06MiB/CPU-s 942.34KiB/CPU-s 12.16KiB/CPU-s 0.0 0.027013 False False
otlp_grpc_to_blackhole -3.7KiB/CPU-s -0.35 100.00% 1.05MiB/CPU-s 38.3KiB/CPU-s 506.3B/CPU-s 0.0 0.035749 1.04MiB/CPU-s 50.45KiB/CPU-s 666.55B/CPU-s 0.0 0.047254 False False
otlp_http_to_blackhole -5.79KiB/CPU-s -0.37 98.88% 1.53MiB/CPU-s 118.03KiB/CPU-s 1.52KiB/CPU-s 0.0 0.075521 1.52MiB/CPU-s 131.86KiB/CPU-s 1.7KiB/CPU-s 0.0 0.084684 False False
http_to_http_json -86.56KiB/CPU-s -0.62 100.00% 13.62MiB/CPU-s 209.18KiB/CPU-s 2.7KiB/CPU-s 0.0 0.014994 13.54MiB/CPU-s 348.18KiB/CPU-s 4.49KiB/CPU-s 0.0 0.025115 False False
splunk_hec_route_s3 -75.71KiB/CPU-s -0.62 100.00% 11.83MiB/CPU-s 539.58KiB/CPU-s 6.96KiB/CPU-s 0.0 0.044524 11.76MiB/CPU-s 527.65KiB/CPU-s 6.81KiB/CPU-s 0.0 0.043813 False False
syslog_splunk_hec_logs -65.34KiB/CPU-s -0.71 100.00% 8.97MiB/CPU-s 238.77KiB/CPU-s 3.08KiB/CPU-s 0.0 0.026001 8.9MiB/CPU-s 306.59KiB/CPU-s 3.95KiB/CPU-s 0.0 0.033626 False False
http_to_http_acks -42.83KiB/CPU-s -0.79 58.56% 5.28MiB/CPU-s 2.87MiB/CPU-s 37.92KiB/CPU-s 0.0 0.543397 5.24MiB/CPU-s 2.74MiB/CPU-s 36.28KiB/CPU-s 0.0 0.524073 True False
syslog_loki -137.88KiB/CPU-s -1.52 100.00% 8.84MiB/CPU-s 185.28KiB/CPU-s 2.39KiB/CPU-s 0.0 0.020467 8.7MiB/CPU-s 298.7KiB/CPU-s 3.86KiB/CPU-s 0.0 0.033507 False False
syslog_regex_logs2metric_ddmetrics -64.41KiB/CPU-s -1.73 100.00% 3.63MiB/CPU-s 426.21KiB/CPU-s 5.5KiB/CPU-s 0.0 0.114742 3.56MiB/CPU-s 467.38KiB/CPU-s 6.03KiB/CPU-s 0.0 0.128045 True False
syslog_log2metric_humio_metrics -218.49KiB/CPU-s -3.44 100.00% 6.21MiB/CPU-s 137.55KiB/CPU-s 1.78KiB/CPU-s 0.0 0.021637 5.99MiB/CPU-s 349.22KiB/CPU-s 4.51KiB/CPU-s 0.0 0.056889 False False
datadog_agent_remap_blackhole -1.34MiB/CPU-s -4.39 100.00% 30.55MiB/CPU-s 1.02MiB/CPU-s 13.5KiB/CPU-s 0.0 0.033406 29.21MiB/CPU-s 1.71MiB/CPU-s 22.58KiB/CPU-s 0.0 0.058461 False False
syslog_log2metric_splunk_hec_metrics -1016.15KiB/CPU-s -10.67 100.00% 9.3MiB/CPU-s 216.95KiB/CPU-s 2.8KiB/CPU-s 0.0 0.02277 8.31MiB/CPU-s 2.41MiB/CPU-s 31.8KiB/CPU-s 0.0 0.289406 True False

github-actions[bot] avatar Jan 13 '23 22:01 github-actions[bot]

Regression Test Results

Run ID: 1db80153-01f3-48db-a963-d09e0096354a
Baseline: 24e89dd8180fa430fc6ff17ec535b2fdd8c768d7
Comparison: 4068e2366db9edd880907d5a6d029b0ef4fe8d79
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
syslog_log2metric_humio_metrics -540.33KiB/CPU-s -8.37 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json 920.94KiB/CPU-s 3.85 100.00% 23.34MiB/CPU-s 556.45KiB/CPU-s 7.18KiB/CPU-s 0.0 0.023284 24.24MiB/CPU-s 762.44KiB/CPU-s 9.84KiB/CPU-s 0.0 0.03072 False False
otlp_http_to_blackhole 48.69KiB/CPU-s 3.23 100.00% 1.47MiB/CPU-s 128.12KiB/CPU-s 1.65KiB/CPU-s 0.0 0.084862 1.52MiB/CPU-s 124.45KiB/CPU-s 1.61KiB/CPU-s 0.0 0.079858 False False
socket_to_socket_blackhole 346.45KiB/CPU-s 2.6 100.00% 12.99MiB/CPU-s 203.4KiB/CPU-s 2.63KiB/CPU-s 0.0 0.015291 13.33MiB/CPU-s 196.37KiB/CPU-s 2.54KiB/CPU-s 0.0 0.014388 False False
datadog_agent_remap_datadog_logs_acks 532.22KiB/CPU-s 1.59 100.00% 32.66MiB/CPU-s 1.61MiB/CPU-s 21.23KiB/CPU-s 0.0 0.049207 33.18MiB/CPU-s 1.22MiB/CPU-s 16.18KiB/CPU-s 0.0 0.036904 False False
syslog_loki 100.02KiB/CPU-s 1.11 100.00% 8.77MiB/CPU-s 332.84KiB/CPU-s 4.3KiB/CPU-s 0.0 0.037076 8.86MiB/CPU-s 134.68KiB/CPU-s 1.74KiB/CPU-s 0.0 0.014837 False False
syslog_regex_logs2metric_ddmetrics 36.34KiB/CPU-s 0.99 99.99% 3.57MiB/CPU-s 500.49KiB/CPU-s 6.46KiB/CPU-s 0.0 0.136847 3.61MiB/CPU-s 515.25KiB/CPU-s 6.65KiB/CPU-s 0.0 0.139496 True False
syslog_log2metric_splunk_hec_metrics 30.93KiB/CPU-s 0.34 100.00% 8.95MiB/CPU-s 331.06KiB/CPU-s 4.27KiB/CPU-s 0.0 0.03612 8.98MiB/CPU-s 357.87KiB/CPU-s 4.62KiB/CPU-s 0.0 0.038914 False False
otlp_grpc_to_blackhole 2.54KiB/CPU-s 0.24 98.47% 1.02MiB/CPU-s 53.32KiB/CPU-s 704.68B/CPU-s 0.0 0.051232 1.02MiB/CPU-s 61.35KiB/CPU-s 810.51B/CPU-s 0.0 0.058807 False False
enterprise_http_to_http 6.33KiB/CPU-s 0.05 84.75% 13.62MiB/CPU-s 283.67KiB/CPU-s 3.66KiB/CPU-s 0.0 0.020344 13.62MiB/CPU-s 192.55KiB/CPU-s 2.49KiB/CPU-s 0.0 0.013803 False False
splunk_hec_to_splunk_hec_logs_noack 6.45KiB/CPU-s 0.05 89.83% 13.62MiB/CPU-s 242.14KiB/CPU-s 3.12KiB/CPU-s 0.0 0.017362 13.62MiB/CPU-s 185.7KiB/CPU-s 2.4KiB/CPU-s 0.0 0.013309 False False
file_to_blackhole 22.4B/CPU-s 0.0 0.07% 54.49MiB/CPU-s 1.27MiB/CPU-s 16.75KiB/CPU-s 0.0 0.023271 54.49MiB/CPU-s 1.26MiB/CPU-s 16.64KiB/CPU-s 0.0 0.023136 False False
fluent_elasticsearch -449.69B/CPU-s -0.0 58.14% 45.41MiB/CPU-s 30.21KiB/CPU-s 394.8B/CPU-s 0.0 0.000649 45.41MiB/CPU-s 29.93KiB/CPU-s 391.42B/CPU-s 0.0 0.000643 False False
http_to_http_json -204.79B/CPU-s -0.0 4.13% 13.62MiB/CPU-s 211.24KiB/CPU-s 2.73KiB/CPU-s 0.0 0.015142 13.62MiB/CPU-s 211.7KiB/CPU-s 2.73KiB/CPU-s 0.0 0.015176 False False
splunk_hec_indexer_ack_blackhole -73.81B/CPU-s -0.0 1.24% 13.62MiB/CPU-s 251.74KiB/CPU-s 3.25KiB/CPU-s 0.0 0.018052 13.62MiB/CPU-s 254.62KiB/CPU-s 3.28KiB/CPU-s 0.0 0.018259 False False
splunk_hec_to_splunk_hec_logs_acks -84.81B/CPU-s -0.0 0.99% 13.62MiB/CPU-s 367.97KiB/CPU-s 4.75KiB/CPU-s 0.0 0.02639 13.62MiB/CPU-s 362.32KiB/CPU-s 4.67KiB/CPU-s 0.0 0.025985 False False
http_to_http_noack -1.87KiB/CPU-s -0.01 27.29% 13.62MiB/CPU-s 281.59KiB/CPU-s 3.63KiB/CPU-s 0.0 0.020195 13.61MiB/CPU-s 305.18KiB/CPU-s 3.94KiB/CPU-s 0.0 0.021889 False False
splunk_hec_route_s3 -155.84KiB/CPU-s -1.3 100.00% 11.74MiB/CPU-s 556.89KiB/CPU-s 7.18KiB/CPU-s 0.0 0.046327 11.59MiB/CPU-s 704.44KiB/CPU-s 9.09KiB/CPU-s 0.0 0.059371 False False
datadog_agent_remap_blackhole -513.91KiB/CPU-s -1.64 100.00% 30.67MiB/CPU-s 1.37MiB/CPU-s 18.08KiB/CPU-s 0.0 0.0446 30.16MiB/CPU-s 1.8MiB/CPU-s 23.77KiB/CPU-s 0.0 0.05964 False False
http_to_http_acks -92.88KiB/CPU-s -1.71 92.88% 5.29MiB/CPU-s 2.76MiB/CPU-s 36.42KiB/CPU-s 0.0 0.520597 5.2MiB/CPU-s 2.75MiB/CPU-s 36.38KiB/CPU-s 0.0 0.528956 True False
datadog_agent_remap_datadog_logs -593.86KiB/CPU-s -1.72 100.00% 33.8MiB/CPU-s 1.33MiB/CPU-s 17.5KiB/CPU-s 0.0 0.039204 33.22MiB/CPU-s 1.6MiB/CPU-s 21.15KiB/CPU-s 0.0 0.048187 False False
syslog_splunk_hec_logs -158.74KiB/CPU-s -1.73 100.00% 8.97MiB/CPU-s 223.99KiB/CPU-s 2.89KiB/CPU-s 0.0 0.024385 8.81MiB/CPU-s 297.55KiB/CPU-s 3.84KiB/CPU-s 0.0 0.032962 False False
datadog_agent_remap_blackhole_acks -620.51KiB/CPU-s -1.95 100.00% 31.03MiB/CPU-s 1.45MiB/CPU-s 19.19KiB/CPU-s 0.0 0.04678 30.42MiB/CPU-s 1.97MiB/CPU-s 26.03KiB/CPU-s 0.0 0.064741 False False
syslog_humio_logs -193.36KiB/CPU-s -2.08 100.00% 9.07MiB/CPU-s 216.01KiB/CPU-s 2.79KiB/CPU-s 0.0 0.023251 8.88MiB/CPU-s 292.78KiB/CPU-s 3.78KiB/CPU-s 0.0 0.032185 False False
syslog_log2metric_humio_metrics -540.33KiB/CPU-s -8.37 100.00% 6.3MiB/CPU-s 98.31KiB/CPU-s 1.27KiB/CPU-s 0.0 0.015235 5.77MiB/CPU-s 350.14KiB/CPU-s 4.52KiB/CPU-s 0.0 0.05922 False False

github-actions[bot] avatar Jan 17 '23 18:01 github-actions[bot]

Regression Test Results

Run ID: fe70e117-be71-4be1-bec5-5ff6a019f6ed
Baseline: 24e89dd8180fa430fc6ff17ec535b2fdd8c768d7
Comparison: cb97b95a157e23a0dc57e2382cbd227006293559
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
http_text_to_http_json 1.33MiB/CPU-s 5.74 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json 1.33MiB/CPU-s 5.74 100.00% 23.12MiB/CPU-s 737.33KiB/CPU-s 9.52KiB/CPU-s 0.0 0.031138 24.45MiB/CPU-s 664.78KiB/CPU-s 8.58KiB/CPU-s 0.0 0.02655 False False
otlp_http_to_blackhole 66.92KiB/CPU-s 4.43 100.00% 1.47MiB/CPU-s 130.35KiB/CPU-s 1.68KiB/CPU-s 0.0 0.086314 1.54MiB/CPU-s 120.8KiB/CPU-s 1.56KiB/CPU-s 0.0 0.076598 False False
socket_to_socket_blackhole 398.35KiB/CPU-s 3.04 100.00% 12.79MiB/CPU-s 526.64KiB/CPU-s 6.8KiB/CPU-s 0.0 0.040217 13.18MiB/CPU-s 416.64KiB/CPU-s 5.38KiB/CPU-s 0.0 0.030877 False False
syslog_loki 170.7KiB/CPU-s 1.92 100.00% 8.7MiB/CPU-s 560.26KiB/CPU-s 7.23KiB/CPU-s 0.0 0.062918 8.86MiB/CPU-s 105.16KiB/CPU-s 1.36KiB/CPU-s 0.0 0.011588 False False
http_to_http_acks 89.83KiB/CPU-s 1.7 91.89% 5.17MiB/CPU-s 2.71MiB/CPU-s 35.82KiB/CPU-s 0.0 0.523746 5.26MiB/CPU-s 2.8MiB/CPU-s 36.99KiB/CPU-s 0.0 0.531745 True False
datadog_agent_remap_blackhole_acks 507.31KiB/CPU-s 1.65 100.00% 30.11MiB/CPU-s 1.56MiB/CPU-s 20.64KiB/CPU-s 0.0 0.051833 30.61MiB/CPU-s 1.66MiB/CPU-s 21.89KiB/CPU-s 0.0 0.054072 False False
datadog_agent_remap_datadog_logs 519.52KiB/CPU-s 1.51 100.00% 33.5MiB/CPU-s 1.2MiB/CPU-s 15.8KiB/CPU-s 0.0 0.035688 34.01MiB/CPU-s 1.21MiB/CPU-s 15.95KiB/CPU-s 0.0 0.03551 False False
otlp_grpc_to_blackhole 11.84KiB/CPU-s 1.13 100.00% 1.02MiB/CPU-s 46.67KiB/CPU-s 616.86B/CPU-s 0.0 0.044628 1.03MiB/CPU-s 51.9KiB/CPU-s 685.88B/CPU-s 0.0 0.049074 False False
datadog_agent_remap_datadog_logs_acks 316.19KiB/CPU-s 0.92 100.00% 33.48MiB/CPU-s 1.78MiB/CPU-s 23.48KiB/CPU-s 0.0 0.053076 33.79MiB/CPU-s 1.25MiB/CPU-s 16.57KiB/CPU-s 0.0 0.037108 False False
syslog_humio_logs 52.61KiB/CPU-s 0.57 100.00% 9.06MiB/CPU-s 331.16KiB/CPU-s 4.27KiB/CPU-s 0.0 0.035681 9.11MiB/CPU-s 192.21KiB/CPU-s 2.48KiB/CPU-s 0.0 0.020593 False False
syslog_splunk_hec_logs 34.5KiB/CPU-s 0.38 100.00% 8.97MiB/CPU-s 283.45KiB/CPU-s 3.66KiB/CPU-s 0.0 0.030861 9.0MiB/CPU-s 229.69KiB/CPU-s 2.96KiB/CPU-s 0.0 0.024914 False False
datadog_agent_remap_blackhole 75.04KiB/CPU-s 0.24 99.22% 30.93MiB/CPU-s 1.58MiB/CPU-s 20.93KiB/CPU-s 0.0 0.051171 31.01MiB/CPU-s 1.43MiB/CPU-s 18.9KiB/CPU-s 0.0 0.046133 False False
splunk_hec_to_splunk_hec_logs_acks 3.64KiB/CPU-s 0.03 43.95% 13.62MiB/CPU-s 351.9KiB/CPU-s 4.54KiB/CPU-s 0.0 0.025238 13.62MiB/CPU-s 332.8KiB/CPU-s 4.29KiB/CPU-s 0.0 0.023861 False False
splunk_hec_to_splunk_hec_logs_noack 2.27KiB/CPU-s 0.02 46.40% 13.62MiB/CPU-s 209.4KiB/CPU-s 2.7KiB/CPU-s 0.0 0.01501 13.63MiB/CPU-s 192.02KiB/CPU-s 2.48KiB/CPU-s 0.0 0.013761 False False
splunk_hec_indexer_ack_blackhole 1.16KiB/CPU-s 0.01 20.36% 13.62MiB/CPU-s 251.37KiB/CPU-s 3.24KiB/CPU-s 0.0 0.018026 13.62MiB/CPU-s 243.09KiB/CPU-s 3.14KiB/CPU-s 0.0 0.017431 False False
enterprise_http_to_http 1.79KiB/CPU-s 0.01 25.87% 13.61MiB/CPU-s 308.69KiB/CPU-s 3.98KiB/CPU-s 0.0 0.022141 13.62MiB/CPU-s 285.29KiB/CPU-s 3.68KiB/CPU-s 0.0 0.02046 False False
http_to_http_noack 166.52B/CPU-s 0.0 2.30% 13.61MiB/CPU-s 311.59KiB/CPU-s 4.02KiB/CPU-s 0.0 0.022349 13.61MiB/CPU-s 307.5KiB/CPU-s 3.97KiB/CPU-s 0.0 0.022056 False False
fluent_elasticsearch -46.71B/CPU-s -0.0 6.74% 45.41MiB/CPU-s 29.79KiB/CPU-s 389.5B/CPU-s 0.0 0.00064 45.41MiB/CPU-s 29.98KiB/CPU-s 392.06B/CPU-s 0.0 0.000645 False False
file_to_blackhole -11.29KiB/CPU-s -0.02 38.90% 54.5MiB/CPU-s 1.18MiB/CPU-s 15.61KiB/CPU-s 0.0 0.021687 54.49MiB/CPU-s 1.2MiB/CPU-s 15.78KiB/CPU-s 0.0 0.021937 False False
http_to_http_json -32.18KiB/CPU-s -0.23 100.00% 13.62MiB/CPU-s 213.22KiB/CPU-s 2.75KiB/CPU-s 0.0 0.015285 13.59MiB/CPU-s 255.8KiB/CPU-s 3.3KiB/CPU-s 0.0 0.018379 False False
syslog_regex_logs2metric_ddmetrics -26.09KiB/CPU-s -0.7 99.75% 3.65MiB/CPU-s 499.11KiB/CPU-s 6.44KiB/CPU-s 0.0 0.13359 3.62MiB/CPU-s 443.98KiB/CPU-s 5.73KiB/CPU-s 0.0 0.11967 True False
splunk_hec_route_s3 -117.31KiB/CPU-s -1.0 100.00% 11.47MiB/CPU-s 722.46KiB/CPU-s 9.32KiB/CPU-s 0.0 0.061506 11.36MiB/CPU-s 805.4KiB/CPU-s 10.4KiB/CPU-s 0.0 0.069259 False False
syslog_log2metric_splunk_hec_metrics -150.03KiB/CPU-s -1.57 100.00% 9.31MiB/CPU-s 469.59KiB/CPU-s 6.06KiB/CPU-s 0.0 0.049228 9.17MiB/CPU-s 413.98KiB/CPU-s 5.34KiB/CPU-s 0.0 0.044092 False False
syslog_log2metric_humio_metrics -171.97KiB/CPU-s -2.85 100.00% 5.88MiB/CPU-s 335.78KiB/CPU-s 4.33KiB/CPU-s 0.0 0.05574 5.71MiB/CPU-s 398.69KiB/CPU-s 5.15KiB/CPU-s 0.0 0.068129 False False

github-actions[bot] avatar Jan 17 '23 18:01 github-actions[bot]

Regression Test Results

Run ID: a4035ab1-fd19-40e7-8f3f-5e3cb643e932
Baseline: c4b971bb96253a43804556854d024788ab7b4741
Comparison: 25a424545efdd00b427d774d1bc16fd10a5c5049
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json 730.23KiB/CPU-s 2.97 100.00% 23.97MiB/CPU-s 543.06KiB/CPU-s 7.01KiB/CPU-s 0.0 0.022123 24.68MiB/CPU-s 590.26KiB/CPU-s 7.62KiB/CPU-s 0.0 0.023351 False False
syslog_log2metric_splunk_hec_metrics 144.91KiB/CPU-s 1.53 100.00% 9.27MiB/CPU-s 429.55KiB/CPU-s 5.55KiB/CPU-s 0.0 0.045245 9.41MiB/CPU-s 215.26KiB/CPU-s 2.78KiB/CPU-s 0.0 0.022332 False False
otlp_http_to_blackhole 15.81KiB/CPU-s 1.02 100.00% 1.52MiB/CPU-s 118.36KiB/CPU-s 1.53KiB/CPU-s 0.0 0.076221 1.53MiB/CPU-s 118.84KiB/CPU-s 1.53KiB/CPU-s 0.0 0.075757 False False
datadog_agent_remap_datadog_logs_acks 315.11KiB/CPU-s 0.91 100.00% 33.74MiB/CPU-s 1.09MiB/CPU-s 14.4KiB/CPU-s 0.0 0.03231 34.04MiB/CPU-s 872.32KiB/CPU-s 11.26KiB/CPU-s 0.0 0.02502 False False
socket_to_socket_blackhole 91.25KiB/CPU-s 0.68 100.00% 13.17MiB/CPU-s 199.73KiB/CPU-s 2.58KiB/CPU-s 0.0 0.014812 13.26MiB/CPU-s 143.27KiB/CPU-s 1.85KiB/CPU-s 0.0 0.010554 False False
datadog_agent_remap_datadog_logs 229.55KiB/CPU-s 0.66 100.00% 33.92MiB/CPU-s 1.11MiB/CPU-s 14.69KiB/CPU-s 0.0 0.032781 34.15MiB/CPU-s 1.25MiB/CPU-s 16.45KiB/CPU-s 0.0 0.036471 False False
syslog_log2metric_humio_metrics 39.83KiB/CPU-s 0.65 100.00% 6.01MiB/CPU-s 367.88KiB/CPU-s 4.75KiB/CPU-s 0.0 0.059741 6.05MiB/CPU-s 221.6KiB/CPU-s 2.86KiB/CPU-s 0.0 0.035756 False False
otlp_grpc_to_blackhole 2.01KiB/CPU-s 0.19 99.18% 1.04MiB/CPU-s 32.67KiB/CPU-s 431.88B/CPU-s 0.0 0.030556 1.05MiB/CPU-s 49.13KiB/CPU-s 649.16B/CPU-s 0.0 0.045869 False False
http_to_http_json 19.09KiB/CPU-s 0.14 100.00% 13.6MiB/CPU-s 246.86KiB/CPU-s 3.18KiB/CPU-s 0.0 0.01772 13.62MiB/CPU-s 210.9KiB/CPU-s 2.72KiB/CPU-s 0.0 0.015118 False False
enterprise_http_to_http 6.38KiB/CPU-s 0.05 80.09% 13.61MiB/CPU-s 310.84KiB/CPU-s 4.01KiB/CPU-s 0.0 0.022296 13.62MiB/CPU-s 227.3KiB/CPU-s 2.93KiB/CPU-s 0.0 0.016296 False False
http_to_http_noack 4.39KiB/CPU-s 0.03 56.75% 13.61MiB/CPU-s 329.93KiB/CPU-s 4.26KiB/CPU-s 0.0 0.023669 13.62MiB/CPU-s 280.83KiB/CPU-s 3.62KiB/CPU-s 0.0 0.02014 False False
splunk_hec_to_splunk_hec_logs_noack 4.45KiB/CPU-s 0.03 75.47% 13.62MiB/CPU-s 226.77KiB/CPU-s 2.93KiB/CPU-s 0.0 0.016258 13.63MiB/CPU-s 191.65KiB/CPU-s 2.47KiB/CPU-s 0.0 0.013736 False False
splunk_hec_to_splunk_hec_logs_acks 3.4KiB/CPU-s 0.02 37.87% 13.61MiB/CPU-s 383.32KiB/CPU-s 4.94KiB/CPU-s 0.0 0.027493 13.62MiB/CPU-s 370.38KiB/CPU-s 4.78KiB/CPU-s 0.0 0.026558 False False
fluent_elasticsearch -156.85B/CPU-s -0.0 22.33% 45.41MiB/CPU-s 29.89KiB/CPU-s 390.67B/CPU-s 0.0 0.000643 45.41MiB/CPU-s 29.95KiB/CPU-s 391.31B/CPU-s 0.0 0.000644 False False
splunk_hec_indexer_ack_blackhole -2.06KiB/CPU-s -0.01 34.72% 13.62MiB/CPU-s 243.98KiB/CPU-s 3.15KiB/CPU-s 0.0 0.017494 13.62MiB/CPU-s 256.89KiB/CPU-s 3.31KiB/CPU-s 0.0 0.018423 False False
file_to_blackhole -9.73KiB/CPU-s -0.02 34.99% 54.49MiB/CPU-s 1.1MiB/CPU-s 14.53KiB/CPU-s 0.0 0.020184 54.48MiB/CPU-s 1.2MiB/CPU-s 15.79KiB/CPU-s 0.0 0.021962 False False
splunk_hec_route_s3 -27.54KiB/CPU-s -0.23 97.14% 11.56MiB/CPU-s 675.69KiB/CPU-s 8.72KiB/CPU-s 0.0 0.057074 11.53MiB/CPU-s 703.18KiB/CPU-s 9.08KiB/CPU-s 0.0 0.059535 False False
syslog_humio_logs -31.51KiB/CPU-s -0.34 100.00% 9.16MiB/CPU-s 247.35KiB/CPU-s 3.19KiB/CPU-s 0.0 0.026362 9.13MiB/CPU-s 98.28KiB/CPU-s 1.27KiB/CPU-s 0.0 0.01051 False False
datadog_agent_remap_blackhole -151.35KiB/CPU-s -0.47 100.00% 31.29MiB/CPU-s 1.0MiB/CPU-s 13.25KiB/CPU-s 0.0 0.032031 31.15MiB/CPU-s 916.2KiB/CPU-s 11.83KiB/CPU-s 0.0 0.028723 False False
datadog_agent_remap_blackhole_acks -152.34KiB/CPU-s -0.47 100.00% 31.48MiB/CPU-s 694.0KiB/CPU-s 8.96KiB/CPU-s 0.0 0.021528 31.33MiB/CPU-s 1.21MiB/CPU-s 16.06KiB/CPU-s 0.0 0.038763 False False
syslog_loki -59.89KiB/CPU-s -0.66 100.00% 8.87MiB/CPU-s 191.8KiB/CPU-s 2.48KiB/CPU-s 0.0 0.021125 8.81MiB/CPU-s 142.95KiB/CPU-s 1.85KiB/CPU-s 0.0 0.015849 False False
syslog_regex_logs2metric_ddmetrics -42.33KiB/CPU-s -1.13 100.00% 3.66MiB/CPU-s 509.72KiB/CPU-s 6.58KiB/CPU-s 0.0 0.135993 3.62MiB/CPU-s 482.98KiB/CPU-s 6.24KiB/CPU-s 0.0 0.13033 True False
http_to_http_acks -71.94KiB/CPU-s -1.33 83.97% 5.29MiB/CPU-s 2.77MiB/CPU-s 36.57KiB/CPU-s 0.0 0.523421 5.22MiB/CPU-s 2.72MiB/CPU-s 35.89KiB/CPU-s 0.0 0.520515 True False
syslog_splunk_hec_logs -259.63KiB/CPU-s -2.79 100.00% 9.08MiB/CPU-s 170.03KiB/CPU-s 2.2KiB/CPU-s 0.0 0.018276 8.83MiB/CPU-s 407.38KiB/CPU-s 5.26KiB/CPU-s 0.0 0.045047 False False

github-actions[bot] avatar Jan 18 '23 18:01 github-actions[bot]

Regression Test Results

Run ID: 6c37d1ef-0c2f-426f-b887-41037376449d
Baseline: 8f2aac8a44b4cb810f7f1bee4201fd67574d5414
Comparison: 89290d16d3c7088a2803dee55366cd34a92b21b1
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
socket_to_socket_blackhole 215.27KiB/CPU-s 1.59 100.00% 13.25MiB/CPU-s 207.23KiB/CPU-s 2.67KiB/CPU-s 0.0 0.015274 13.46MiB/CPU-s 108.88KiB/CPU-s 1.41KiB/CPU-s 0.0 0.0079 False False
otlp_http_to_blackhole 20.97KiB/CPU-s 1.33 100.00% 1.54MiB/CPU-s 112.59KiB/CPU-s 1.45KiB/CPU-s 0.0 0.071483 1.56MiB/CPU-s 105.36KiB/CPU-s 1.36KiB/CPU-s 0.0 0.066018 False False
otlp_grpc_to_blackhole 13.63KiB/CPU-s 1.3 100.00% 1.02MiB/CPU-s 46.84KiB/CPU-s 619.12B/CPU-s 0.0 0.044725 1.04MiB/CPU-s 54.18KiB/CPU-s 715.94B/CPU-s 0.0 0.051072 False False
http_text_to_http_json 291.76KiB/CPU-s 1.19 100.00% 23.93MiB/CPU-s 576.83KiB/CPU-s 7.45KiB/CPU-s 0.0 0.023537 24.22MiB/CPU-s 752.99KiB/CPU-s 9.72KiB/CPU-s 0.0 0.030363 False False
datadog_agent_remap_blackhole 374.42KiB/CPU-s 1.17 100.00% 31.21MiB/CPU-s 1.03MiB/CPU-s 13.66KiB/CPU-s 0.0 0.033114 31.58MiB/CPU-s 840.75KiB/CPU-s 10.85KiB/CPU-s 0.0 0.025999 False False
syslog_log2metric_humio_metrics 51.42KiB/CPU-s 0.81 100.00% 6.21MiB/CPU-s 286.35KiB/CPU-s 3.7KiB/CPU-s 0.0 0.045042 6.26MiB/CPU-s 109.9KiB/CPU-s 1.42KiB/CPU-s 0.0 0.017148 False False
splunk_hec_route_s3 68.33KiB/CPU-s 0.57 100.00% 11.72MiB/CPU-s 531.03KiB/CPU-s 6.85KiB/CPU-s 0.0 0.044253 11.78MiB/CPU-s 558.14KiB/CPU-s 7.2KiB/CPU-s 0.0 0.046248 False False
datadog_agent_remap_datadog_logs 163.86KiB/CPU-s 0.47 100.00% 33.82MiB/CPU-s 1.15MiB/CPU-s 15.23KiB/CPU-s 0.0 0.034076 33.98MiB/CPU-s 1.2MiB/CPU-s 15.91KiB/CPU-s 0.0 0.035445 False False
http_to_http_acks 22.35KiB/CPU-s 0.42 33.78% 5.23MiB/CPU-s 2.72MiB/CPU-s 35.99KiB/CPU-s 0.0 0.520306 5.26MiB/CPU-s 2.75MiB/CPU-s 36.35KiB/CPU-s 0.0 0.523307 True False
http_to_http_noack 10.32KiB/CPU-s 0.07 96.42% 13.61MiB/CPU-s 328.52KiB/CPU-s 4.24KiB/CPU-s 0.0 0.023567 13.62MiB/CPU-s 192.77KiB/CPU-s 2.49KiB/CPU-s 0.0 0.013819 False False
splunk_hec_to_splunk_hec_logs_noack 5.59KiB/CPU-s 0.04 84.41% 13.62MiB/CPU-s 238.56KiB/CPU-s 3.08KiB/CPU-s 0.0 0.017104 13.63MiB/CPU-s 190.68KiB/CPU-s 2.46KiB/CPU-s 0.0 0.013665 False False
enterprise_http_to_http 3.36KiB/CPU-s 0.02 44.01% 13.61MiB/CPU-s 328.05KiB/CPU-s 4.23KiB/CPU-s 0.0 0.023534 13.62MiB/CPU-s 302.85KiB/CPU-s 3.91KiB/CPU-s 0.0 0.02172 False False
splunk_hec_to_splunk_hec_logs_acks 2.77KiB/CPU-s 0.02 34.66% 13.62MiB/CPU-s 354.92KiB/CPU-s 4.58KiB/CPU-s 0.0 0.025455 13.62MiB/CPU-s 321.47KiB/CPU-s 4.15KiB/CPU-s 0.0 0.023052 False False
splunk_hec_indexer_ack_blackhole 1.18KiB/CPU-s 0.01 20.07% 13.62MiB/CPU-s 256.44KiB/CPU-s 3.31KiB/CPU-s 0.0 0.01839 13.62MiB/CPU-s 252.76KiB/CPU-s 3.26KiB/CPU-s 0.0 0.018125 False False
fluent_elasticsearch -99.84B/CPU-s -0.0 14.20% 45.41MiB/CPU-s 30.32KiB/CPU-s 396.49B/CPU-s 0.0 0.000652 45.41MiB/CPU-s 30.05KiB/CPU-s 392.83B/CPU-s 0.0 0.000646 False False
file_to_blackhole -10.2KiB/CPU-s -0.02 35.38% 54.5MiB/CPU-s 1.18MiB/CPU-s 15.54KiB/CPU-s 0.0 0.021592 54.49MiB/CPU-s 1.2MiB/CPU-s 15.89KiB/CPU-s 0.0 0.022089 False False
http_to_http_json -11.65KiB/CPU-s -0.08 99.18% 13.61MiB/CPU-s 232.95KiB/CPU-s 3.01KiB/CPU-s 0.0 0.01671 13.6MiB/CPU-s 249.53KiB/CPU-s 3.22KiB/CPU-s 0.0 0.017914 False False
datadog_agent_remap_blackhole_acks -36.59KiB/CPU-s -0.11 99.86% 31.89MiB/CPU-s 585.75KiB/CPU-s 7.56KiB/CPU-s 0.0 0.017937 31.85MiB/CPU-s 669.42KiB/CPU-s 8.64KiB/CPU-s 0.0 0.020522 False False
syslog_loki -15.36KiB/CPU-s -0.17 99.22% 8.72MiB/CPU-s 377.73KiB/CPU-s 4.87KiB/CPU-s 0.0 0.042317 8.7MiB/CPU-s 238.88KiB/CPU-s 3.08KiB/CPU-s 0.0 0.026808 False False
datadog_agent_remap_datadog_logs_acks -81.43KiB/CPU-s -0.24 100.00% 33.53MiB/CPU-s 1.09MiB/CPU-s 14.46KiB/CPU-s 0.0 0.032626 33.45MiB/CPU-s 1.01MiB/CPU-s 13.38KiB/CPU-s 0.0 0.03025 False False
syslog_splunk_hec_logs -153.57KiB/CPU-s -1.65 100.00% 9.11MiB/CPU-s 142.97KiB/CPU-s 1.85KiB/CPU-s 0.0 0.01532 8.96MiB/CPU-s 252.67KiB/CPU-s 3.26KiB/CPU-s 0.0 0.027527 False False
syslog_regex_logs2metric_ddmetrics -111.05KiB/CPU-s -2.88 100.00% 3.77MiB/CPU-s 442.62KiB/CPU-s 5.71KiB/CPU-s 0.0 0.114714 3.66MiB/CPU-s 402.59KiB/CPU-s 5.2KiB/CPU-s 0.0 0.107432 True False
syslog_humio_logs -310.45KiB/CPU-s -3.27 100.00% 9.28MiB/CPU-s 152.45KiB/CPU-s 1.97KiB/CPU-s 0.0 0.016047 8.97MiB/CPU-s 324.23KiB/CPU-s 4.19KiB/CPU-s 0.0 0.035282 False False
syslog_log2metric_splunk_hec_metrics -323.72KiB/CPU-s -3.3 100.00% 9.59MiB/CPU-s 257.12KiB/CPU-s 3.32KiB/CPU-s 0.0 0.026175 9.28MiB/CPU-s 435.36KiB/CPU-s 5.62KiB/CPU-s 0.0 0.04583 False False

github-actions[bot] avatar Jan 18 '23 19:01 github-actions[bot]

Regression Test Results

Run ID: b43078a0-c70b-4cc7-9343-9d5375d1ebc0
Baseline: 614961588662ac4349fae9c3df6405c219a07107
Comparison: 9a409591e9ee2182763e2cbea0b4c096ab90a56f
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
socket_to_socket_blackhole 407.91KiB/CPU-s 3.14 100.00% 12.7MiB/CPU-s 760.6KiB/CPU-s 9.82KiB/CPU-s 0.0 0.058474 13.1MiB/CPU-s 466.31KiB/CPU-s 6.02KiB/CPU-s 0.0 0.03476 False False
syslog_regex_logs2metric_ddmetrics 68.31KiB/CPU-s 1.94 100.00% 3.43MiB/CPU-s 387.82KiB/CPU-s 5.01KiB/CPU-s 0.0 0.110293 3.5MiB/CPU-s 426.23KiB/CPU-s 5.5KiB/CPU-s 0.0 0.118908 True False
otlp_http_to_blackhole 29.85KiB/CPU-s 1.92 100.00% 1.52MiB/CPU-s 116.96KiB/CPU-s 1.51KiB/CPU-s 0.0 0.075082 1.55MiB/CPU-s 116.87KiB/CPU-s 1.51KiB/CPU-s 0.0 0.073614 False False
syslog_log2metric_humio_metrics 101.04KiB/CPU-s 1.6 100.00% 6.15MiB/CPU-s 110.67KiB/CPU-s 1.43KiB/CPU-s 0.0 0.017566 6.25MiB/CPU-s 125.68KiB/CPU-s 1.62KiB/CPU-s 0.0 0.019633 False False
datadog_agent_remap_blackhole 504.37KiB/CPU-s 1.6 100.00% 30.82MiB/CPU-s 861.77KiB/CPU-s 11.13KiB/CPU-s 0.0 0.027307 31.31MiB/CPU-s 822.61KiB/CPU-s 10.62KiB/CPU-s 0.0 0.025656 False False
splunk_hec_route_s3 75.42KiB/CPU-s 0.64 100.00% 11.44MiB/CPU-s 818.14KiB/CPU-s 10.55KiB/CPU-s 0.0 0.06986 11.51MiB/CPU-s 569.22KiB/CPU-s 7.34KiB/CPU-s 0.0 0.048293 False False
http_to_http_acks 29.42KiB/CPU-s 0.55 43.45% 5.22MiB/CPU-s 2.74MiB/CPU-s 36.17KiB/CPU-s 0.0 0.523788 5.25MiB/CPU-s 2.74MiB/CPU-s 36.22KiB/CPU-s 0.0 0.521707 True False
otlp_grpc_to_blackhole 3.71KiB/CPU-s 0.35 100.00% 1.04MiB/CPU-s 38.08KiB/CPU-s 503.39B/CPU-s 0.0 0.035782 1.04MiB/CPU-s 48.22KiB/CPU-s 637.18B/CPU-s 0.0 0.04515 False False
splunk_hec_to_splunk_hec_logs_noack 7.63KiB/CPU-s 0.05 94.00% 13.62MiB/CPU-s 250.42KiB/CPU-s 3.23KiB/CPU-s 0.0 0.017957 13.62MiB/CPU-s 189.77KiB/CPU-s 2.45KiB/CPU-s 0.0 0.013601 False False
http_to_http_noack 3.37KiB/CPU-s 0.02 44.49% 13.61MiB/CPU-s 327.25KiB/CPU-s 4.22KiB/CPU-s 0.0 0.023476 13.62MiB/CPU-s 297.53KiB/CPU-s 3.84KiB/CPU-s 0.0 0.021339 False False
datadog_agent_remap_blackhole_acks 6.92KiB/CPU-s 0.02 24.09% 31.08MiB/CPU-s 946.85KiB/CPU-s 12.23KiB/CPU-s 0.0 0.029746 31.09MiB/CPU-s 1.43MiB/CPU-s 18.97KiB/CPU-s 0.0 0.046147 False False
enterprise_http_to_http 2.1KiB/CPU-s 0.02 41.79% 13.62MiB/CPU-s 224.46KiB/CPU-s 2.9KiB/CPU-s 0.0 0.016093 13.62MiB/CPU-s 191.83KiB/CPU-s 2.48KiB/CPU-s 0.0 0.013752 False False
splunk_hec_indexer_ack_blackhole -307.59B/CPU-s -0.0 5.39% 13.62MiB/CPU-s 242.84KiB/CPU-s 3.13KiB/CPU-s 0.0 0.017413 13.62MiB/CPU-s 244.21KiB/CPU-s 3.15KiB/CPU-s 0.0 0.017512 False False
fluent_elasticsearch -227.83B/CPU-s -0.0 31.72% 45.41MiB/CPU-s 30.34KiB/CPU-s 396.56B/CPU-s 0.0 0.000652 45.41MiB/CPU-s 29.98KiB/CPU-s 391.97B/CPU-s 0.0 0.000645 False False
splunk_hec_to_splunk_hec_logs_acks -1.8KiB/CPU-s -0.01 20.85% 13.62MiB/CPU-s 347.77KiB/CPU-s 4.49KiB/CPU-s 0.0 0.024939 13.62MiB/CPU-s 397.37KiB/CPU-s 5.13KiB/CPU-s 0.0 0.0285 False False
file_to_blackhole -8.78KiB/CPU-s -0.02 31.56% 54.49MiB/CPU-s 1.06MiB/CPU-s 14.01KiB/CPU-s 0.0 0.01946 54.48MiB/CPU-s 1.25MiB/CPU-s 16.45KiB/CPU-s 0.0 0.022874 False False
http_text_to_http_json -30.29KiB/CPU-s -0.12 97.80% 23.94MiB/CPU-s 545.53KiB/CPU-s 7.04KiB/CPU-s 0.0 0.022254 23.91MiB/CPU-s 867.11KiB/CPU-s 11.19KiB/CPU-s 0.0 0.035415 False False
http_to_http_json -23.35KiB/CPU-s -0.17 99.96% 13.52MiB/CPU-s 352.55KiB/CPU-s 4.55KiB/CPU-s 0.0 0.025461 13.5MiB/CPU-s 366.42KiB/CPU-s 4.73KiB/CPU-s 0.0 0.026508 False False
datadog_agent_remap_datadog_logs_acks -100.56KiB/CPU-s -0.29 100.00% 33.58MiB/CPU-s 1.2MiB/CPU-s 15.8KiB/CPU-s 0.0 0.035592 33.48MiB/CPU-s 1.21MiB/CPU-s 16.04KiB/CPU-s 0.0 0.036244 False False
syslog_loki -47.8KiB/CPU-s -0.52 100.00% 8.9MiB/CPU-s 258.58KiB/CPU-s 3.34KiB/CPU-s 0.0 0.028365 8.86MiB/CPU-s 101.9KiB/CPU-s 1.32KiB/CPU-s 0.0 0.011237 False False
syslog_splunk_hec_logs -63.3KiB/CPU-s -0.69 100.00% 8.9MiB/CPU-s 373.17KiB/CPU-s 4.82KiB/CPU-s 0.0 0.040951 8.84MiB/CPU-s 271.91KiB/CPU-s 3.51KiB/CPU-s 0.0 0.030047 False False
syslog_log2metric_splunk_hec_metrics -100.42KiB/CPU-s -1.06 100.00% 9.26MiB/CPU-s 436.89KiB/CPU-s 5.64KiB/CPU-s 0.0 0.046082 9.16MiB/CPU-s 464.98KiB/CPU-s 6.0KiB/CPU-s 0.0 0.049569 False False
datadog_agent_remap_datadog_logs -451.96KiB/CPU-s -1.3 100.00% 33.87MiB/CPU-s 1.39MiB/CPU-s 18.42KiB/CPU-s 0.0 0.041161 33.43MiB/CPU-s 1.66MiB/CPU-s 21.89KiB/CPU-s 0.0 0.049564 False False
syslog_humio_logs -182.52KiB/CPU-s -1.93 100.00% 9.22MiB/CPU-s 160.6KiB/CPU-s 2.07KiB/CPU-s 0.0 0.017001 9.05MiB/CPU-s 249.23KiB/CPU-s 3.22KiB/CPU-s 0.0 0.026903 False False

github-actions[bot] avatar Jan 18 '23 23:01 github-actions[bot]