vector icon indicating copy to clipboard operation
vector copied to clipboard

Published 20 hours ago • verified publisher icon vectordotdev

feat(sources): Add Envoy ALS source

Open wez470 opened this issue 2 years ago • 17 comments

Adds support for Envoy ALS as a source.

Structure/Notes:

  • API/Protos:lib/envoy-proto contains the necessary copied protos from Envoy. It also contains src/convert.rs, which does the heavy lifting of converting between the protos and Vector Value
  • Server: src/sources/envoy_als contains the ALS grpc server + tests
  • Tests: There is an integration that tests receiving a log from Envoy as well as a unit test that tests the conversion of the various fields set on the API
  • Docs: I've added pages for the new source atm, but likely need to do more there. I am interested in feedback before filling this out fully in case things need changing.
  • Limitations: To start, I've only implemented support for HTTP logs from Envoy (This is documented). I figure this is a fairly large chunk of work as is and TCP log support could be added in a follow up.

closes: #14027

wez470 avatar Jan 05 '23 17:01 wez470

Deploy Preview for vrl-playground ready!

Name Link
Latest commit 0199203c8a7a04e34597a2bef1dde1e27486a8a8
Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/63ec10cc78f60400083689ae
Deploy Preview https://deploy-preview-15835--vrl-playground.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

netlify[bot] avatar Jan 05 '23 17:01 netlify[bot]

Deploy Preview for vector-project failed.

Name Link
Latest commit 0199203c8a7a04e34597a2bef1dde1e27486a8a8
Latest deploy log https://app.netlify.com/sites/vector-project/deploys/63ec10cdeacca8000813590c

netlify[bot] avatar Jan 05 '23 17:01 netlify[bot]

Regression Test Results

Run ID: b31f6c81-338d-4983-86d7-962de900824b
Baseline: bca52c3d3d26c3993656eee844295af8a27eb221
Comparison: 734496cc7be7bfdabf1524a57e73f2132fc179a6
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
syslog_log2metric_splunk_hec_metrics -512.44KiB/CPU-s -5.21 100.00%
syslog_regex_logs2metric_ddmetrics -219.17KiB/CPU-s -5.96 100.00%
syslog_splunk_hec_logs -735.89KiB/CPU-s -7.84 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_humio_metrics 221.66KiB/CPU-s 3.85 100.00% 5.62MiB/CPU-s 438.91KiB/CPU-s 5.67KiB/CPU-s 0.0 0.076255 5.84MiB/CPU-s 442.89KiB/CPU-s 5.72KiB/CPU-s 0.0 0.074094 False False
datadog_agent_remap_blackhole_acks 272.53KiB/CPU-s 0.89 100.00% 29.87MiB/CPU-s 1.92MiB/CPU-s 25.36KiB/CPU-s 0.0 0.064205 30.14MiB/CPU-s 873.33KiB/CPU-s 11.28KiB/CPU-s 0.0 0.028295 False False
datadog_agent_remap_datadog_logs 220.52KiB/CPU-s 0.65 100.00% 32.93MiB/CPU-s 2.62MiB/CPU-s 34.65KiB/CPU-s 0.0 0.079653 33.15MiB/CPU-s 1.35MiB/CPU-s 17.78KiB/CPU-s 0.0 0.040615 False False
otlp_http_to_blackhole 5.54KiB/CPU-s 0.37 96.97% 1.48MiB/CPU-s 128.22KiB/CPU-s 1.65KiB/CPU-s 0.0 0.084879 1.48MiB/CPU-s 151.16KiB/CPU-s 1.95KiB/CPU-s 0.0 0.099693 False False
http_to_http_acks 19.03KiB/CPU-s 0.36 28.77% 5.23MiB/CPU-s 2.74MiB/CPU-s 36.21KiB/CPU-s 0.0 0.523701 5.25MiB/CPU-s 2.78MiB/CPU-s 36.78KiB/CPU-s 0.0 0.529816 True False
enterprise_http_to_http 8.23KiB/CPU-s 0.06 95.22% 13.62MiB/CPU-s 285.82KiB/CPU-s 3.69KiB/CPU-s 0.0 0.020498 13.62MiB/CPU-s 149.03KiB/CPU-s 1.92KiB/CPU-s 0.0 0.010681 False False
http_to_http_noack 6.28KiB/CPU-s 0.05 71.78% 13.61MiB/CPU-s 353.12KiB/CPU-s 4.56KiB/CPU-s 0.0 0.025336 13.62MiB/CPU-s 282.7KiB/CPU-s 3.65KiB/CPU-s 0.0 0.020274 False False
file_to_blackhole 17.94KiB/CPU-s 0.03 51.13% 54.47MiB/CPU-s 1.55MiB/CPU-s 20.52KiB/CPU-s 0.0 0.028538 54.49MiB/CPU-s 1.2MiB/CPU-s 15.81KiB/CPU-s 0.0 0.021987 False False
splunk_hec_to_splunk_hec_logs_noack 1.95KiB/CPU-s 0.01 40.71% 13.62MiB/CPU-s 208.49KiB/CPU-s 2.69KiB/CPU-s 0.0 0.014944 13.63MiB/CPU-s 190.14KiB/CPU-s 2.45KiB/CPU-s 0.0 0.013627 False False
splunk_hec_indexer_ack_blackhole 887.34B/CPU-s 0.01 14.83% 13.62MiB/CPU-s 255.48KiB/CPU-s 3.3KiB/CPU-s 0.0 0.018321 13.62MiB/CPU-s 252.7KiB/CPU-s 3.26KiB/CPU-s 0.0 0.018121 False False
fluent_elasticsearch -157.05B/CPU-s -0.0 22.18% 45.41MiB/CPU-s 30.16KiB/CPU-s 394.16B/CPU-s 0.0 0.000648 45.41MiB/CPU-s 30.16KiB/CPU-s 394.22B/CPU-s 0.0 0.000648 False False
splunk_hec_to_splunk_hec_logs_acks -1.96KiB/CPU-s -0.01 23.47% 13.62MiB/CPU-s 349.99KiB/CPU-s 4.51KiB/CPU-s 0.0 0.025101 13.61MiB/CPU-s 367.99KiB/CPU-s 4.75KiB/CPU-s 0.0 0.026396 False False
http_to_http_json -44.46KiB/CPU-s -0.32 100.00% 13.62MiB/CPU-s 212.58KiB/CPU-s 2.74KiB/CPU-s 0.0 0.015238 13.58MiB/CPU-s 305.46KiB/CPU-s 3.94KiB/CPU-s 0.0 0.021967 False False
otlp_grpc_to_blackhole -5.55KiB/CPU-s -0.53 100.00% 1.02MiB/CPU-s 61.12KiB/CPU-s 808.04B/CPU-s 0.0 0.05836 1.02MiB/CPU-s 52.23KiB/CPU-s 690.16B/CPU-s 0.0 0.050132 False False
datadog_agent_remap_datadog_logs_acks -267.25KiB/CPU-s -0.8 100.00% 32.52MiB/CPU-s 2.41MiB/CPU-s 31.86KiB/CPU-s 0.0 0.07415 32.26MiB/CPU-s 2.37MiB/CPU-s 31.32KiB/CPU-s 0.0 0.073458 False False
datadog_agent_remap_blackhole -614.84KiB/CPU-s -1.93 100.00% 31.05MiB/CPU-s 1.26MiB/CPU-s 16.6KiB/CPU-s 0.0 0.040441 30.45MiB/CPU-s 1.38MiB/CPU-s 18.19KiB/CPU-s 0.0 0.045191 False False
socket_to_socket_blackhole -426.32KiB/CPU-s -3.12 100.00% 13.34MiB/CPU-s 618.15KiB/CPU-s 7.98KiB/CPU-s 0.0 0.045247 12.92MiB/CPU-s 676.31KiB/CPU-s 8.74KiB/CPU-s 0.0 0.051099 False False
syslog_humio_logs -336.55KiB/CPU-s -3.66 100.00% 8.98MiB/CPU-s 351.3KiB/CPU-s 4.53KiB/CPU-s 0.0 0.038219 8.65MiB/CPU-s 307.69KiB/CPU-s 3.97KiB/CPU-s 0.0 0.034747 False False
splunk_hec_route_s3 -447.59KiB/CPU-s -3.71 100.00% 11.79MiB/CPU-s 547.43KiB/CPU-s 7.06KiB/CPU-s 0.0 0.045328 11.36MiB/CPU-s 889.52KiB/CPU-s 11.48KiB/CPU-s 0.0 0.07649 False False
syslog_loki -385.16KiB/CPU-s -4.31 100.00% 8.72MiB/CPU-s 205.44KiB/CPU-s 2.65KiB/CPU-s 0.0 0.022997 8.35MiB/CPU-s 279.11KiB/CPU-s 3.6KiB/CPU-s 0.0 0.03265 False False
http_text_to_http_json -1.1MiB/CPU-s -4.43 100.00% 24.81MiB/CPU-s 1.24MiB/CPU-s 16.33KiB/CPU-s 0.0 0.049807 23.71MiB/CPU-s 1.12MiB/CPU-s 14.86KiB/CPU-s 0.0 0.047424 False False
syslog_log2metric_splunk_hec_metrics -512.44KiB/CPU-s -5.21 100.00% 9.6MiB/CPU-s 158.98KiB/CPU-s 2.05KiB/CPU-s 0.0 0.016173 9.1MiB/CPU-s 210.65KiB/CPU-s 2.72KiB/CPU-s 0.0 0.022607 False False
syslog_regex_logs2metric_ddmetrics -219.17KiB/CPU-s -5.96 100.00% 3.59MiB/CPU-s 537.25KiB/CPU-s 6.94KiB/CPU-s 0.0 0.146149 3.38MiB/CPU-s 526.31KiB/CPU-s 6.79KiB/CPU-s 0.0 0.152251 True False
syslog_splunk_hec_logs -735.89KiB/CPU-s -7.84 100.00% 9.16MiB/CPU-s 204.74KiB/CPU-s 2.64KiB/CPU-s 0.0 0.021824 8.44MiB/CPU-s 492.56KiB/CPU-s 6.36KiB/CPU-s 0.0 0.056972 False False

github-actions[bot] avatar Jan 05 '23 18:01 github-actions[bot]

Regression Test Results

Run ID: ee373886-01c5-4a1c-a12d-e6f601dc54f9
Baseline: bca52c3d3d26c3993656eee844295af8a27eb221
Comparison: 4fdc28cee1b66517860a84dafd861374a75134dd
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_humio_metrics 242.41KiB/CPU-s 4.14 100.00% 5.71MiB/CPU-s 411.23KiB/CPU-s 5.31KiB/CPU-s 0.0 0.07027 5.95MiB/CPU-s 298.23KiB/CPU-s 3.85KiB/CPU-s 0.0 0.048934 False False
http_to_http_acks 25.77KiB/CPU-s 0.48 38.50% 5.22MiB/CPU-s 2.78MiB/CPU-s 36.72KiB/CPU-s 0.0 0.531887 5.25MiB/CPU-s 2.7MiB/CPU-s 35.72KiB/CPU-s 0.0 0.514774 True False
enterprise_http_to_http 16.58KiB/CPU-s 0.12 99.85% 13.61MiB/CPU-s 374.5KiB/CPU-s 4.83KiB/CPU-s 0.0 0.026874 13.62MiB/CPU-s 150.75KiB/CPU-s 1.95KiB/CPU-s 0.0 0.010805 False False
splunk_hec_to_splunk_hec_logs_noack 2.54KiB/CPU-s 0.02 39.52% 13.62MiB/CPU-s 237.69KiB/CPU-s 3.07KiB/CPU-s 0.0 0.017042 13.62MiB/CPU-s 297.32KiB/CPU-s 3.84KiB/CPU-s 0.0 0.021313 False False
splunk_hec_to_splunk_hec_logs_acks 1.09KiB/CPU-s 0.01 12.86% 13.61MiB/CPU-s 367.39KiB/CPU-s 4.74KiB/CPU-s 0.0 0.02635 13.62MiB/CPU-s 371.17KiB/CPU-s 4.79KiB/CPU-s 0.0 0.026619 False False
file_to_blackhole 474.12B/CPU-s 0.0 1.74% 54.49MiB/CPU-s 1.11MiB/CPU-s 14.65KiB/CPU-s 0.0 0.020353 54.49MiB/CPU-s 1.16MiB/CPU-s 15.33KiB/CPU-s 0.0 0.021315 False False
fluent_elasticsearch -177.68B/CPU-s -0.0 25.32% 45.41MiB/CPU-s 29.89KiB/CPU-s 390.55B/CPU-s 0.0 0.000643 45.41MiB/CPU-s 29.68KiB/CPU-s 387.73B/CPU-s 0.0 0.000638 False False
http_to_http_noack 112.99B/CPU-s 0.0 1.72% 13.62MiB/CPU-s 279.64KiB/CPU-s 3.61KiB/CPU-s 0.0 0.020054 13.62MiB/CPU-s 281.12KiB/CPU-s 3.63KiB/CPU-s 0.0 0.020161 False False
splunk_hec_indexer_ack_blackhole 173.32B/CPU-s 0.0 2.93% 13.62MiB/CPU-s 251.21KiB/CPU-s 3.24KiB/CPU-s 0.0 0.018015 13.62MiB/CPU-s 253.99KiB/CPU-s 3.28KiB/CPU-s 0.0 0.018214 False False
http_to_http_json -5.64KiB/CPU-s -0.04 74.76% 13.59MiB/CPU-s 258.75KiB/CPU-s 3.34KiB/CPU-s 0.0 0.01859 13.59MiB/CPU-s 280.61KiB/CPU-s 3.62KiB/CPU-s 0.0 0.020169 False False
splunk_hec_route_s3 -108.96KiB/CPU-s -0.91 100.00% 11.65MiB/CPU-s 603.35KiB/CPU-s 7.78KiB/CPU-s 0.0 0.050576 11.54MiB/CPU-s 818.48KiB/CPU-s 10.56KiB/CPU-s 0.0 0.069243 False False
otlp_http_to_blackhole -23.84KiB/CPU-s -1.51 100.00% 1.54MiB/CPU-s 115.81KiB/CPU-s 1.49KiB/CPU-s 0.0 0.07351 1.52MiB/CPU-s 129.22KiB/CPU-s 1.67KiB/CPU-s 0.0 0.083279 False False
syslog_loki -137.77KiB/CPU-s -1.62 100.00% 8.3MiB/CPU-s 584.21KiB/CPU-s 7.54KiB/CPU-s 0.0 0.068705 8.17MiB/CPU-s 363.66KiB/CPU-s 4.69KiB/CPU-s 0.0 0.043472 False False
syslog_regex_logs2metric_ddmetrics -59.56KiB/CPU-s -1.67 100.00% 3.49MiB/CPU-s 362.83KiB/CPU-s 4.68KiB/CPU-s 0.0 0.101663 3.43MiB/CPU-s 288.04KiB/CPU-s 3.72KiB/CPU-s 0.0 0.082078 True False
otlp_grpc_to_blackhole -17.98KiB/CPU-s -1.7 100.00% 1.03MiB/CPU-s 50.05KiB/CPU-s 661.42B/CPU-s 0.0 0.047321 1.02MiB/CPU-s 50.91KiB/CPU-s 672.75B/CPU-s 0.0 0.048965 False False
datadog_agent_remap_blackhole -895.03KiB/CPU-s -2.87 100.00% 30.46MiB/CPU-s 1.18MiB/CPU-s 15.59KiB/CPU-s 0.0 0.038709 29.58MiB/CPU-s 1.11MiB/CPU-s 14.63KiB/CPU-s 0.0 0.037422 False False
datadog_agent_remap_blackhole_acks -984.81KiB/CPU-s -3.19 100.00% 30.18MiB/CPU-s 1.12MiB/CPU-s 14.79KiB/CPU-s 0.0 0.037056 29.22MiB/CPU-s 1.07MiB/CPU-s 14.21KiB/CPU-s 0.0 0.036786 False False
datadog_agent_remap_datadog_logs -1.24MiB/CPU-s -3.73 100.00% 33.28MiB/CPU-s 1.37MiB/CPU-s 18.08KiB/CPU-s 0.0 0.041127 32.04MiB/CPU-s 1.35MiB/CPU-s 17.9KiB/CPU-s 0.0 0.042287 False False
http_text_to_http_json -957.44KiB/CPU-s -3.76 100.00% 24.86MiB/CPU-s 732.47KiB/CPU-s 9.46KiB/CPU-s 0.0 0.028766 23.93MiB/CPU-s 617.18KiB/CPU-s 7.96KiB/CPU-s 0.0 0.025185 False False
datadog_agent_remap_datadog_logs_acks -1.35MiB/CPU-s -3.89 100.00% 34.67MiB/CPU-s 992.82KiB/CPU-s 12.81KiB/CPU-s 0.0 0.027963 33.32MiB/CPU-s 699.06KiB/CPU-s 9.02KiB/CPU-s 0.0 0.020487 False False
syslog_log2metric_splunk_hec_metrics -398.05KiB/CPU-s -4.3 100.00% 9.05MiB/CPU-s 498.71KiB/CPU-s 6.44KiB/CPU-s 0.0 0.053808 8.66MiB/CPU-s 599.58KiB/CPU-s 7.74KiB/CPU-s 0.0 0.067595 False False
syslog_humio_logs -423.14KiB/CPU-s -4.48 100.00% 9.22MiB/CPU-s 278.68KiB/CPU-s 3.6KiB/CPU-s 0.0 0.029518 8.81MiB/CPU-s 98.94KiB/CPU-s 1.28KiB/CPU-s 0.0 0.010972 False False
syslog_splunk_hec_logs -449.3KiB/CPU-s -4.77 100.00% 9.2MiB/CPU-s 212.53KiB/CPU-s 2.74KiB/CPU-s 0.0 0.022552 8.76MiB/CPU-s 187.02KiB/CPU-s 2.41KiB/CPU-s 0.0 0.020839 False False
socket_to_socket_blackhole -682.17KiB/CPU-s -4.95 100.00% 13.45MiB/CPU-s 365.08KiB/CPU-s 4.71KiB/CPU-s 0.0 0.026498 12.79MiB/CPU-s 797.3KiB/CPU-s 10.3KiB/CPU-s 0.0 0.060884 False False

github-actions[bot] avatar Jan 05 '23 18:01 github-actions[bot]

Discussed with my co-workers and something we want to do is to add the node info to each log event so that each log can be properly identified. I am going to update to do this. Second, I'm going to remove the extensions field on the node object. It doesn't provide a lot of value for logs (indicates every extensions available on the particular envoy instance that connected) and is a huge field to be logging (it's a list with > 200 object entries).

wez470 avatar Jan 05 '23 19:01 wez470

Regression Test Results

Run ID: b9046488-289c-4826-8fad-67d68b88d4ef
Baseline: bca52c3d3d26c3993656eee844295af8a27eb221
Comparison: 26c54ea4f27b5479a4b8516ab29361b57625804b
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
syslog_splunk_hec_logs -526.37KiB/CPU-s -5.57 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_humio_metrics 275.55KiB/CPU-s 4.57 100.00% 5.89MiB/CPU-s 263.85KiB/CPU-s 3.41KiB/CPU-s 0.0 0.043755 6.16MiB/CPU-s 99.72KiB/CPU-s 1.29KiB/CPU-s 0.0 0.015815 False False
http_to_http_acks 65.84KiB/CPU-s 1.22 80.52% 5.26MiB/CPU-s 2.7MiB/CPU-s 35.63KiB/CPU-s 0.0 0.512727 5.32MiB/CPU-s 2.74MiB/CPU-s 36.18KiB/CPU-s 0.0 0.514428 True False
datadog_agent_remap_datadog_logs_acks 234.13KiB/CPU-s 0.68 100.00% 33.48MiB/CPU-s 1.64MiB/CPU-s 21.73KiB/CPU-s 0.0 0.049123 33.71MiB/CPU-s 794.73KiB/CPU-s 10.26KiB/CPU-s 0.0 0.023022 False False
http_to_http_json 47.61KiB/CPU-s 0.34 100.00% 13.58MiB/CPU-s 286.45KiB/CPU-s 3.7KiB/CPU-s 0.0 0.020604 13.62MiB/CPU-s 205.03KiB/CPU-s 2.64KiB/CPU-s 0.0 0.014697 False False
enterprise_http_to_http 16.38KiB/CPU-s 0.12 99.84% 13.61MiB/CPU-s 373.62KiB/CPU-s 4.82KiB/CPU-s 0.0 0.026811 13.62MiB/CPU-s 149.41KiB/CPU-s 1.93KiB/CPU-s 0.0 0.010709 False False
splunk_hec_to_splunk_hec_logs_noack 2.29KiB/CPU-s 0.02 46.56% 13.62MiB/CPU-s 211.54KiB/CPU-s 2.73KiB/CPU-s 0.0 0.015163 13.63MiB/CPU-s 191.59KiB/CPU-s 2.47KiB/CPU-s 0.0 0.013731 False False
splunk_hec_to_splunk_hec_logs_acks 1.51KiB/CPU-s 0.01 17.83% 13.61MiB/CPU-s 369.02KiB/CPU-s 4.76KiB/CPU-s 0.0 0.026467 13.62MiB/CPU-s 367.17KiB/CPU-s 4.74KiB/CPU-s 0.0 0.026331 False False
fluent_elasticsearch 44.04B/CPU-s 0.0 1.87% 45.41MiB/CPU-s 140.91KiB/CPU-s 1.8KiB/CPU-s 0.0 0.00303 45.41MiB/CPU-s 29.78KiB/CPU-s 389.32B/CPU-s 0.0 0.00064 False False
http_to_http_noack 210.95B/CPU-s 0.0 2.93% 13.61MiB/CPU-s 308.59KiB/CPU-s 3.98KiB/CPU-s 0.0 0.022134 13.61MiB/CPU-s 305.8KiB/CPU-s 3.95KiB/CPU-s 0.0 0.021934 False False
splunk_hec_indexer_ack_blackhole -2.02KiB/CPU-s -0.01 34.28% 13.62MiB/CPU-s 243.82KiB/CPU-s 3.15KiB/CPU-s 0.0 0.017483 13.62MiB/CPU-s 255.34KiB/CPU-s 3.29KiB/CPU-s 0.0 0.018312 False False
file_to_blackhole -15.03KiB/CPU-s -0.03 49.09% 54.5MiB/CPU-s 1.21MiB/CPU-s 15.95KiB/CPU-s 0.0 0.022155 54.48MiB/CPU-s 1.23MiB/CPU-s 16.23KiB/CPU-s 0.0 0.022574 False False
otlp_http_to_blackhole -11.96KiB/CPU-s -0.76 100.00% 1.53MiB/CPU-s 129.11KiB/CPU-s 1.67KiB/CPU-s 0.0 0.082595 1.51MiB/CPU-s 127.05KiB/CPU-s 1.64KiB/CPU-s 0.0 0.081905 False False
splunk_hec_route_s3 -132.91KiB/CPU-s -1.1 100.00% 11.79MiB/CPU-s 584.25KiB/CPU-s 7.54KiB/CPU-s 0.0 0.048393 11.66MiB/CPU-s 565.98KiB/CPU-s 7.3KiB/CPU-s 0.0 0.047402 False False
datadog_agent_remap_blackhole -410.83KiB/CPU-s -1.27 100.00% 31.49MiB/CPU-s 1.23MiB/CPU-s 16.26KiB/CPU-s 0.0 0.039068 31.09MiB/CPU-s 682.69KiB/CPU-s 8.81KiB/CPU-s 0.0 0.02144 False False
syslog_log2metric_splunk_hec_metrics -190.2KiB/CPU-s -2.01 100.00% 9.26MiB/CPU-s 375.75KiB/CPU-s 4.85KiB/CPU-s 0.0 0.039637 9.07MiB/CPU-s 240.6KiB/CPU-s 3.11KiB/CPU-s 0.0 0.025901 False False
syslog_loki -251.31KiB/CPU-s -2.87 100.00% 8.55MiB/CPU-s 447.57KiB/CPU-s 5.78KiB/CPU-s 0.0 0.051126 8.3MiB/CPU-s 352.25KiB/CPU-s 4.55KiB/CPU-s 0.0 0.041426 False False
otlp_grpc_to_blackhole -42.78KiB/CPU-s -3.99 100.00% 1.05MiB/CPU-s 36.45KiB/CPU-s 481.8B/CPU-s 0.0 0.033952 1.01MiB/CPU-s 51.5KiB/CPU-s 680.48B/CPU-s 0.0 0.049959 False False
datadog_agent_remap_datadog_logs -1.44MiB/CPU-s -4.16 100.00% 34.55MiB/CPU-s 1.27MiB/CPU-s 16.77KiB/CPU-s 0.0 0.036751 33.11MiB/CPU-s 1.64MiB/CPU-s 21.61KiB/CPU-s 0.0 0.049406 False False
syslog_humio_logs -388.86KiB/CPU-s -4.17 100.00% 9.12MiB/CPU-s 332.97KiB/CPU-s 4.3KiB/CPU-s 0.0 0.035665 8.74MiB/CPU-s 132.61KiB/CPU-s 1.71KiB/CPU-s 0.0 0.014822 False False
socket_to_socket_blackhole -589.13KiB/CPU-s -4.3 100.00% 13.38MiB/CPU-s 464.64KiB/CPU-s 6.0KiB/CPU-s 0.0 0.033916 12.8MiB/CPU-s 325.5KiB/CPU-s 4.2KiB/CPU-s 0.0 0.024827 False False
http_text_to_http_json -1.09MiB/CPU-s -4.39 100.00% 24.74MiB/CPU-s 951.97KiB/CPU-s 12.29KiB/CPU-s 0.0 0.037575 23.65MiB/CPU-s 545.17KiB/CPU-s 7.03KiB/CPU-s 0.0 0.022507 False False
datadog_agent_remap_blackhole_acks -1.41MiB/CPU-s -4.44 100.00% 31.68MiB/CPU-s 1.03MiB/CPU-s 13.65KiB/CPU-s 0.0 0.032576 30.27MiB/CPU-s 1.41MiB/CPU-s 18.59KiB/CPU-s 0.0 0.046466 False False
syslog_regex_logs2metric_ddmetrics -180.84KiB/CPU-s -4.8 100.00% 3.68MiB/CPU-s 494.44KiB/CPU-s 6.38KiB/CPU-s 0.0 0.131096 3.51MiB/CPU-s 391.99KiB/CPU-s 5.06KiB/CPU-s 0.0 0.109168 True False
syslog_splunk_hec_logs -526.37KiB/CPU-s -5.57 100.00% 9.23MiB/CPU-s 121.26KiB/CPU-s 1.57KiB/CPU-s 0.0 0.012822 8.72MiB/CPU-s 255.07KiB/CPU-s 3.29KiB/CPU-s 0.0 0.02856 False False

github-actions[bot] avatar Jan 05 '23 20:01 github-actions[bot]

Regression Test Results

Run ID: 916f02a9-ba18-4a1b-b4c6-5d0618170bc4
Baseline: 65cb0c5e031127ee8fff1eb309ab0044843043a4
Comparison: 0f7a32597569b20677d4ffeb4ae2262add022c24
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
syslog_log2metric_splunk_hec_metrics -609.8KiB/CPU-s -6.31 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
datadog_agent_remap_blackhole 531.15KiB/CPU-s 1.7 100.00% 30.45MiB/CPU-s 1.92MiB/CPU-s 25.38KiB/CPU-s 0.0 0.063004 30.97MiB/CPU-s 843.31KiB/CPU-s 10.89KiB/CPU-s 0.0 0.026589 False False
http_to_http_acks 9.59KiB/CPU-s 0.18 15.18% 5.19MiB/CPU-s 2.62MiB/CPU-s 34.57KiB/CPU-s 0.0 0.503956 5.2MiB/CPU-s 2.74MiB/CPU-s 36.24KiB/CPU-s 0.0 0.527454 True False
enterprise_http_to_http 8.44KiB/CPU-s 0.06 89.50% 13.61MiB/CPU-s 334.61KiB/CPU-s 4.32KiB/CPU-s 0.0 0.024005 13.62MiB/CPU-s 225.44KiB/CPU-s 2.91KiB/CPU-s 0.0 0.016163 False False
splunk_hec_to_splunk_hec_logs_noack 3.44KiB/CPU-s 0.02 64.40% 13.62MiB/CPU-s 215.37KiB/CPU-s 2.78KiB/CPU-s 0.0 0.015439 13.63MiB/CPU-s 192.41KiB/CPU-s 2.48KiB/CPU-s 0.0 0.013789 False False
splunk_hec_indexer_ack_blackhole 1.83KiB/CPU-s 0.01 31.12% 13.62MiB/CPU-s 253.83KiB/CPU-s 3.27KiB/CPU-s 0.0 0.018204 13.62MiB/CPU-s 247.73KiB/CPU-s 3.2KiB/CPU-s 0.0 0.017764 False False
file_to_blackhole 1.28KiB/CPU-s 0.0 4.80% 54.5MiB/CPU-s 1.15MiB/CPU-s 15.21KiB/CPU-s 0.0 0.021133 54.5MiB/CPU-s 1.13MiB/CPU-s 14.91KiB/CPU-s 0.0 0.020718 False False
fluent_elasticsearch -340.5B/CPU-s -0.0 46.25% 45.41MiB/CPU-s 29.94KiB/CPU-s 391.41B/CPU-s 0.0 0.000644 45.41MiB/CPU-s 29.82KiB/CPU-s 389.62B/CPU-s 0.0 0.000641 False False
splunk_hec_to_splunk_hec_logs_acks -257.21B/CPU-s -0.0 2.98% 13.61MiB/CPU-s 367.27KiB/CPU-s 4.74KiB/CPU-s 0.0 0.026341 13.61MiB/CPU-s 370.3KiB/CPU-s 4.78KiB/CPU-s 0.0 0.026559 False False
http_to_http_json -1.02KiB/CPU-s -0.01 21.28% 13.62MiB/CPU-s 208.48KiB/CPU-s 2.69KiB/CPU-s 0.0 0.014943 13.62MiB/CPU-s 205.88KiB/CPU-s 2.66KiB/CPU-s 0.0 0.014759 False False
http_to_http_noack -1.99KiB/CPU-s -0.01 28.69% 13.62MiB/CPU-s 283.1KiB/CPU-s 3.65KiB/CPU-s 0.0 0.020303 13.61MiB/CPU-s 307.92KiB/CPU-s 3.97KiB/CPU-s 0.0 0.022086 False False
datadog_agent_remap_datadog_logs_acks -285.84KiB/CPU-s -0.83 100.00% 33.82MiB/CPU-s 1.7MiB/CPU-s 22.41KiB/CPU-s 0.0 0.050155 33.54MiB/CPU-s 1.04MiB/CPU-s 13.71KiB/CPU-s 0.0 0.030939 False False
syslog_log2metric_humio_metrics -82.7KiB/CPU-s -1.32 100.00% 6.1MiB/CPU-s 267.95KiB/CPU-s 3.46KiB/CPU-s 0.0 0.042919 6.02MiB/CPU-s 376.53KiB/CPU-s 4.86KiB/CPU-s 0.0 0.06112 False False
syslog_regex_logs2metric_ddmetrics -55.34KiB/CPU-s -1.54 100.00% 3.51MiB/CPU-s 335.78KiB/CPU-s 4.33KiB/CPU-s 0.0 0.093373 3.46MiB/CPU-s 315.23KiB/CPU-s 4.07KiB/CPU-s 0.0 0.089029 False False
splunk_hec_route_s3 -186.28KiB/CPU-s -1.56 100.00% 11.64MiB/CPU-s 621.01KiB/CPU-s 8.01KiB/CPU-s 0.0 0.052102 11.46MiB/CPU-s 776.52KiB/CPU-s 10.02KiB/CPU-s 0.0 0.066183 False False
datadog_agent_remap_blackhole_acks -712.55KiB/CPU-s -2.23 100.00% 31.27MiB/CPU-s 1.34MiB/CPU-s 17.7KiB/CPU-s 0.0 0.04279 30.57MiB/CPU-s 1.37MiB/CPU-s 18.07KiB/CPU-s 0.0 0.044691 False False
otlp_http_to_blackhole -41.98KiB/CPU-s -2.68 100.00% 1.53MiB/CPU-s 110.94KiB/CPU-s 1.43KiB/CPU-s 0.0 0.07093 1.49MiB/CPU-s 133.82KiB/CPU-s 1.73KiB/CPU-s 0.0 0.087921 False False
datadog_agent_remap_datadog_logs -1.01MiB/CPU-s -3.03 100.00% 33.52MiB/CPU-s 1.29MiB/CPU-s 16.99KiB/CPU-s 0.0 0.038374 32.51MiB/CPU-s 1.04MiB/CPU-s 13.79KiB/CPU-s 0.0 0.032109 False False
syslog_loki -287.44KiB/CPU-s -3.37 100.00% 8.33MiB/CPU-s 537.23KiB/CPU-s 6.93KiB/CPU-s 0.0 0.063005 8.05MiB/CPU-s 409.5KiB/CPU-s 5.29KiB/CPU-s 0.0 0.049701 False False
syslog_splunk_hec_logs -401.65KiB/CPU-s -4.42 100.00% 8.87MiB/CPU-s 158.57KiB/CPU-s 2.05KiB/CPU-s 0.0 0.017457 8.48MiB/CPU-s 269.29KiB/CPU-s 3.47KiB/CPU-s 0.0 0.031018 False False
syslog_humio_logs -447.64KiB/CPU-s -4.81 100.00% 9.09MiB/CPU-s 273.65KiB/CPU-s 3.53KiB/CPU-s 0.0 0.029392 8.65MiB/CPU-s 131.37KiB/CPU-s 1.7KiB/CPU-s 0.0 0.014822 False False
socket_to_socket_blackhole -670.85KiB/CPU-s -4.84 100.00% 13.52MiB/CPU-s 203.9KiB/CPU-s 2.63KiB/CPU-s 0.0 0.014722 12.87MiB/CPU-s 303.94KiB/CPU-s 3.92KiB/CPU-s 0.0 0.023062 False False
otlp_grpc_to_blackhole -52.04KiB/CPU-s -4.86 100.00% 1.05MiB/CPU-s 33.3KiB/CPU-s 440.16B/CPU-s 0.0 0.031064 1019.7KiB/CPU-s 49.61KiB/CPU-s 655.56B/CPU-s 0.0 0.048643 False False
http_text_to_http_json -1.25MiB/CPU-s -4.98 100.00% 25.02MiB/CPU-s 605.03KiB/CPU-s 7.81KiB/CPU-s 0.0 0.023609 23.78MiB/CPU-s 644.2KiB/CPU-s 8.31KiB/CPU-s 0.0 0.026454 False False
syslog_log2metric_splunk_hec_metrics -609.8KiB/CPU-s -6.31 100.00% 9.44MiB/CPU-s 419.22KiB/CPU-s 5.41KiB/CPU-s 0.0 0.043354 8.85MiB/CPU-s 430.79KiB/CPU-s 5.56KiB/CPU-s 0.0 0.047549 False False

github-actions[bot] avatar Jan 06 '23 18:01 github-actions[bot]

Hi @wez470 , thanks for this contribution!

I see the PR is in draft status. Let me know when it's ready for a review and I'll dive in 🙂

neuronull avatar Jan 06 '23 23:01 neuronull

Sure, I was looking to get some initial feedback. Moved to ready for review to kick things off.

wez470 avatar Jan 09 '23 17:01 wez470

I've added @StephenWakely as a reviewer, as this source seems to have similarities to the dnstap source.

neuronull avatar Jan 09 '23 21:01 neuronull

Thanks for the initial review @neuronull. There is one other thing I wanted to ask about for the cue docs. I have initially just documented the top level fields and linked to envoy docs to further dive into those structs. Is that acceptable, or do we need to document every field? I'd initially gone this route b/c there are a lot of subfields and the structure just follows the envoy proto fields.

wez470 avatar Jan 10 '23 19:01 wez470

Regression Test Results

Run ID: c8e7cbd6-33ce-4e28-97f2-b2e10c8292a8
Baseline: 4b2b7130e71d82fbbebce91cfab4f0c5afbfa2fe
Comparison: fd286bf1b0e2fa49778e90e241b05afe39b0e3ea
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
syslog_loki -681.59KiB/CPU-s -7.58 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
datadog_agent_remap_blackhole 472.42KiB/CPU-s 1.53 100.00% 30.22MiB/CPU-s 1.51MiB/CPU-s 19.96KiB/CPU-s 0.0 0.049947 30.68MiB/CPU-s 984.22KiB/CPU-s 12.71KiB/CPU-s 0.0 0.031328 False False
otlp_http_to_blackhole 16.66KiB/CPU-s 1.08 100.00% 1.5MiB/CPU-s 125.08KiB/CPU-s 1.61KiB/CPU-s 0.0 0.081448 1.52MiB/CPU-s 126.38KiB/CPU-s 1.63KiB/CPU-s 0.0 0.081409 False False
datadog_agent_remap_blackhole_acks 204.77KiB/CPU-s 0.66 100.00% 30.21MiB/CPU-s 1.85MiB/CPU-s 24.46KiB/CPU-s 0.0 0.061243 30.41MiB/CPU-s 1.76MiB/CPU-s 23.25KiB/CPU-s 0.0 0.057824 False False
http_to_http_acks 20.01KiB/CPU-s 0.38 30.34% 5.21MiB/CPU-s 2.78MiB/CPU-s 36.72KiB/CPU-s 0.0 0.533793 5.23MiB/CPU-s 2.71MiB/CPU-s 35.84KiB/CPU-s 0.0 0.518904 True False
splunk_hec_to_splunk_hec_logs_noack 5.64KiB/CPU-s 0.04 84.97% 13.62MiB/CPU-s 234.71KiB/CPU-s 3.03KiB/CPU-s 0.0 0.016828 13.63MiB/CPU-s 193.12KiB/CPU-s 2.49KiB/CPU-s 0.0 0.01384 False False
http_to_http_noack 6.18KiB/CPU-s 0.04 67.95% 13.61MiB/CPU-s 370.32KiB/CPU-s 4.78KiB/CPU-s 0.0 0.026573 13.61MiB/CPU-s 308.17KiB/CPU-s 3.98KiB/CPU-s 0.0 0.022104 False False
otlp_grpc_to_blackhole 263.61B/CPU-s 0.02 23.71% 1.02MiB/CPU-s 39.99KiB/CPU-s 528.63B/CPU-s 0.0 0.038209 1.02MiB/CPU-s 52.66KiB/CPU-s 695.68B/CPU-s 0.0 0.0503 False False
splunk_hec_indexer_ack_blackhole 839.73B/CPU-s 0.01 14.06% 13.62MiB/CPU-s 257.22KiB/CPU-s 3.32KiB/CPU-s 0.0 0.018447 13.62MiB/CPU-s 250.3KiB/CPU-s 3.23KiB/CPU-s 0.0 0.01795 False False
file_to_blackhole 3.55KiB/CPU-s 0.01 12.72% 54.49MiB/CPU-s 1.18MiB/CPU-s 15.57KiB/CPU-s 0.0 0.021631 54.5MiB/CPU-s 1.19MiB/CPU-s 15.75KiB/CPU-s 0.0 0.021889 False False
enterprise_http_to_http 80.43B/CPU-s 0.0 1.79% 13.62MiB/CPU-s 189.98KiB/CPU-s 2.45KiB/CPU-s 0.0 0.013618 13.62MiB/CPU-s 193.5KiB/CPU-s 2.5KiB/CPU-s 0.0 0.01387 False False
fluent_elasticsearch -561.43B/CPU-s -0.0 68.96% 45.41MiB/CPU-s 29.92KiB/CPU-s 391.04B/CPU-s 0.0 0.000643 45.41MiB/CPU-s 29.97KiB/CPU-s 391.68B/CPU-s 0.0 0.000644 False False
splunk_hec_to_splunk_hec_logs_acks -2.08KiB/CPU-s -0.01 25.81% 13.62MiB/CPU-s 338.13KiB/CPU-s 4.36KiB/CPU-s 0.0 0.024246 13.62MiB/CPU-s 354.64KiB/CPU-s 4.58KiB/CPU-s 0.0 0.025434 False False
syslog_regex_logs2metric_ddmetrics -4.25KiB/CPU-s -0.11 38.39% 3.7MiB/CPU-s 506.88KiB/CPU-s 6.54KiB/CPU-s 0.0 0.133844 3.69MiB/CPU-s 416.47KiB/CPU-s 5.38KiB/CPU-s 0.0 0.110092 True False
syslog_log2metric_humio_metrics -57.72KiB/CPU-s -0.97 100.00% 5.81MiB/CPU-s 412.73KiB/CPU-s 5.33KiB/CPU-s 0.0 0.06936 5.75MiB/CPU-s 398.09KiB/CPU-s 5.14KiB/CPU-s 0.0 0.067555 False False
http_to_http_json -148.8KiB/CPU-s -1.07 100.00% 13.61MiB/CPU-s 239.33KiB/CPU-s 3.09KiB/CPU-s 0.0 0.017169 13.47MiB/CPU-s 359.93KiB/CPU-s 4.64KiB/CPU-s 0.0 0.026098 False False
socket_to_socket_blackhole -206.06KiB/CPU-s -1.52 100.00% 13.25MiB/CPU-s 283.78KiB/CPU-s 3.66KiB/CPU-s 0.0 0.020916 13.05MiB/CPU-s 399.73KiB/CPU-s 5.16KiB/CPU-s 0.0 0.029917 False False
splunk_hec_route_s3 -300.64KiB/CPU-s -2.49 100.00% 11.78MiB/CPU-s 587.95KiB/CPU-s 7.58KiB/CPU-s 0.0 0.04875 11.48MiB/CPU-s 818.14KiB/CPU-s 10.56KiB/CPU-s 0.0 0.069571 False False
syslog_log2metric_splunk_hec_metrics -323.28KiB/CPU-s -3.36 100.00% 9.4MiB/CPU-s 348.14KiB/CPU-s 4.49KiB/CPU-s 0.0 0.036183 9.08MiB/CPU-s 172.2KiB/CPU-s 2.22KiB/CPU-s 0.0 0.01852 False False
datadog_agent_remap_datadog_logs_acks -1.2MiB/CPU-s -3.52 100.00% 34.01MiB/CPU-s 1.32MiB/CPU-s 17.43KiB/CPU-s 0.0 0.03879 32.81MiB/CPU-s 1.56MiB/CPU-s 20.59KiB/CPU-s 0.0 0.04749 False False
syslog_humio_logs -372.45KiB/CPU-s -3.99 100.00% 9.12MiB/CPU-s 236.08KiB/CPU-s 3.05KiB/CPU-s 0.0 0.025285 8.75MiB/CPU-s 101.81KiB/CPU-s 1.31KiB/CPU-s 0.0 0.011357 False False
datadog_agent_remap_datadog_logs -1.36MiB/CPU-s -4.03 100.00% 33.78MiB/CPU-s 1.44MiB/CPU-s 18.98KiB/CPU-s 0.0 0.042523 32.42MiB/CPU-s 1.5MiB/CPU-s 19.75KiB/CPU-s 0.0 0.046126 False False
syslog_splunk_hec_logs -386.0KiB/CPU-s -4.26 100.00% 8.84MiB/CPU-s 324.39KiB/CPU-s 4.19KiB/CPU-s 0.0 0.035828 8.46MiB/CPU-s 430.64KiB/CPU-s 5.56KiB/CPU-s 0.0 0.049681 False False
http_text_to_http_json -1.06MiB/CPU-s -4.31 100.00% 24.67MiB/CPU-s 835.56KiB/CPU-s 10.79KiB/CPU-s 0.0 0.033067 23.61MiB/CPU-s 875.56KiB/CPU-s 11.3KiB/CPU-s 0.0 0.036211 False False
syslog_loki -681.59KiB/CPU-s -7.58 100.00% 8.78MiB/CPU-s 370.54KiB/CPU-s 4.78KiB/CPU-s 0.0 0.041199 8.12MiB/CPU-s 403.77KiB/CPU-s 5.21KiB/CPU-s 0.0 0.048575 False False

github-actions[bot] avatar Jan 11 '23 20:01 github-actions[bot]

Regression Test Results

Run ID: 3324b3d0-d012-4dc7-bc8a-12dffb5afbc1
Baseline: 4b2b7130e71d82fbbebce91cfab4f0c5afbfa2fe
Comparison: 0e85bdcb62d925cea2ed87be9a91dc42b0bc454b
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
otlp_http_to_blackhole 15.78KiB/CPU-s 1.02 100.00% 1.51MiB/CPU-s 118.55KiB/CPU-s 1.53KiB/CPU-s 0.0 0.076821 1.52MiB/CPU-s 117.96KiB/CPU-s 1.52KiB/CPU-s 0.0 0.075669 False False
syslog_regex_logs2metric_ddmetrics 36.24KiB/CPU-s 0.99 100.00% 3.57MiB/CPU-s 473.92KiB/CPU-s 6.12KiB/CPU-s 0.0 0.129721 3.6MiB/CPU-s 462.78KiB/CPU-s 5.97KiB/CPU-s 0.0 0.125426 True False
http_to_http_acks 26.31KiB/CPU-s 0.49 39.04% 5.21MiB/CPU-s 2.75MiB/CPU-s 36.31KiB/CPU-s 0.0 0.526943 5.24MiB/CPU-s 2.76MiB/CPU-s 36.54KiB/CPU-s 0.0 0.527571 True False
socket_to_socket_blackhole 4.46KiB/CPU-s 0.03 46.23% 13.01MiB/CPU-s 471.83KiB/CPU-s 6.09KiB/CPU-s 0.0 0.035409 13.02MiB/CPU-s 303.08KiB/CPU-s 3.91KiB/CPU-s 0.0 0.022737 False False
splunk_hec_to_splunk_hec_logs_noack 4.68KiB/CPU-s 0.03 77.71% 13.62MiB/CPU-s 227.86KiB/CPU-s 2.94KiB/CPU-s 0.0 0.016336 13.63MiB/CPU-s 191.29KiB/CPU-s 2.47KiB/CPU-s 0.0 0.013709 False False
splunk_hec_to_splunk_hec_logs_acks 3.68KiB/CPU-s 0.03 43.84% 13.62MiB/CPU-s 354.77KiB/CPU-s 4.58KiB/CPU-s 0.0 0.025443 13.62MiB/CPU-s 339.93KiB/CPU-s 4.39KiB/CPU-s 0.0 0.024372 False False
http_to_http_noack 2.35KiB/CPU-s 0.02 29.70% 13.61MiB/CPU-s 350.11KiB/CPU-s 4.52KiB/CPU-s 0.0 0.02512 13.61MiB/CPU-s 326.12KiB/CPU-s 4.21KiB/CPU-s 0.0 0.023395 False False
file_to_blackhole -97.03B/CPU-s -0.0 0.37% 54.49MiB/CPU-s 1.09MiB/CPU-s 14.44KiB/CPU-s 0.0 0.020053 54.49MiB/CPU-s 1.1MiB/CPU-s 14.48KiB/CPU-s 0.0 0.020132 False False
fluent_elasticsearch -386.53B/CPU-s -0.0 52.13% 45.41MiB/CPU-s 29.66KiB/CPU-s 387.49B/CPU-s 0.0 0.000638 45.41MiB/CPU-s 29.4KiB/CPU-s 384.13B/CPU-s 0.0 0.000632 False False
enterprise_http_to_http -2.04KiB/CPU-s -0.01 48.59% 13.62MiB/CPU-s 150.95KiB/CPU-s 1.95KiB/CPU-s 0.0 0.010819 13.62MiB/CPU-s 189.87KiB/CPU-s 2.45KiB/CPU-s 0.0 0.013611 False False
splunk_hec_indexer_ack_blackhole -1.38KiB/CPU-s -0.01 24.56% 13.62MiB/CPU-s 239.63KiB/CPU-s 3.09KiB/CPU-s 0.0 0.017181 13.62MiB/CPU-s 244.93KiB/CPU-s 3.16KiB/CPU-s 0.0 0.017563 False False
http_to_http_json -68.04KiB/CPU-s -0.49 100.00% 13.62MiB/CPU-s 212.08KiB/CPU-s 2.74KiB/CPU-s 0.0 0.015203 13.56MiB/CPU-s 325.93KiB/CPU-s 4.2KiB/CPU-s 0.0 0.023478 False False
datadog_agent_remap_blackhole -312.2KiB/CPU-s -1.01 100.00% 30.11MiB/CPU-s 1.74MiB/CPU-s 22.98KiB/CPU-s 0.0 0.057726 29.81MiB/CPU-s 1.46MiB/CPU-s 19.29KiB/CPU-s 0.0 0.048969 False False
datadog_agent_remap_datadog_logs -381.01KiB/CPU-s -1.11 100.00% 33.51MiB/CPU-s 1.81MiB/CPU-s 23.9KiB/CPU-s 0.0 0.054005 33.13MiB/CPU-s 1.42MiB/CPU-s 18.76KiB/CPU-s 0.0 0.042861 False False
datadog_agent_remap_blackhole_acks -379.98KiB/CPU-s -1.21 100.00% 30.56MiB/CPU-s 1.56MiB/CPU-s 20.56KiB/CPU-s 0.0 0.050886 30.19MiB/CPU-s 1.45MiB/CPU-s 19.13KiB/CPU-s 0.0 0.04794 False False
splunk_hec_route_s3 -178.28KiB/CPU-s -1.48 100.00% 11.8MiB/CPU-s 548.74KiB/CPU-s 7.08KiB/CPU-s 0.0 0.045428 11.62MiB/CPU-s 657.77KiB/CPU-s 8.49KiB/CPU-s 0.0 0.055271 False False
syslog_log2metric_humio_metrics -103.43KiB/CPU-s -1.69 100.00% 5.98MiB/CPU-s 333.98KiB/CPU-s 4.31KiB/CPU-s 0.0 0.054521 5.88MiB/CPU-s 320.0KiB/CPU-s 4.13KiB/CPU-s 0.0 0.053137 False False
otlp_grpc_to_blackhole -20.86KiB/CPU-s -1.98 100.00% 1.03MiB/CPU-s 44.46KiB/CPU-s 587.71B/CPU-s 0.0 0.042233 1.01MiB/CPU-s 56.79KiB/CPU-s 750.29B/CPU-s 0.0 0.055034 False False
datadog_agent_remap_datadog_logs_acks -701.05KiB/CPU-s -2.07 100.00% 33.15MiB/CPU-s 1.46MiB/CPU-s 19.31KiB/CPU-s 0.0 0.044091 32.47MiB/CPU-s 1.23MiB/CPU-s 16.26KiB/CPU-s 0.0 0.037883 False False
syslog_splunk_hec_logs -231.05KiB/CPU-s -2.51 100.00% 8.99MiB/CPU-s 239.89KiB/CPU-s 3.1KiB/CPU-s 0.0 0.026063 8.76MiB/CPU-s 170.21KiB/CPU-s 2.2KiB/CPU-s 0.0 0.018968 False False
http_text_to_http_json -778.03KiB/CPU-s -3.07 100.00% 24.77MiB/CPU-s 619.71KiB/CPU-s 8.0KiB/CPU-s 0.0 0.024434 24.01MiB/CPU-s 541.51KiB/CPU-s 6.99KiB/CPU-s 0.0 0.022026 False False
syslog_loki -286.99KiB/CPU-s -3.26 100.00% 8.59MiB/CPU-s 462.69KiB/CPU-s 5.97KiB/CPU-s 0.0 0.052599 8.31MiB/CPU-s 245.24KiB/CPU-s 3.17KiB/CPU-s 0.0 0.028819 False False
syslog_log2metric_splunk_hec_metrics -418.34KiB/CPU-s -4.37 100.00% 9.35MiB/CPU-s 380.81KiB/CPU-s 4.92KiB/CPU-s 0.0 0.039759 8.94MiB/CPU-s 366.06KiB/CPU-s 4.73KiB/CPU-s 0.0 0.039965 False False
syslog_humio_logs -446.31KiB/CPU-s -4.91 100.00% 8.87MiB/CPU-s 427.99KiB/CPU-s 5.52KiB/CPU-s 0.0 0.047106 8.44MiB/CPU-s 304.43KiB/CPU-s 3.93KiB/CPU-s 0.0 0.035238 False False

github-actions[bot] avatar Jan 11 '23 20:01 github-actions[bot]

Regression Test Results

Run ID: 27880ec9-9143-4c69-99e1-b016b1976b4a
Baseline: 4af6d78b88661d874547f336a740b845aacd9334
Comparison: b2ff3051e07469e3531d00bc7feaa4d3bf5db743
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
datadog_agent_remap_blackhole_acks 889.14KiB/CPU-s 2.85 100.00% 30.46MiB/CPU-s 1.45MiB/CPU-s 19.23KiB/CPU-s 0.0 0.047744 31.32MiB/CPU-s 1.23MiB/CPU-s 16.27KiB/CPU-s 0.0 0.039286 False False
datadog_agent_remap_blackhole 869.44KiB/CPU-s 2.76 100.00% 30.71MiB/CPU-s 1.28MiB/CPU-s 16.92KiB/CPU-s 0.0 0.041667 31.56MiB/CPU-s 684.4KiB/CPU-s 8.83KiB/CPU-s 0.0 0.021177 False False
syslog_log2metric_humio_metrics 122.89KiB/CPU-s 1.96 100.00% 6.13MiB/CPU-s 214.82KiB/CPU-s 2.77KiB/CPU-s 0.0 0.034214 6.25MiB/CPU-s 111.93KiB/CPU-s 1.44KiB/CPU-s 0.0 0.017484 False False
http_to_http_acks 84.27KiB/CPU-s 1.57 89.46% 5.24MiB/CPU-s 2.75MiB/CPU-s 36.37KiB/CPU-s 0.0 0.525358 5.32MiB/CPU-s 2.82MiB/CPU-s 37.23KiB/CPU-s 0.0 0.529245 True False
syslog_humio_logs 135.64KiB/CPU-s 1.48 100.00% 8.96MiB/CPU-s 504.08KiB/CPU-s 6.51KiB/CPU-s 0.0 0.05496 9.09MiB/CPU-s 315.75KiB/CPU-s 4.08KiB/CPU-s 0.0 0.033925 False False
datadog_agent_remap_datadog_logs_acks 494.49KiB/CPU-s 1.42 100.00% 34.07MiB/CPU-s 1.48MiB/CPU-s 19.56KiB/CPU-s 0.0 0.043447 34.55MiB/CPU-s 611.76KiB/CPU-s 7.9KiB/CPU-s 0.0 0.017288 False False
syslog_loki 42.48KiB/CPU-s 0.46 100.00% 8.94MiB/CPU-s 120.14KiB/CPU-s 1.55KiB/CPU-s 0.0 0.01312 8.98MiB/CPU-s 77.51KiB/CPU-s 1.0KiB/CPU-s 0.0 0.008426 False False
http_to_http_json 25.64KiB/CPU-s 0.18 100.00% 13.6MiB/CPU-s 249.05KiB/CPU-s 3.21KiB/CPU-s 0.0 0.017886 13.62MiB/CPU-s 211.13KiB/CPU-s 2.72KiB/CPU-s 0.0 0.015134 False False
enterprise_http_to_http 10.55KiB/CPU-s 0.08 94.80% 13.61MiB/CPU-s 355.6KiB/CPU-s 4.59KiB/CPU-s 0.0 0.025514 13.62MiB/CPU-s 224.7KiB/CPU-s 2.9KiB/CPU-s 0.0 0.01611 False False
http_to_http_noack 6.13KiB/CPU-s 0.04 70.35% 13.61MiB/CPU-s 354.37KiB/CPU-s 4.57KiB/CPU-s 0.0 0.025426 13.62MiB/CPU-s 284.95KiB/CPU-s 3.68KiB/CPU-s 0.0 0.020436 False False
splunk_hec_to_splunk_hec_logs_noack 5.84KiB/CPU-s 0.04 86.56% 13.62MiB/CPU-s 234.41KiB/CPU-s 3.02KiB/CPU-s 0.0 0.016806 13.63MiB/CPU-s 190.77KiB/CPU-s 2.46KiB/CPU-s 0.0 0.013671 False False
splunk_hec_to_splunk_hec_logs_acks 3.22KiB/CPU-s 0.02 39.16% 13.61MiB/CPU-s 349.96KiB/CPU-s 4.51KiB/CPU-s 0.0 0.025102 13.62MiB/CPU-s 338.26KiB/CPU-s 4.36KiB/CPU-s 0.0 0.024257 False False
fluent_elasticsearch -395.39B/CPU-s -0.0 52.21% 45.41MiB/CPU-s 30.23KiB/CPU-s 395.11B/CPU-s 0.0 0.00065 45.41MiB/CPU-s 30.05KiB/CPU-s 392.85B/CPU-s 0.0 0.000646 False False
splunk_hec_indexer_ack_blackhole -254.07B/CPU-s -0.0 4.35% 13.62MiB/CPU-s 248.81KiB/CPU-s 3.21KiB/CPU-s 0.0 0.017841 13.62MiB/CPU-s 249.41KiB/CPU-s 3.22KiB/CPU-s 0.0 0.017884 False False
file_to_blackhole -7.36KiB/CPU-s -0.01 27.77% 54.5MiB/CPU-s 1.1MiB/CPU-s 14.54KiB/CPU-s 0.0 0.020202 54.49MiB/CPU-s 1.12MiB/CPU-s 14.74KiB/CPU-s 0.0 0.020488 False False
datadog_agent_remap_datadog_logs -30.46KiB/CPU-s -0.09 80.45% 34.59MiB/CPU-s 1.17MiB/CPU-s 15.48KiB/CPU-s 0.0 0.033889 34.56MiB/CPU-s 1.34MiB/CPU-s 17.71KiB/CPU-s 0.0 0.038797 False False
socket_to_socket_blackhole -14.75KiB/CPU-s -0.11 99.82% 13.52MiB/CPU-s 268.47KiB/CPU-s 3.47KiB/CPU-s 0.0 0.019387 13.51MiB/CPU-s 248.19KiB/CPU-s 3.21KiB/CPU-s 0.0 0.017941 False False
syslog_regex_logs2metric_ddmetrics -7.89KiB/CPU-s -0.2 64.95% 3.8MiB/CPU-s 463.95KiB/CPU-s 5.99KiB/CPU-s 0.0 0.119275 3.79MiB/CPU-s 461.73KiB/CPU-s 5.96KiB/CPU-s 0.0 0.118945 True False
syslog_log2metric_splunk_hec_metrics -22.49KiB/CPU-s -0.23 100.00% 9.55MiB/CPU-s 125.95KiB/CPU-s 1.63KiB/CPU-s 0.0 0.012876 9.53MiB/CPU-s 122.16KiB/CPU-s 1.58KiB/CPU-s 0.0 0.012518 False False
syslog_splunk_hec_logs -107.13KiB/CPU-s -1.15 100.00% 9.11MiB/CPU-s 206.59KiB/CPU-s 2.67KiB/CPU-s 0.0 0.022148 9.0MiB/CPU-s 322.72KiB/CPU-s 4.16KiB/CPU-s 0.0 0.035 False False
otlp_http_to_blackhole -18.24KiB/CPU-s -1.16 100.00% 1.53MiB/CPU-s 109.06KiB/CPU-s 1.41KiB/CPU-s 0.0 0.069647 1.51MiB/CPU-s 125.74KiB/CPU-s 1.62KiB/CPU-s 0.0 0.081244 False False
splunk_hec_route_s3 -149.71KiB/CPU-s -1.24 100.00% 11.76MiB/CPU-s 654.49KiB/CPU-s 8.44KiB/CPU-s 0.0 0.054324 11.62MiB/CPU-s 760.36KiB/CPU-s 9.81KiB/CPU-s 0.0 0.063905 False False
otlp_grpc_to_blackhole -13.36KiB/CPU-s -1.27 100.00% 1.03MiB/CPU-s 42.51KiB/CPU-s 561.93B/CPU-s 0.0 0.04041 1.01MiB/CPU-s 62.48KiB/CPU-s 825.41B/CPU-s 0.0 0.060161 False False
http_text_to_http_json -495.94KiB/CPU-s -1.9 100.00% 25.52MiB/CPU-s 563.45KiB/CPU-s 7.27KiB/CPU-s 0.0 0.021558 25.04MiB/CPU-s 717.0KiB/CPU-s 9.25KiB/CPU-s 0.0 0.027964 False False

github-actions[bot] avatar Jan 12 '23 16:01 github-actions[bot]

Just wanted to communicate that this is on my radar and I'll set aside some time today to review your updates.

neuronull avatar Jan 12 '23 16:01 neuronull

Thanks for the initial review @neuronull. There is one other thing I wanted to ask about for the cue docs. I have initially just documented the top level fields and linked to envoy docs to further dive into those structs. Is that acceptable, or do we need to document every field? I'd initially gone this route b/c there are a lot of subfields and the structure just follows the envoy proto fields.

On the subject of the cue docs (thanks for asking about it btw, I was going to mention it as well) - I think the best thing to do is keep consistency with our other components, and generally speaking we're looking at components like opentelemetry and kubernetes_logs which have lots of custom fields. And in those, we do document each one.

This issue for otel discusses some of the nuances of protobuf field translation into vector data model: https://github.com/vectordotdev/vector/issues/15500

neuronull avatar Jan 12 '23 23:01 neuronull

Regression Test Results

Run ID: da51a7ac-8f1a-442e-99a5-8d1281efab06
Baseline: 34af7e601cc043635efd1081b5a2823c8f225092
Comparison: 98766e690b40600b60238934734f369a289f0c62
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
http_text_to_http_json 1.52MiB/CPU-s 6.37 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json 1.52MiB/CPU-s 6.37 100.00% 23.91MiB/CPU-s 627.87KiB/CPU-s 8.11KiB/CPU-s 0.0 0.025638 25.44MiB/CPU-s 630.26KiB/CPU-s 8.13KiB/CPU-s 0.0 0.024193 False False
socket_to_socket_blackhole 335.3KiB/CPU-s 2.5 100.00% 13.12MiB/CPU-s 296.6KiB/CPU-s 3.83KiB/CPU-s 0.0 0.022073 13.45MiB/CPU-s 311.28KiB/CPU-s 4.02KiB/CPU-s 0.0 0.022601 False False
datadog_agent_remap_blackhole_acks 639.54KiB/CPU-s 2.03 100.00% 30.72MiB/CPU-s 1.61MiB/CPU-s 21.29KiB/CPU-s 0.0 0.052426 31.34MiB/CPU-s 1.14MiB/CPU-s 15.11KiB/CPU-s 0.0 0.036456 False False
splunk_hec_route_s3 142.87KiB/CPU-s 1.2 100.00% 11.6MiB/CPU-s 561.22KiB/CPU-s 7.24KiB/CPU-s 0.0 0.047235 11.74MiB/CPU-s 584.41KiB/CPU-s 7.54KiB/CPU-s 0.0 0.048602 False False
datadog_agent_remap_datadog_logs 397.31KiB/CPU-s 1.15 100.00% 33.64MiB/CPU-s 1.32MiB/CPU-s 17.4KiB/CPU-s 0.0 0.039155 34.03MiB/CPU-s 1.28MiB/CPU-s 16.94KiB/CPU-s 0.0 0.037685 False False
syslog_log2metric_splunk_hec_metrics 56.35KiB/CPU-s 0.59 100.00% 9.31MiB/CPU-s 252.2KiB/CPU-s 3.26KiB/CPU-s 0.0 0.02645 9.37MiB/CPU-s 209.65KiB/CPU-s 2.71KiB/CPU-s 0.0 0.021858 False False
datadog_agent_remap_datadog_logs_acks 109.48KiB/CPU-s 0.32 100.00% 33.47MiB/CPU-s 1.19MiB/CPU-s 15.68KiB/CPU-s 0.0 0.035467 33.58MiB/CPU-s 840.9KiB/CPU-s 10.85KiB/CPU-s 0.0 0.024455 False False
syslog_regex_logs2metric_ddmetrics 9.75KiB/CPU-s 0.26 81.43% 3.61MiB/CPU-s 388.86KiB/CPU-s 5.02KiB/CPU-s 0.0 0.10525 3.62MiB/CPU-s 417.57KiB/CPU-s 5.39KiB/CPU-s 0.0 0.112724 True False
otlp_http_to_blackhole 3.35KiB/CPU-s 0.21 89.20% 1.54MiB/CPU-s 109.17KiB/CPU-s 1.41KiB/CPU-s 0.0 0.069391 1.54MiB/CPU-s 119.02KiB/CPU-s 1.54KiB/CPU-s 0.0 0.075493 False False
enterprise_http_to_http 6.17KiB/CPU-s 0.04 89.08% 13.62MiB/CPU-s 257.94KiB/CPU-s 3.33KiB/CPU-s 0.0 0.018495 13.62MiB/CPU-s 149.66KiB/CPU-s 1.93KiB/CPU-s 0.0 0.010727 False False
splunk_hec_to_splunk_hec_logs_noack 5.63KiB/CPU-s 0.04 85.28% 13.62MiB/CPU-s 234.22KiB/CPU-s 3.02KiB/CPU-s 0.0 0.016793 13.63MiB/CPU-s 189.04KiB/CPU-s 2.44KiB/CPU-s 0.0 0.013548 False False
splunk_hec_to_splunk_hec_logs_acks 3.49KiB/CPU-s 0.03 41.25% 13.61MiB/CPU-s 360.93KiB/CPU-s 4.66KiB/CPU-s 0.0 0.025889 13.62MiB/CPU-s 345.04KiB/CPU-s 4.45KiB/CPU-s 0.0 0.024744 False False
fluent_elasticsearch 9.7KiB/CPU-s 0.02 100.00% 45.41MiB/CPU-s 30.02KiB/CPU-s 392.4B/CPU-s 0.0 0.000646 45.42MiB/CPU-s 42.16KiB/CPU-s 550.97B/CPU-s 0.0 0.000906 False False
splunk_hec_indexer_ack_blackhole 55.86B/CPU-s 0.0 0.94% 13.62MiB/CPU-s 253.79KiB/CPU-s 3.27KiB/CPU-s 0.0 0.0182 13.62MiB/CPU-s 252.48KiB/CPU-s 3.26KiB/CPU-s 0.0 0.018106 False False
file_to_blackhole -3.83KiB/CPU-s -0.01 14.62% 54.5MiB/CPU-s 1.11MiB/CPU-s 14.62KiB/CPU-s 0.0 0.020306 54.49MiB/CPU-s 1.12MiB/CPU-s 14.78KiB/CPU-s 0.0 0.020554 False False
http_to_http_noack -10.24KiB/CPU-s -0.07 89.98% 13.62MiB/CPU-s 285.57KiB/CPU-s 3.69KiB/CPU-s 0.0 0.02048 13.61MiB/CPU-s 389.05KiB/CPU-s 5.02KiB/CPU-s 0.0 0.027922 False False
http_to_http_json -18.8KiB/CPU-s -0.13 100.00% 13.62MiB/CPU-s 211.29KiB/CPU-s 2.73KiB/CPU-s 0.0 0.015147 13.6MiB/CPU-s 249.71KiB/CPU-s 3.22KiB/CPU-s 0.0 0.017925 False False
http_to_http_acks -14.31KiB/CPU-s -0.26 21.83% 5.32MiB/CPU-s 2.78MiB/CPU-s 36.73KiB/CPU-s 0.0 0.521929 5.31MiB/CPU-s 2.75MiB/CPU-s 36.3KiB/CPU-s 0.0 0.516969 True False
syslog_humio_logs -59.43KiB/CPU-s -0.63 100.00% 9.15MiB/CPU-s 227.57KiB/CPU-s 2.94KiB/CPU-s 0.0 0.024293 9.09MiB/CPU-s 120.78KiB/CPU-s 1.56KiB/CPU-s 0.0 0.012976 False False
syslog_splunk_hec_logs -68.97KiB/CPU-s -0.75 100.00% 9.01MiB/CPU-s 157.61KiB/CPU-s 2.03KiB/CPU-s 0.0 0.017088 8.94MiB/CPU-s 213.74KiB/CPU-s 2.76KiB/CPU-s 0.0 0.023348 False False
otlp_grpc_to_blackhole -8.49KiB/CPU-s -0.8 100.00% 1.04MiB/CPU-s 40.62KiB/CPU-s 536.83B/CPU-s 0.0 0.038116 1.03MiB/CPU-s 56.91KiB/CPU-s 751.77B/CPU-s 0.0 0.053838 False False
syslog_log2metric_humio_metrics -49.68KiB/CPU-s -0.81 100.00% 6.02MiB/CPU-s 188.88KiB/CPU-s 2.44KiB/CPU-s 0.0 0.030657 5.97MiB/CPU-s 204.64KiB/CPU-s 2.64KiB/CPU-s 0.0 0.033486 False False
syslog_loki -110.85KiB/CPU-s -1.21 100.00% 8.91MiB/CPU-s 158.22KiB/CPU-s 2.04KiB/CPU-s 0.0 0.017337 8.8MiB/CPU-s 111.42KiB/CPU-s 1.44KiB/CPU-s 0.0 0.012359 False False
datadog_agent_remap_blackhole -382.42KiB/CPU-s -1.21 100.00% 30.78MiB/CPU-s 1.03MiB/CPU-s 13.67KiB/CPU-s 0.0 0.033617 30.41MiB/CPU-s 1.47MiB/CPU-s 19.45KiB/CPU-s 0.0 0.048387 False False

github-actions[bot] avatar Jan 17 '23 05:01 github-actions[bot]

Regression Test Results

Run ID: b2992090-1213-40b2-b20a-5b345d6f585f
Baseline: 00b0d9c68b6991e6e953ad80dad243dd7a10ac2f
Comparison: c52df206cffc0c166ccf3bdbd5f6ace382e73a65
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
syslog_log2metric_humio_metrics -367.19KiB/CPU-s -5.81 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
socket_to_socket_blackhole 377.63KiB/CPU-s 2.88 100.00% 12.81MiB/CPU-s 482.79KiB/CPU-s 6.23KiB/CPU-s 0.0 0.036813 13.18MiB/CPU-s 417.42KiB/CPU-s 5.39KiB/CPU-s 0.0 0.030938 False False
otlp_http_to_blackhole 33.7KiB/CPU-s 2.2 100.00% 1.5MiB/CPU-s 119.81KiB/CPU-s 1.55KiB/CPU-s 0.0 0.078166 1.53MiB/CPU-s 122.71KiB/CPU-s 1.58KiB/CPU-s 0.0 0.07834 False False
splunk_hec_route_s3 206.97KiB/CPU-s 1.78 100.00% 11.36MiB/CPU-s 789.53KiB/CPU-s 10.18KiB/CPU-s 0.0 0.067839 11.57MiB/CPU-s 554.29KiB/CPU-s 7.15KiB/CPU-s 0.0 0.046794 False False
http_text_to_http_json 425.65KiB/CPU-s 1.76 100.00% 23.61MiB/CPU-s 540.99KiB/CPU-s 6.98KiB/CPU-s 0.0 0.02237 24.03MiB/CPU-s 622.13KiB/CPU-s 8.03KiB/CPU-s 0.0 0.02528 False False
http_to_http_acks 71.26KiB/CPU-s 1.34 83.16% 5.2MiB/CPU-s 2.77MiB/CPU-s 36.59KiB/CPU-s 0.0 0.53277 5.27MiB/CPU-s 2.77MiB/CPU-s 36.57KiB/CPU-s 0.0 0.525369 True False
syslog_loki 100.66KiB/CPU-s 1.16 100.00% 8.51MiB/CPU-s 353.5KiB/CPU-s 4.56KiB/CPU-s 0.0 0.040585 8.6MiB/CPU-s 308.86KiB/CPU-s 3.99KiB/CPU-s 0.0 0.035055 False False
syslog_humio_logs 72.07KiB/CPU-s 0.77 100.00% 9.14MiB/CPU-s 196.66KiB/CPU-s 2.54KiB/CPU-s 0.0 0.021002 9.21MiB/CPU-s 127.91KiB/CPU-s 1.65KiB/CPU-s 0.0 0.013556 False False
datadog_agent_remap_datadog_logs 259.68KiB/CPU-s 0.75 100.00% 33.79MiB/CPU-s 1.1MiB/CPU-s 14.47KiB/CPU-s 0.0 0.032413 34.04MiB/CPU-s 1.11MiB/CPU-s 14.71KiB/CPU-s 0.0 0.032706 False False
syslog_splunk_hec_logs 57.58KiB/CPU-s 0.62 100.00% 9.0MiB/CPU-s 141.9KiB/CPU-s 1.83KiB/CPU-s 0.0 0.015395 9.06MiB/CPU-s 212.02KiB/CPU-s 2.74KiB/CPU-s 0.0 0.02286 False False
otlp_grpc_to_blackhole 5.47KiB/CPU-s 0.52 100.00% 1.04MiB/CPU-s 34.69KiB/CPU-s 458.55B/CPU-s 0.0 0.032651 1.04MiB/CPU-s 54.1KiB/CPU-s 714.71B/CPU-s 0.0 0.050663 False False
http_to_http_json 26.91KiB/CPU-s 0.19 100.00% 13.59MiB/CPU-s 259.05KiB/CPU-s 3.34KiB/CPU-s 0.0 0.018612 13.62MiB/CPU-s 240.43KiB/CPU-s 3.1KiB/CPU-s 0.0 0.01724 False False
enterprise_http_to_http 12.62KiB/CPU-s 0.09 99.25% 13.61MiB/CPU-s 333.54KiB/CPU-s 4.3KiB/CPU-s 0.0 0.023928 13.62MiB/CPU-s 150.27KiB/CPU-s 1.94KiB/CPU-s 0.0 0.01077 False False
http_to_http_noack 6.43KiB/CPU-s 0.05 73.24% 13.61MiB/CPU-s 352.41KiB/CPU-s 4.55KiB/CPU-s 0.0 0.025285 13.62MiB/CPU-s 278.93KiB/CPU-s 3.6KiB/CPU-s 0.0 0.020004 False False
splunk_hec_to_splunk_hec_logs_noack 5.77KiB/CPU-s 0.04 86.05% 13.62MiB/CPU-s 233.58KiB/CPU-s 3.01KiB/CPU-s 0.0 0.016746 13.63MiB/CPU-s 191.91KiB/CPU-s 2.48KiB/CPU-s 0.0 0.013754 False False
splunk_hec_indexer_ack_blackhole 278.24B/CPU-s 0.0 4.68% 13.62MiB/CPU-s 256.73KiB/CPU-s 3.31KiB/CPU-s 0.0 0.018412 13.62MiB/CPU-s 251.35KiB/CPU-s 3.24KiB/CPU-s 0.0 0.018026 False False
splunk_hec_to_splunk_hec_logs_acks -403.89B/CPU-s -0.0 4.78% 13.61MiB/CPU-s 366.55KiB/CPU-s 4.73KiB/CPU-s 0.0 0.02629 13.61MiB/CPU-s 354.35KiB/CPU-s 4.57KiB/CPU-s 0.0 0.025415 False False
fluent_elasticsearch -3.86KiB/CPU-s -0.01 68.86% 45.41MiB/CPU-s 30.07KiB/CPU-s 392.78B/CPU-s 0.0 0.000646 45.41MiB/CPU-s 297.29KiB/CPU-s 3.79KiB/CPU-s 0.0 0.006393 False False
file_to_blackhole -5.45KiB/CPU-s -0.01 19.56% 54.5MiB/CPU-s 1.18MiB/CPU-s 15.54KiB/CPU-s 0.0 0.021595 54.49MiB/CPU-s 1.18MiB/CPU-s 15.58KiB/CPU-s 0.0 0.021664 False False
datadog_agent_remap_blackhole_acks -225.59KiB/CPU-s -0.7 100.00% 31.49MiB/CPU-s 827.9KiB/CPU-s 10.69KiB/CPU-s 0.0 0.02567 31.27MiB/CPU-s 1.03MiB/CPU-s 13.55KiB/CPU-s 0.0 0.03278 False False
syslog_regex_logs2metric_ddmetrics -42.19KiB/CPU-s -1.13 100.00% 3.64MiB/CPU-s 365.39KiB/CPU-s 4.72KiB/CPU-s 0.0 0.09789 3.6MiB/CPU-s 346.7KiB/CPU-s 4.48KiB/CPU-s 0.0 0.093945 False False
datadog_agent_remap_datadog_logs_acks -415.61KiB/CPU-s -1.2 100.00% 33.71MiB/CPU-s 938.76KiB/CPU-s 12.11KiB/CPU-s 0.0 0.027189 33.31MiB/CPU-s 692.46KiB/CPU-s 8.94KiB/CPU-s 0.0 0.0203 False False
syslog_log2metric_splunk_hec_metrics -248.38KiB/CPU-s -2.59 100.00% 9.38MiB/CPU-s 247.53KiB/CPU-s 3.2KiB/CPU-s 0.0 0.025774 9.14MiB/CPU-s 474.55KiB/CPU-s 6.13KiB/CPU-s 0.0 0.050725 False False
datadog_agent_remap_blackhole -1.02MiB/CPU-s -3.24 100.00% 31.46MiB/CPU-s 589.87KiB/CPU-s 7.62KiB/CPU-s 0.0 0.018308 30.44MiB/CPU-s 1.38MiB/CPU-s 18.3KiB/CPU-s 0.0 0.045463 False False
syslog_log2metric_humio_metrics -367.19KiB/CPU-s -5.81 100.00% 6.18MiB/CPU-s 250.93KiB/CPU-s 3.24KiB/CPU-s 0.0 0.039669 5.82MiB/CPU-s 435.66KiB/CPU-s 5.62KiB/CPU-s 0.0 0.073116 False False

github-actions[bot] avatar Jan 17 '23 23:01 github-actions[bot]

I posted this in the discord (https://discord.com/channels/742820443487993987/746070604192415834/1065653117313630308), but posting here as well for visibility / traction. I've been trying to figure out how to properly document all the nested fields on this log struct. I can't seems to get nested fields to show up when I try to gen/view the website. I noticed it wasn't just a problem with what I was trying, dnstap is the same. If you look at requestData here https://vector.dev/docs/reference/configuration/sources/dnstap/#event-fields, all there is is the top level field. But in the code there are a lot of subfields that don't show up on the website https://github.com/vectordotdev/vector/blob/master/website/cue/reference/components/sources/dnstap.cue#L409. So I guess I could use some help in figuring out how to properly document nested fields.

Also, thanks again for all your guidance / help @neuronull

wez470 avatar Jan 19 '23 19:01 wez470

I've been trying to figure out how to properly document all the nested fields on this log struct. I can't seems to get nested fields to show up when I try to gen/view the website. I noticed it wasn't just a problem with what I was trying, dnstap is the same.

Interesting. It looks like this is a bug with the website generation and there is probably nothing that you can do to fix this. If you can document the nested fields in a manner similar to dnstap then we can get the website dev team to take a look and fix that issue.

StephenWakely avatar Jan 20 '23 13:01 StephenWakely

I've been trying to figure out how to properly document all the nested fields on this log struct. I can't seems to get nested fields to show up when I try to gen/view the website. I noticed it wasn't just a problem with what I was trying, dnstap is the same.

Interesting. It looks like this is a bug with the website generation and there is probably nothing that you can do to fix this. If you can document the nested fields in a manner similar to dnstap then we can get the website dev team to take a look and fix that issue.

Actually, there is a better way. Have a look at the docs for the k8s log source. This is the correct way. Each field in the output should be it's own field, subfields are just distinguished by separating the parent and child field with . - ie.. parent.child.

The dnstap source docs need updating to work this way too.

StephenWakely avatar Jan 23 '23 12:01 StephenWakely

I'm really sorry about this, but I am closing this PR. The company I worked at where I wanted this feature did layoffs and I don't have a reason to continue at this point. I appreciate all the effort that was put in to review this work. Also, if anyone wants to pick up this work going forward, feel free to copy what I've done and do the remaining pieces (mainly implement the log name spacing).

wez470 avatar Feb 19 '23 00:02 wez470

I'm really sorry about this, but I am closing this PR. The company I worked at where I wanted this feature did layoffs and I don't have a reason to continue at this point. I appreciate all the effort that was put in to review this work. Also, if anyone wants to pick up this work going forward, feel free to copy what I've done and do the remaining pieces (mainly implement the log name spacing).

No worries @wez470. Thanks for taking the time to let us know. We hope to see you around in future PRs ~

neuronull avatar Feb 21 '23 15:02 neuronull