vector
vector copied to clipboard
feat(vrl): Adds `keys()` and `values()` function
Issue: https://github.com/vectordotdev/vector/issues/11888
This adds a values
method and a keys
method to print out only the keys or values of a given iterable vrl Value
input.
Examples is yet to be implemented, future iteration is needed and feedback is welcome.
Testing: To be determined
Deploy Preview for vector-project ready!
Name | Link |
---|---|
Latest commit | f8a673b2772e5d128ef4a38e218d3fe7e3601ea4 |
Latest deploy log | https://app.netlify.com/sites/vector-project/deploys/632dcfb442738b0008ed5580 |
Deploy Preview | https://deploy-preview-14441--vector-project.netlify.app |
Preview on mobile | Toggle QR Code...Use your smartphone camera to open QR code link. |
To edit notification comments on pull requests, go to your Netlify site settings.
Soak Test Results
Baseline: a6a1f95c93ff60cdb4672725ab385fe3e14f2498 Comparison: d5ec9177341f09fad546b92e0d588092424df1ba Total Vector CPUs: 4
Explanation
A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.
No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:
Fine details of change detection per experiment.
experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
http_text_to_http_json | 1.17MiB | 3.12 | 100.00% | 37.37MiB | 831.4KiB | 16.97KiB | 0 | 0.0217218 | 38.54MiB | 843.17KiB | 17.22KiB | 0 | 0.0213621 | False | False |
socket_to_socket_blackhole | 513.19KiB | 2.21 | 100.00% | 22.72MiB | 940.69KiB | 19.2KiB | 0 | 0.0404262 | 23.22MiB | 791.74KiB | 16.17KiB | 0 | 0.0332906 | False | False |
http_pipelines_blackhole | 26.34KiB | 1.57 | 100.00% | 1.63MiB | 104.58KiB | 2.14KiB | 0 | 0.0625015 | 1.66MiB | 151.39KiB | 3.08KiB | 0 | 0.0890738 | False | False |
http_pipelines_no_grok_blackhole | 56.62KiB | 0.53 | 97.87% | 10.5MiB | 461.94KiB | 9.43KiB | 0 | 0.0429411 | 10.56MiB | 1.09MiB | 22.69KiB | 0 | 0.103157 | False | False |
splunk_hec_to_splunk_hec_logs_noack | 21.34KiB | 0.09 | 91.41% | 23.82MiB | 508.77KiB | 10.39KiB | 0 | 0.0208574 | 23.84MiB | 333.64KiB | 6.81KiB | 0 | 0.013666 | False | False |
splunk_hec_indexer_ack_blackhole | 4.18KiB | 0.02 | 14.17% | 23.76MiB | 821.06KiB | 16.7KiB | 0 | 0.0337421 | 23.76MiB | 805.95KiB | 16.4KiB | 0 | 0.0331156 | False | False |
splunk_hec_to_splunk_hec_logs_acks | 1.3KiB | 0.01 | 4.45% | 23.76MiB | 816.82KiB | 16.62KiB | 0 | 0.033563 | 23.76MiB | 803.31KiB | 16.35KiB | 0 | 0.0330059 | False | False |
enterprise_http_to_http | -966.87B | -0 | 10.52% | 23.85MiB | 245.19KiB | 5.0KiB | 0 | 0.0100392 | 23.85MiB | 248.9KiB | 5.09KiB | 0 | 0.0101914 | False | False |
file_to_blackhole | -66.89KiB | -0.07 | 43.92% | 95.3MiB | 3.68MiB | 76.18KiB | 0 | 0.0385962 | 95.24MiB | 4.15MiB | 86.17KiB | 0 | 0.0435291 | False | False |
http_to_http_json | -37.79KiB | -0.15 | 99.69% | 23.85MiB | 330.74KiB | 6.75KiB | 0 | 0.0135416 | 23.81MiB | 530.59KiB | 10.84KiB | 0 | 0.0217577 | False | False |
datadog_agent_remap_blackhole | -100.86KiB | -0.16 | 68.70% | 60.62MiB | 3.77MiB | 78.58KiB | 0 | 0.0621496 | 60.52MiB | 2.96MiB | 61.78KiB | 0 | 0.0489205 | False | False |
fluent_elasticsearch | -267.23KiB | -0.33 | 100.00% | 79.47MiB | 54.69KiB | 1.1KiB | 0 | 0.000671842 | 79.21MiB | 2.4MiB | 49.25KiB | 0 | 0.0302394 | False | False |
http_to_http_noack | -105.73KiB | -0.43 | 99.99% | 23.83MiB | 519.1KiB | 10.6KiB | 0 | 0.0212691 | 23.73MiB | 1.19MiB | 24.8KiB | 0 | 0.0501415 | False | False |
http_to_http_acks | -76.59KiB | -0.43 | 24.90% | 17.43MiB | 8.03MiB | 167.92KiB | 0 | 0.460824 | 17.35MiB | 8.3MiB | 173.33KiB | 0 | 0.477914 | True | True |
syslog_regex_logs2metric_ddmetrics | -65.31KiB | -0.51 | 99.99% | 12.49MiB | 586.65KiB | 11.95KiB | 0 | 0.0458503 | 12.43MiB | 558.36KiB | 11.38KiB | 0 | 0.0438637 | False | False |
syslog_humio_logs | -141.5KiB | -0.84 | 100.00% | 16.42MiB | 223.86KiB | 4.57KiB | 0 | 0.0133084 | 16.28MiB | 126.08KiB | 2.58KiB | 0 | 0.00755936 | False | False |
datadog_agent_remap_datadog_logs_acks | -607.07KiB | -0.96 | 100.00% | 62.04MiB | 2.99MiB | 62.61KiB | 0 | 0.048251 | 61.45MiB | 4.37MiB | 91.06KiB | 0 | 0.071178 | False | False |
syslog_splunk_hec_logs | -167.06KiB | -1 | 100.00% | 16.37MiB | 766.58KiB | 15.6KiB | 0 | 0.0457213 | 16.21MiB | 573.34KiB | 11.7KiB | 0 | 0.0345397 | False | False |
datadog_agent_remap_datadog_logs | -728.54KiB | -1.17 | 100.00% | 60.95MiB | 700.5KiB | 14.33KiB | 0 | 0.0112205 | 60.24MiB | 4.05MiB | 84.37KiB | 0 | 0.0672358 | False | False |
syslog_log2metric_splunk_hec_metrics | -217.11KiB | -1.2 | 100.00% | 17.66MiB | 627.55KiB | 12.78KiB | 0 | 0.0346887 | 17.45MiB | 767.47KiB | 15.63KiB | 0 | 0.0429387 | False | False |
http_pipelines_blackhole_acks | -19.19KiB | -1.52 | 100.00% | 1.24MiB | 115.68KiB | 2.35KiB | 0 | 0.0913566 | 1.22MiB | 60.81KiB | 1.24KiB | 0 | 0.0487624 | False | False |
splunk_hec_route_s3 | -309.5KiB | -1.64 | 100.00% | 18.45MiB | 2.31MiB | 48.01KiB | 0 | 0.124937 | 18.15MiB | 2.27MiB | 47.47KiB | 0 | 0.125039 | False | False |
syslog_loki | -373.41KiB | -2.46 | 100.00% | 14.8MiB | 371.32KiB | 7.6KiB | 0 | 0.0244912 | 14.44MiB | 746.38KiB | 15.17KiB | 0 | 0.0504722 | False | False |
datadog_agent_remap_blackhole_acks | -1.84MiB | -2.95 | 100.00% | 62.39MiB | 4.24MiB | 88.24KiB | 0 | 0.067907 | 60.55MiB | 3.54MiB | 73.95KiB | 0 | 0.0584647 | False | False |
syslog_log2metric_humio_metrics | -468.62KiB | -3.54 | 100.00% | 12.94MiB | 225.08KiB | 4.59KiB | 0 | 0.0169779 | 12.49MiB | 641.77KiB | 13.06KiB | 0 | 0.0501836 | False | False |
Soak Test Results
Baseline: d498040a770ae2bb5c9d25efce62acadcb17ee57 Comparison: a7bf74f42fc6ae12b69ee18db6dc895d9da91238 Total Vector CPUs: 4
Explanation
A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.
No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:
Fine details of change detection per experiment.
experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
http_text_to_http_json | 422.99KiB | 1.1 | 100.00% | 37.49MiB | 1.23MiB | 25.74KiB | 0 | 0.0328363 | 37.9MiB | 1.09MiB | 22.76KiB | 0 | 0.0287214 | False | False |
syslog_loki | 114.73KiB | 0.78 | 100.00% | 14.35MiB | 392.51KiB | 8.03KiB | 0 | 0.0267113 | 14.46MiB | 721.87KiB | 14.67KiB | 0 | 0.0487438 | False | False |
datadog_agent_remap_datadog_logs_acks | 190.62KiB | 0.31 | 86.59% | 60.95MiB | 3.77MiB | 78.87KiB | 0 | 0.0618657 | 61.14MiB | 4.8MiB | 99.84KiB | 0 | 0.0784355 | False | False |
datadog_agent_remap_blackhole | 118.41KiB | 0.19 | 73.44% | 59.8MiB | 4.17MiB | 86.75KiB | 0 | 0.0696482 | 59.92MiB | 2.95MiB | 61.52KiB | 0 | 0.0492052 | False | False |
datadog_agent_remap_datadog_logs | 47.6KiB | 0.08 | 38.17% | 61.7MiB | 2.28MiB | 47.73KiB | 0 | 0.0368999 | 61.75MiB | 3.97MiB | 82.74KiB | 0 | 0.0642536 | False | False |
enterprise_http_to_http | 104.47B | 0 | 1.12% | 23.85MiB | 250.13KiB | 5.11KiB | 0 | 0.0102418 | 23.85MiB | 252.66KiB | 5.17KiB | 0 | 0.0103452 | False | False |
splunk_hec_to_splunk_hec_logs_noack | -2.02KiB | -0.01 | 16.61% | 23.84MiB | 328.65KiB | 6.71KiB | 0 | 0.0134599 | 23.84MiB | 339.44KiB | 6.93KiB | 0 | 0.0139032 | False | False |
file_to_blackhole | -34.44KiB | -0.04 | 25.57% | 95.32MiB | 3.56MiB | 73.77KiB | 0 | 0.0373504 | 95.29MiB | 3.64MiB | 75.54KiB | 0 | 0.0381547 | False | False |
splunk_hec_to_splunk_hec_logs_acks | -13.91KiB | -0.06 | 42.21% | 23.76MiB | 853.45KiB | 17.36KiB | 0 | 0.0350717 | 23.75MiB | 883.87KiB | 17.97KiB | 0 | 0.0363427 | False | False |
splunk_hec_indexer_ack_blackhole | -20.58KiB | -0.08 | 61.57% | 23.76MiB | 778.73KiB | 15.84KiB | 0 | 0.0320002 | 23.74MiB | 863.97KiB | 17.56KiB | 0 | 0.0355329 | False | False |
http_to_http_json | -28.55KiB | -0.12 | 97.92% | 23.84MiB | 348.17KiB | 7.11KiB | 0 | 0.0142568 | 23.82MiB | 493.7KiB | 10.09KiB | 0 | 0.0202394 | False | False |
datadog_agent_remap_blackhole_acks | -96.86KiB | -0.16 | 49.05% | 59.74MiB | 5.43MiB | 113.02KiB | 0 | 0.0908232 | 59.65MiB | 4.48MiB | 93.75KiB | 0 | 0.0751597 | False | False |
syslog_log2metric_humio_metrics | -25.02KiB | -0.2 | 94.28% | 12.49MiB | 357.12KiB | 7.29KiB | 0 | 0.0279165 | 12.47MiB | 537.72KiB | 10.95KiB | 0 | 0.0421171 | False | False |
http_to_http_noack | -61.09KiB | -0.25 | 99.90% | 23.85MiB | 252.93KiB | 5.17KiB | 0 | 0.0103561 | 23.79MiB | 872.08KiB | 17.78KiB | 0 | 0.0357965 | False | False |
http_pipelines_blackhole_acks | -3.4KiB | -0.28 | 72.41% | 1.19MiB | 128.48KiB | 2.61KiB | 0 | 0.105048 | 1.19MiB | 83.64KiB | 1.71KiB | 0 | 0.0685727 | False | False |
splunk_hec_route_s3 | -98.85KiB | -0.52 | 85.26% | 18.41MiB | 2.33MiB | 48.44KiB | 0 | 0.126354 | 18.32MiB | 2.3MiB | 48.02KiB | 0 | 0.12543 | False | False |
syslog_regex_logs2metric_ddmetrics | -67.57KiB | -0.53 | 99.99% | 12.41MiB | 606.32KiB | 12.36KiB | 0 | 0.047698 | 12.35MiB | 562.17KiB | 11.46KiB | 0 | 0.0444613 | False | False |
fluent_elasticsearch | -460.31KiB | -0.57 | 100.00% | 79.47MiB | 54.44KiB | 1.1KiB | 0 | 0.000668811 | 79.02MiB | 4.51MiB | 92.57KiB | 0 | 0.0570605 | False | False |
http_pipelines_blackhole | -9.55KiB | -0.57 | 100.00% | 1.64MiB | 21.11KiB | 441.58B | 0 | 0.0125693 | 1.63MiB | 90.94KiB | 1.85KiB | 0 | 0.0544643 | False | False |
http_to_http_acks | -135.86KiB | -0.76 | 42.77% | 17.45MiB | 8.0MiB | 167.29KiB | 0 | 0.458422 | 17.32MiB | 8.28MiB | 172.87KiB | 0 | 0.477737 | True | True |
http_pipelines_no_grok_blackhole | -91.46KiB | -0.83 | 100.00% | 10.7MiB | 63.42KiB | 1.29KiB | 0 | 0.00578471 | 10.61MiB | 992.75KiB | 20.2KiB | 0 | 0.0913131 | False | False |
syslog_splunk_hec_logs | -307.74KiB | -1.83 | 100.00% | 16.41MiB | 755.5KiB | 15.37KiB | 0 | 0.0449384 | 16.11MiB | 620.69KiB | 12.66KiB | 0 | 0.0376084 | False | False |
socket_to_socket_blackhole | -587.4KiB | -2.5 | 100.00% | 22.98MiB | 659.73KiB | 13.47KiB | 0 | 0.0280263 | 22.41MiB | 625.16KiB | 12.77KiB | 0 | 0.0272375 | False | False |
syslog_humio_logs | -528.09KiB | -3.09 | 100.00% | 16.7MiB | 588.77KiB | 12.03KiB | 0 | 0.0344244 | 16.18MiB | 663.59KiB | 13.59KiB | 0 | 0.0400352 | False | False |
syslog_log2metric_splunk_hec_metrics | -823.41KiB | -4.53 | 100.00% | 17.73MiB | 1.01MiB | 21.05KiB | 0 | 0.0568606 | 16.93MiB | 1.19MiB | 24.77KiB | 0 | 0.0701829 | False | False |
Soak Test Results
Baseline: e80c7afaf7601cf936c7c3468bd7b4b230ef6149 Comparison: 351f40c9ea5bb8c46a5ad2387e4275f0aff3dde7 Total Vector CPUs: 4
Explanation
A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.
No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:
Fine details of change detection per experiment.
experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
http_to_http_acks | 232.15KiB | 1.34 | 67.38% | 16.91MiB | 8.0MiB | 167.36KiB | 0 | 0.473169 | 17.14MiB | 7.99MiB | 167.02KiB | 0 | 0.466263 | True | True |
http_pipelines_blackhole_acks | 9.53KiB | 0.8 | 99.54% | 1.16MiB | 131.51KiB | 2.68KiB | 0 | 0.110834 | 1.17MiB | 99.75KiB | 2.03KiB | 0 | 0.0834014 | False | False |
http_pipelines_blackhole | 6.16KiB | 0.37 | 92.22% | 1.62MiB | 97.31KiB | 1.99KiB | 0 | 0.0588284 | 1.62MiB | 140.79KiB | 2.87KiB | 0 | 0.0847979 | False | False |
splunk_hec_to_splunk_hec_logs_noack | 22.18KiB | 0.09 | 93.04% | 23.82MiB | 499.77KiB | 10.2KiB | 0 | 0.0204881 | 23.84MiB | 329.82KiB | 6.73KiB | 0 | 0.0135088 | False | False |
splunk_hec_indexer_ack_blackhole | 562.93B | 0 | 1.71% | 23.76MiB | 897.54KiB | 18.26KiB | 0 | 0.0368869 | 23.76MiB | 889.53KiB | 18.1KiB | 0 | 0.036557 | False | False |
splunk_hec_to_splunk_hec_logs_acks | 472.42B | 0 | 1.51% | 23.75MiB | 843.59KiB | 17.16KiB | 0 | 0.0346727 | 23.76MiB | 849.2KiB | 17.28KiB | 0 | 0.0349029 | False | False |
enterprise_http_to_http | 445.36B | 0 | 4.79% | 23.85MiB | 250.28KiB | 5.11KiB | 0 | 0.010248 | 23.85MiB | 250.35KiB | 5.12KiB | 0 | 0.0102504 | False | False |
syslog_regex_logs2metric_ddmetrics | -587.95B | -0 | 3.04% | 12.25MiB | 527.94KiB | 10.76KiB | 0 | 0.042081 | 12.25MiB | 518.76KiB | 10.57KiB | 0 | 0.0413514 | False | False |
file_to_blackhole | -48.81KiB | -0.05 | 40.00% | 95.34MiB | 3.04MiB | 62.92KiB | 0 | 0.031828 | 95.29MiB | 3.3MiB | 68.57KiB | 0 | 0.0345825 | False | False |
http_to_http_json | -40.31KiB | -0.17 | 99.81% | 23.85MiB | 329.78KiB | 6.73KiB | 0 | 0.0135012 | 23.81MiB | 542.2KiB | 11.07KiB | 0 | 0.0222344 | False | False |
fluent_elasticsearch | -199.07KiB | -0.24 | 100.00% | 79.47MiB | 53.25KiB | 1.08KiB | 0 | 0.000654207 | 79.28MiB | 1.72MiB | 35.47KiB | 0 | 0.021743 | False | False |
syslog_log2metric_splunk_hec_metrics | -58.84KiB | -0.34 | 93.90% | 16.91MiB | 1.04MiB | 21.62KiB | 0 | 0.0612589 | 16.86MiB | 1.09MiB | 22.78KiB | 0 | 0.0648229 | False | False |
http_text_to_http_json | -159.5KiB | -0.41 | 100.00% | 38.31MiB | 890.89KiB | 18.19KiB | 0 | 0.0227072 | 38.15MiB | 913.0KiB | 18.64KiB | 0 | 0.0233657 | False | False |
http_to_http_noack | -104.57KiB | -0.43 | 100.00% | 23.84MiB | 407.82KiB | 8.34KiB | 0 | 0.0167041 | 23.73MiB | 1.15MiB | 23.99KiB | 0 | 0.0484855 | False | False |
datadog_agent_remap_datadog_logs_acks | -331.71KiB | -0.53 | 99.62% | 61.4MiB | 3.38MiB | 70.6KiB | 0 | 0.0550436 | 61.08MiB | 4.34MiB | 90.26KiB | 0 | 0.0709798 | False | False |
syslog_loki | -79.19KiB | -0.54 | 99.98% | 14.34MiB | 610.22KiB | 12.5KiB | 0 | 0.0415426 | 14.26MiB | 844.33KiB | 17.16KiB | 0 | 0.0577918 | False | False |
syslog_log2metric_humio_metrics | -73.88KiB | -0.57 | 100.00% | 12.73MiB | 193.44KiB | 3.95KiB | 0 | 0.0148362 | 12.66MiB | 520.13KiB | 10.59KiB | 0 | 0.0401188 | False | False |
syslog_splunk_hec_logs | -154.06KiB | -0.93 | 100.00% | 16.1MiB | 738.01KiB | 15.02KiB | 0 | 0.0447508 | 15.95MiB | 582.22KiB | 11.88KiB | 0 | 0.0356373 | False | False |
datadog_agent_remap_datadog_logs | -610.57KiB | -0.96 | 100.00% | 62.1MiB | 794.71KiB | 16.27KiB | 0 | 0.0124938 | 61.51MiB | 4.2MiB | 87.54KiB | 0 | 0.0683305 | False | False |
datadog_agent_remap_blackhole_acks | -608.2KiB | -0.96 | 100.00% | 61.89MiB | 4.13MiB | 85.93KiB | 0 | 0.0666388 | 61.3MiB | 3.03MiB | 63.31KiB | 0 | 0.0494146 | False | False |
syslog_humio_logs | -220.44KiB | -1.3 | 100.00% | 16.57MiB | 121.84KiB | 2.49KiB | 0 | 0.00718059 | 16.35MiB | 176.52KiB | 3.61KiB | 0 | 0.0105403 | False | False |
http_pipelines_no_grok_blackhole | -150.13KiB | -1.34 | 100.00% | 10.95MiB | 42.22KiB | 882.52B | 0 | 0.00376477 | 10.8MiB | 1.1MiB | 22.95KiB | 0 | 0.102021 | False | False |
splunk_hec_route_s3 | -313.83KiB | -1.63 | 100.00% | 18.83MiB | 2.2MiB | 45.86KiB | 0 | 0.116906 | 18.53MiB | 2.2MiB | 45.96KiB | 0 | 0.118689 | False | False |
datadog_agent_remap_blackhole | -1.39MiB | -2.35 | 100.00% | 59.05MiB | 4.67MiB | 97.24KiB | 0 | 0.0790678 | 57.66MiB | 4.16MiB | 86.79KiB | 0 | 0.0721196 | False | False |
socket_to_socket_blackhole | -822.59KiB | -3.35 | 100.00% | 23.97MiB | 299.33KiB | 6.11KiB | 0 | 0.0121938 | 23.16MiB | 120.6KiB | 2.46KiB | 0 | 0.00508306 | False | False |
Soak Test Results
Baseline: a53cfefa5c5fd0c5c4603c0725adbdce4d4aeedd Comparison: f8a673b2772e5d128ef4a38e218d3fe7e3601ea4 Total Vector CPUs: 4
Explanation
A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.
No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:
Fine details of change detection per experiment.
experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
http_pipelines_no_grok_blackhole | 393.24KiB | 3.78 | 100.00% | 10.15MiB | 579.82KiB | 11.84KiB | 0 | 0.0557511 | 10.54MiB | 1.18MiB | 24.6KiB | 0 | 0.112141 | False | False |
syslog_log2metric_humio_metrics | 316.54KiB | 2.51 | 100.00% | 12.3MiB | 719.99KiB | 14.7KiB | 0 | 0.0571416 | 12.61MiB | 641.32KiB | 13.06KiB | 0 | 0.0496499 | False | False |
syslog_regex_logs2metric_ddmetrics | 109.38KiB | 0.88 | 100.00% | 12.13MiB | 451.51KiB | 9.2KiB | 0 | 0.0363372 | 12.24MiB | 487.44KiB | 9.94KiB | 0 | 0.0388863 | False | False |
http_pipelines_blackhole_acks | 6.94KiB | 0.56 | 97.88% | 1.22MiB | 119.0KiB | 2.42KiB | 0 | 0.0951874 | 1.23MiB | 87.75KiB | 1.79KiB | 0 | 0.0698008 | False | False |
http_to_http_acks | 91.51KiB | 0.52 | 30.54% | 17.35MiB | 7.57MiB | 158.23KiB | 0 | 0.436153 | 17.44MiB | 8.19MiB | 171.09KiB | 0 | 0.46979 | True | True |
datadog_agent_remap_datadog_logs_acks | 179.17KiB | 0.3 | 88.37% | 58.44MiB | 3.43MiB | 71.58KiB | 0 | 0.0586329 | 58.62MiB | 4.27MiB | 88.81KiB | 0 | 0.0727703 | False | False |
splunk_hec_to_splunk_hec_logs_noack | 17.64KiB | 0.07 | 86.42% | 23.82MiB | 474.88KiB | 9.7KiB | 0 | 0.0194649 | 23.84MiB | 331.41KiB | 6.77KiB | 0 | 0.0135746 | False | False |
splunk_hec_indexer_ack_blackhole | 10.6KiB | 0.04 | 32.26% | 23.75MiB | 898.97KiB | 18.29KiB | 0 | 0.0369587 | 23.76MiB | 871.32KiB | 17.73KiB | 0 | 0.0358065 | False | False |
enterprise_http_to_http | -317.25B | -0 | 3.38% | 23.85MiB | 254.19KiB | 5.19KiB | 0 | 0.0104079 | 23.85MiB | 251.89KiB | 5.16KiB | 0 | 0.0103136 | False | False |
splunk_hec_to_splunk_hec_logs_acks | -1.11KiB | -0 | 3.78% | 23.76MiB | 804.1KiB | 16.36KiB | 0 | 0.0330385 | 23.76MiB | 816.78KiB | 16.62KiB | 0 | 0.0335611 | False | False |
file_to_blackhole | -86.99KiB | -0.09 | 70.18% | 95.36MiB | 2.9MiB | 60.04KiB | 0 | 0.030366 | 95.27MiB | 2.8MiB | 58.19KiB | 0 | 0.0293817 | False | False |
http_to_http_json | -38.34KiB | -0.16 | 99.70% | 23.85MiB | 329.36KiB | 6.72KiB | 0 | 0.0134848 | 23.81MiB | 539.88KiB | 11.02KiB | 0 | 0.0221388 | False | False |
datadog_agent_remap_datadog_logs | -216.15KiB | -0.34 | 98.15% | 61.4MiB | 1.66MiB | 34.81KiB | 0 | 0.0270834 | 61.19MiB | 4.08MiB | 84.88KiB | 0 | 0.06658 | False | False |
http_to_http_noack | -95.51KiB | -0.39 | 100.00% | 23.85MiB | 251.87KiB | 5.15KiB | 0 | 0.010313 | 23.75MiB | 1.06MiB | 22.19KiB | 0 | 0.0447928 | False | False |
fluent_elasticsearch | -355.95KiB | -0.44 | 100.00% | 79.47MiB | 53.43KiB | 1.08KiB | 0 | 0.000656375 | 79.13MiB | 3.59MiB | 73.73KiB | 0 | 0.0453355 | False | False |
http_pipelines_blackhole | -11.65KiB | -0.67 | 100.00% | 1.71MiB | 27.61KiB | 577.86B | 0 | 0.0158089 | 1.69MiB | 123.92KiB | 2.52KiB | 0 | 0.0714267 | False | False |
splunk_hec_route_s3 | -131.81KiB | -0.7 | 95.40% | 18.51MiB | 2.25MiB | 46.87KiB | 0 | 0.121587 | 18.38MiB | 2.22MiB | 46.5KiB | 0 | 0.120948 | False | False |
datadog_agent_remap_blackhole_acks | -585.57KiB | -0.94 | 100.00% | 60.91MiB | 4.4MiB | 91.66KiB | 0 | 0.0722451 | 60.34MiB | 3.67MiB | 76.7KiB | 0 | 0.0608246 | False | False |
http_text_to_http_json | -498.31KiB | -1.25 | 100.00% | 38.79MiB | 838.02KiB | 17.11KiB | 0 | 0.0210926 | 38.3MiB | 845.23KiB | 17.25KiB | 0 | 0.0215443 | False | False |
socket_to_socket_blackhole | -310.14KiB | -1.3 | 100.00% | 23.22MiB | 256.08KiB | 5.23KiB | 0 | 0.0107693 | 22.91MiB | 333.26KiB | 6.8KiB | 0 | 0.0142001 | False | False |
datadog_agent_remap_blackhole | -919.53KiB | -1.49 | 100.00% | 60.46MiB | 3.89MiB | 81.15KiB | 0 | 0.0643849 | 59.57MiB | 3.11MiB | 64.86KiB | 0 | 0.0521892 | False | False |
syslog_loki | -248.64KiB | -1.68 | 100.00% | 14.48MiB | 429.41KiB | 8.79KiB | 0 | 0.0289532 | 14.24MiB | 738.06KiB | 15.0KiB | 0 | 0.0506135 | False | False |
syslog_splunk_hec_logs | -375.83KiB | -2.23 | 100.00% | 16.49MiB | 741.76KiB | 15.1KiB | 0 | 0.0439174 | 16.12MiB | 560.61KiB | 11.44KiB | 0 | 0.0339477 | False | False |
syslog_humio_logs | -388.67KiB | -2.3 | 100.00% | 16.48MiB | 261.31KiB | 5.34KiB | 0 | 0.0154797 | 16.1MiB | 215.05KiB | 4.4KiB | 0 | 0.0130391 | False | False |
syslog_log2metric_splunk_hec_metrics | -401.7KiB | -2.33 | 100.00% | 16.85MiB | 1014.1KiB | 20.64KiB | 0 | 0.0587467 | 16.46MiB | 975.54KiB | 19.84KiB | 0 | 0.0578598 | False | False |