vector
vector copied to clipboard
vectordotdev
chore: Spelling
This PR corrects misspellings identified by the check-spelling action.
The misspellings have been reported at #15969 (closes #15969)
The action reports that the changes in this PR would make it happy: https://github.com/jsoref/vector/actions/runs/3934856534#summary-10686499778
Note: this PR does not include the action. If you're interested in running a spell check on every PR and push, that can be offered separately.
Deploy Preview for vector-project canceled.
| Name | Link |
|---|---|
| Latest commit | f05a73d3f1a5ee9d5a7dfb5fbc1548e810427b09 |
| Latest deploy log | https://app.netlify.com/sites/vector-project/deploys/63c8516b0cef5400090fc3d4 |
![netlify[bot] avatar](https://avatars.githubusercontent.com/in/13473?v=4 "Published by a pub.dev verified publisher"){kind=small}
Deploy Preview for vrl-playground ready!
| Name | Link |
|---|---|
| Latest commit | f05a73d3f1a5ee9d5a7dfb5fbc1548e810427b09 |
| Latest deploy log | https://app.netlify.com/sites/vrl-playground/deploys/63c8516ba5cf8800099f1327 |
| Deploy Preview | https://deploy-preview-15970--vrl-playground.netlify.app |
| Preview on mobile | Toggle QR Code...Use your smartphone camera to open QR code link. |
To edit notification comments on pull requests, go to your Netlify site settings.
Regression Test Results
Run ID: 515c8fa3-a2e9-4578-ad54-3f28b63072b4
Baseline: 34af7e601cc043635efd1081b5a2823c8f225092
Comparison: 1d949d1b9a6f0178b81c3c704acb6f4155a69225
Total vector CPUs: 7
Explanation
A regression test is an integrated performance test for vector in
a repeatable rig, with varying configuration for vector. What
follows is a statistical summary of a brief vector run for each
configuration across SHAs given above. The goal of these tests are to determine,
quickly, if vector performance is changed and to what degree by a
pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a
statistically significant change in their bytes_written_per_cpu_second performance
between baseline and comparison SHAs, with 90.0%
confidence OR have been detected as newly erratic. Negative values mean that
baseline is faster, positive comparison. Results that do not exhibit more than a
±5% change in mean bytes_written_per_cpu_second are
discarded. An experiment is erratic if its coefficient of variation is greater
than 0.1. The abbreviated table will be
omitted if no interesting changes are observed.
No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.
Fine details of change detection per experiment.
| experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| socket_to_socket_blackhole | 363.44KiB/CPU-s | 2.72 | 100.00% | 13.04MiB/CPU-s | 421.45KiB/CPU-s | 5.44KiB/CPU-s | 0.0 | 0.031564 | 13.39MiB/CPU-s | 146.2KiB/CPU-s | 1.89KiB/CPU-s | 0.0 | 0.010659 | False | False |
| otlp_http_to_blackhole | 20.84KiB/CPU-s | 1.34 | 100.00% | 1.52MiB/CPU-s | 115.38KiB/CPU-s | 1.49KiB/CPU-s | 0.0 | 0.074054 | 1.54MiB/CPU-s | 114.34KiB/CPU-s | 1.48KiB/CPU-s | 0.0 | 0.072423 | False | False |
| syslog_log2metric_humio_metrics | 66.89KiB/CPU-s | 1.12 | 100.00% | 5.84MiB/CPU-s | 358.48KiB/CPU-s | 4.63KiB/CPU-s | 0.0 | 0.059937 | 5.91MiB/CPU-s | 283.8KiB/CPU-s | 3.66KiB/CPU-s | 0.0 | 0.046927 | False | False |
| http_text_to_http_json | 232.71KiB/CPU-s | 0.94 | 100.00% | 24.05MiB/CPU-s | 520.7KiB/CPU-s | 6.72KiB/CPU-s | 0.0 | 0.021138 | 24.28MiB/CPU-s | 581.65KiB/CPU-s | 7.51KiB/CPU-s | 0.0 | 0.023392 | False | False |
| http_to_http_acks | 45.01KiB/CPU-s | 0.84 | 61.60% | 5.22MiB/CPU-s | 2.74MiB/CPU-s | 36.2KiB/CPU-s | 0.0 | 0.524621 | 5.27MiB/CPU-s | 2.79MiB/CPU-s | 36.91KiB/CPU-s | 0.0 | 0.530427 | True | False |
| datadog_agent_remap_datadog_logs_acks | 103.52KiB/CPU-s | 0.3 | 100.00% | 33.48MiB/CPU-s | 1.22MiB/CPU-s | 16.13KiB/CPU-s | 0.0 | 0.036465 | 33.58MiB/CPU-s | 1.28MiB/CPU-s | 16.91KiB/CPU-s | 0.0 | 0.038108 | False | False |
| datadog_agent_remap_datadog_logs | 72.77KiB/CPU-s | 0.21 | 99.56% | 33.62MiB/CPU-s | 1.3MiB/CPU-s | 17.2KiB/CPU-s | 0.0 | 0.038724 | 33.69MiB/CPU-s | 1.43MiB/CPU-s | 18.89KiB/CPU-s | 0.0 | 0.04245 | False | False |
| enterprise_http_to_http | 12.26KiB/CPU-s | 0.09 | 98.17% | 13.61MiB/CPU-s | 353.16KiB/CPU-s | 4.56KiB/CPU-s | 0.0 | 0.025338 | 13.62MiB/CPU-s | 193.06KiB/CPU-s | 2.49KiB/CPU-s | 0.0 | 0.01384 | False | False |
| http_to_http_noack | 5.96KiB/CPU-s | 0.04 | 69.64% | 13.61MiB/CPU-s | 347.49KiB/CPU-s | 4.48KiB/CPU-s | 0.0 | 0.024931 | 13.62MiB/CPU-s | 284.63KiB/CPU-s | 3.67KiB/CPU-s | 0.0 | 0.020413 | False | False |
| splunk_hec_to_splunk_hec_logs_acks | 5.87KiB/CPU-s | 0.04 | 62.33% | 13.61MiB/CPU-s | 376.7KiB/CPU-s | 4.86KiB/CPU-s | 0.0 | 0.02702 | 13.62MiB/CPU-s | 350.95KiB/CPU-s | 4.53KiB/CPU-s | 0.0 | 0.025162 | False | False |
| splunk_hec_to_splunk_hec_logs_noack | 2.4KiB/CPU-s | 0.02 | 48.45% | 13.62MiB/CPU-s | 211.58KiB/CPU-s | 2.73KiB/CPU-s | 0.0 | 0.015166 | 13.63MiB/CPU-s | 192.96KiB/CPU-s | 2.49KiB/CPU-s | 0.0 | 0.013829 | False | False |
| fluent_elasticsearch | -160.1B/CPU-s | -0.0 | 22.46% | 45.41MiB/CPU-s | 30.45KiB/CPU-s | 397.6B/CPU-s | 0.0 | 0.000655 | 45.41MiB/CPU-s | 30.32KiB/CPU-s | 396.04B/CPU-s | 0.0 | 0.000652 | False | False |
| splunk_hec_indexer_ack_blackhole | 161.94B/CPU-s | 0.0 | 2.75% | 13.62MiB/CPU-s | 251.77KiB/CPU-s | 3.25KiB/CPU-s | 0.0 | 0.018055 | 13.62MiB/CPU-s | 250.49KiB/CPU-s | 3.23KiB/CPU-s | 0.0 | 0.017963 | False | False |
| file_to_blackhole | -13.82KiB/CPU-s | -0.02 | 47.11% | 54.5MiB/CPU-s | 1.08MiB/CPU-s | 14.2KiB/CPU-s | 0.0 | 0.019726 | 54.48MiB/CPU-s | 1.27MiB/CPU-s | 16.74KiB/CPU-s | 0.0 | 0.023273 | False | False |
| syslog_regex_logs2metric_ddmetrics | -4.12KiB/CPU-s | -0.11 | 39.49% | 3.7MiB/CPU-s | 415.12KiB/CPU-s | 5.36KiB/CPU-s | 0.0 | 0.109651 | 3.69MiB/CPU-s | 457.34KiB/CPU-s | 5.9KiB/CPU-s | 0.0 | 0.120935 | True | False |
| http_to_http_json | -28.52KiB/CPU-s | -0.2 | 100.00% | 13.62MiB/CPU-s | 209.9KiB/CPU-s | 2.71KiB/CPU-s | 0.0 | 0.015046 | 13.59MiB/CPU-s | 278.33KiB/CPU-s | 3.59KiB/CPU-s | 0.0 | 0.019992 | False | False |
| otlp_grpc_to_blackhole | -4.65KiB/CPU-s | -0.43 | 100.00% | 1.05MiB/CPU-s | 42.83KiB/CPU-s | 566.01B/CPU-s | 0.0 | 0.04 | 1.04MiB/CPU-s | 55.0KiB/CPU-s | 726.57B/CPU-s | 0.0 | 0.051592 | False | False |
| datadog_agent_remap_blackhole | -166.15KiB/CPU-s | -0.52 | 100.00% | 30.94MiB/CPU-s | 1.27MiB/CPU-s | 16.76KiB/CPU-s | 0.0 | 0.040981 | 30.78MiB/CPU-s | 1.6MiB/CPU-s | 21.14KiB/CPU-s | 0.0 | 0.051986 | False | False |
| syslog_splunk_hec_logs | -66.54KiB/CPU-s | -0.75 | 100.00% | 8.71MiB/CPU-s | 381.9KiB/CPU-s | 4.93KiB/CPU-s | 0.0 | 0.04282 | 8.64MiB/CPU-s | 325.65KiB/CPU-s | 4.2KiB/CPU-s | 0.0 | 0.036788 | False | False |
| syslog_humio_logs | -90.28KiB/CPU-s | -0.97 | 100.00% | 9.07MiB/CPU-s | 265.34KiB/CPU-s | 3.43KiB/CPU-s | 0.0 | 0.028567 | 8.98MiB/CPU-s | 312.61KiB/CPU-s | 4.04KiB/CPU-s | 0.0 | 0.033986 | False | False |
| syslog_loki | -119.56KiB/CPU-s | -1.32 | 100.00% | 8.87MiB/CPU-s | 219.36KiB/CPU-s | 2.83KiB/CPU-s | 0.0 | 0.024156 | 8.75MiB/CPU-s | 288.69KiB/CPU-s | 3.73KiB/CPU-s | 0.0 | 0.032215 | False | False |
| splunk_hec_route_s3 | -188.36KiB/CPU-s | -1.57 | 100.00% | 11.68MiB/CPU-s | 580.76KiB/CPU-s | 7.49KiB/CPU-s | 0.0 | 0.048537 | 11.5MiB/CPU-s | 724.49KiB/CPU-s | 9.35KiB/CPU-s | 0.0 | 0.061517 | False | False |
| datadog_agent_remap_blackhole_acks | -770.89KiB/CPU-s | -2.38 | 100.00% | 31.68MiB/CPU-s | 674.14KiB/CPU-s | 8.7KiB/CPU-s | 0.0 | 0.02078 | 30.93MiB/CPU-s | 2.18MiB/CPU-s | 28.83KiB/CPU-s | 0.0 | 0.070528 | False | False |
| syslog_log2metric_splunk_hec_metrics | -332.95KiB/CPU-s | -3.53 | 100.00% | 9.21MiB/CPU-s | 393.45KiB/CPU-s | 5.08KiB/CPU-s | 0.0 | 0.041696 | 8.89MiB/CPU-s | 575.28KiB/CPU-s | 7.43KiB/CPU-s | 0.0 | 0.063196 | False | False |
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Sorry, I won't do force-pushes once someone starts reviewing -- this should clear up the cue complaints (which are whitespace drift from changing the length of certain fields)
Regression Test Results
Run ID: 2252c0a2-e466-4d58-99d4-7f2cadfaff17
Baseline: e12a08ca7bd2d6f1e0cadecad7616cd4adf587ee
Comparison: 05e78e8baa174acdb83739425f8ad15cc8a420a8
Total vector CPUs: 7
Explanation
A regression test is an integrated performance test for vector in
a repeatable rig, with varying configuration for vector. What
follows is a statistical summary of a brief vector run for each
configuration across SHAs given above. The goal of these tests are to determine,
quickly, if vector performance is changed and to what degree by a
pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a
statistically significant change in their bytes_written_per_cpu_second performance
between baseline and comparison SHAs, with 90.0%
confidence OR have been detected as newly erratic. Negative values mean that
baseline is faster, positive comparison. Results that do not exhibit more than a
±5% change in mean bytes_written_per_cpu_second are
discarded. An experiment is erratic if its coefficient of variation is greater
than 0.1. The abbreviated table will be
omitted if no interesting changes are observed.
Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:
| experiment | Δ mean | Δ mean % | confidence |
|---|---|---|---|
| syslog_regex_logs2metric_ddmetrics | 320.92KiB/CPU-s | 9.07 | 100.00% |
Fine details of change detection per experiment.
| experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| syslog_regex_logs2metric_ddmetrics | 320.92KiB/CPU-s | 9.07 | 100.00% | 3.45MiB/CPU-s | 560.58KiB/CPU-s | 7.24KiB/CPU-s | 0.0 | 0.158478 | 3.77MiB/CPU-s | 440.35KiB/CPU-s | 5.68KiB/CPU-s | 0.0 | 0.114133 | True | False |
| datadog_agent_remap_blackhole_acks | 1.23MiB/CPU-s | 4.0 | 100.00% | 30.64MiB/CPU-s | 1.71MiB/CPU-s | 22.64KiB/CPU-s | 0.0 | 0.055872 | 31.87MiB/CPU-s | 1.32MiB/CPU-s | 17.48KiB/CPU-s | 0.0 | 0.041504 | False | False |
| http_text_to_http_json | 822.77KiB/CPU-s | 3.4 | 100.00% | 23.66MiB/CPU-s | 852.28KiB/CPU-s | 11.0KiB/CPU-s | 0.0 | 0.035179 | 24.46MiB/CPU-s | 519.31KiB/CPU-s | 6.7KiB/CPU-s | 0.0 | 0.020731 | False | False |
| otlp_http_to_blackhole | 31.06KiB/CPU-s | 2.0 | 100.00% | 1.52MiB/CPU-s | 118.64KiB/CPU-s | 1.53KiB/CPU-s | 0.0 | 0.076238 | 1.55MiB/CPU-s | 115.26KiB/CPU-s | 1.49KiB/CPU-s | 0.0 | 0.072619 | False | False |
| otlp_grpc_to_blackhole | 16.91KiB/CPU-s | 1.61 | 100.00% | 1.02MiB/CPU-s | 44.63KiB/CPU-s | 590.04B/CPU-s | 0.0 | 0.042535 | 1.04MiB/CPU-s | 55.13KiB/CPU-s | 728.39B/CPU-s | 0.0 | 0.051714 | False | False |
| http_to_http_acks | 81.51KiB/CPU-s | 1.55 | 89.13% | 5.14MiB/CPU-s | 2.75MiB/CPU-s | 36.38KiB/CPU-s | 0.0 | 0.53527 | 5.22MiB/CPU-s | 2.68MiB/CPU-s | 35.48KiB/CPU-s | 0.0 | 0.513901 | True | False |
| syslog_humio_logs | 85.33KiB/CPU-s | 0.92 | 100.00% | 9.07MiB/CPU-s | 249.72KiB/CPU-s | 3.22KiB/CPU-s | 0.0 | 0.02687 | 9.16MiB/CPU-s | 163.18KiB/CPU-s | 2.11KiB/CPU-s | 0.0 | 0.017398 | False | False |
| syslog_log2metric_humio_metrics | 42.14KiB/CPU-s | 0.69 | 100.00% | 5.97MiB/CPU-s | 311.76KiB/CPU-s | 4.02KiB/CPU-s | 0.0 | 0.051011 | 6.01MiB/CPU-s | 283.09KiB/CPU-s | 3.66KiB/CPU-s | 0.0 | 0.046002 | False | False |
| socket_to_socket_blackhole | 80.07KiB/CPU-s | 0.6 | 100.00% | 13.11MiB/CPU-s | 334.48KiB/CPU-s | 4.32KiB/CPU-s | 0.0 | 0.024905 | 13.19MiB/CPU-s | 394.96KiB/CPU-s | 5.1KiB/CPU-s | 0.0 | 0.029234 | False | False |
| syslog_splunk_hec_logs | 50.94KiB/CPU-s | 0.55 | 100.00% | 9.0MiB/CPU-s | 352.28KiB/CPU-s | 4.55KiB/CPU-s | 0.0 | 0.038231 | 9.05MiB/CPU-s | 229.71KiB/CPU-s | 2.96KiB/CPU-s | 0.0 | 0.024792 | False | False |
| enterprise_http_to_http | 8.24KiB/CPU-s | 0.06 | 92.46% | 13.61MiB/CPU-s | 303.85KiB/CPU-s | 3.92KiB/CPU-s | 0.0 | 0.021794 | 13.62MiB/CPU-s | 191.75KiB/CPU-s | 2.48KiB/CPU-s | 0.0 | 0.013746 | False | False |
| splunk_hec_to_splunk_hec_logs_noack | 4.69KiB/CPU-s | 0.03 | 78.08% | 13.62MiB/CPU-s | 225.12KiB/CPU-s | 2.91KiB/CPU-s | 0.0 | 0.016139 | 13.63MiB/CPU-s | 192.12KiB/CPU-s | 2.48KiB/CPU-s | 0.0 | 0.013768 | False | False |
| http_to_http_noack | 2.1KiB/CPU-s | 0.02 | 32.77% | 13.62MiB/CPU-s | 285.62KiB/CPU-s | 3.69KiB/CPU-s | 0.0 | 0.020484 | 13.62MiB/CPU-s | 256.51KiB/CPU-s | 3.31KiB/CPU-s | 0.0 | 0.018393 | False | False |
| fluent_elasticsearch | 134.74B/CPU-s | 0.0 | 19.06% | 45.41MiB/CPU-s | 30.03KiB/CPU-s | 392.77B/CPU-s | 0.0 | 0.000646 | 45.41MiB/CPU-s | 30.4KiB/CPU-s | 397.3B/CPU-s | 0.0 | 0.000654 | False | False |
| splunk_hec_to_splunk_hec_logs_acks | 146.41B/CPU-s | 0.0 | 1.90% | 13.62MiB/CPU-s | 334.12KiB/CPU-s | 4.31KiB/CPU-s | 0.0 | 0.023961 | 13.62MiB/CPU-s | 325.38KiB/CPU-s | 4.2KiB/CPU-s | 0.0 | 0.023333 | False | False |
| file_to_blackhole | -7.89KiB/CPU-s | -0.01 | 28.12% | 54.5MiB/CPU-s | 1.15MiB/CPU-s | 15.19KiB/CPU-s | 0.0 | 0.021097 | 54.49MiB/CPU-s | 1.2MiB/CPU-s | 15.81KiB/CPU-s | 0.0 | 0.021973 | False | False |
| splunk_hec_indexer_ack_blackhole | -4.07KiB/CPU-s | -0.03 | 63.60% | 13.62MiB/CPU-s | 235.46KiB/CPU-s | 3.04KiB/CPU-s | 0.0 | 0.016882 | 13.62MiB/CPU-s | 256.08KiB/CPU-s | 3.3KiB/CPU-s | 0.0 | 0.018365 | False | False |
| datadog_agent_remap_datadog_logs | -43.04KiB/CPU-s | -0.12 | 91.81% | 33.95MiB/CPU-s | 1.34MiB/CPU-s | 17.64KiB/CPU-s | 0.0 | 0.039346 | 33.9MiB/CPU-s | 1.31MiB/CPU-s | 17.34KiB/CPU-s | 0.0 | 0.038727 | False | False |
| datadog_agent_remap_datadog_logs_acks | -57.95KiB/CPU-s | -0.17 | 96.16% | 33.26MiB/CPU-s | 1.41MiB/CPU-s | 18.66KiB/CPU-s | 0.0 | 0.042452 | 33.2MiB/CPU-s | 1.58MiB/CPU-s | 20.86KiB/CPU-s | 0.0 | 0.047557 | False | False |
| syslog_log2metric_splunk_hec_metrics | -22.77KiB/CPU-s | -0.24 | 99.90% | 9.36MiB/CPU-s | 391.66KiB/CPU-s | 5.06KiB/CPU-s | 0.0 | 0.040878 | 9.33MiB/CPU-s | 368.55KiB/CPU-s | 4.76KiB/CPU-s | 0.0 | 0.038558 | False | False |
| http_to_http_json | -117.83KiB/CPU-s | -0.84 | 100.00% | 13.62MiB/CPU-s | 210.43KiB/CPU-s | 2.71KiB/CPU-s | 0.0 | 0.015085 | 13.51MiB/CPU-s | 348.3KiB/CPU-s | 4.49KiB/CPU-s | 0.0 | 0.02518 | False | False |
| splunk_hec_route_s3 | -138.29KiB/CPU-s | -1.15 | 100.00% | 11.7MiB/CPU-s | 600.38KiB/CPU-s | 7.74KiB/CPU-s | 0.0 | 0.05011 | 11.56MiB/CPU-s | 661.87KiB/CPU-s | 8.54KiB/CPU-s | 0.0 | 0.055887 | False | False |
| datadog_agent_remap_blackhole | -543.71KiB/CPU-s | -1.7 | 100.00% | 31.18MiB/CPU-s | 1.24MiB/CPU-s | 16.36KiB/CPU-s | 0.0 | 0.039707 | 30.65MiB/CPU-s | 1.58MiB/CPU-s | 20.84KiB/CPU-s | 0.0 | 0.051457 | False | False |
| syslog_loki | -186.48KiB/CPU-s | -2.13 | 100.00% | 8.56MiB/CPU-s | 274.44KiB/CPU-s | 3.54KiB/CPU-s | 0.0 | 0.03131 | 8.38MiB/CPU-s | 430.42KiB/CPU-s | 5.56KiB/CPU-s | 0.0 | 0.050173 | False | False |
cc @jszwedko for thoughts - I think these are populated by the commit messages, but I'm not really aware of how we feel about changing changelogs (do you need an item in the changelog for changing the changelog? 😂 )
I do have to update these sometimes as the scopes aren't always valid, but maybe this is an interesting use-case for @jsoref to consider since the changelogs are directly taken from commit messages and so we may want to preserve the (incorrect) spelling and/or run the spell check action against pull request titles and descriptions to avoid misspellings getting into there and only being discovered later when generating the changelog.
Also noting, the CI failures for cross were caused by a defect that got merged in - re-merging master should resolve.
check-spelling can check titles/descriptions precisely for this use case.
Note: I don't have many customers using this feature yet (it's fairly new), so you might be the first, but I should be responsive enough to help out (just tag me here, or possibly on discord) if you run into problems w/ it.
Afaict, it more or less just works. I suspect that people will want some way to say "don't check this PR", but in order to design that, I'd need a project that uses the feature to provide input (people have asked for the feature, but we haven't started using it internally, and our internal PRs aren't rigorous enough for it to work atm, although that might change). Offhand, this project seems to use labels, so, my guess is that you'd want it to be responsive to a label (which wouldn't be particularly hard). But, input for how such a thing should work is definitely wanted/desired.
The config is currently active in the branch I used for this: https://github.com/jsoref/vector/blob/e8475d01f8880d34b896a24379bab309fc091d7f/.github/workflows/spelling.yml#L108
Regression Test Results
Run ID: 7903935e-33e1-436e-8c5a-1aaf48260edc
Baseline: 00b0d9c68b6991e6e953ad80dad243dd7a10ac2f
Comparison: b374cb7c96689f88a3a2f0f51f7120ffe0c94fcc
Total vector CPUs: 7
Explanation
A regression test is an integrated performance test for vector in
a repeatable rig, with varying configuration for vector. What
follows is a statistical summary of a brief vector run for each
configuration across SHAs given above. The goal of these tests are to determine,
quickly, if vector performance is changed and to what degree by a
pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a
statistically significant change in their bytes_written_per_cpu_second performance
between baseline and comparison SHAs, with 90.0%
confidence OR have been detected as newly erratic. Negative values mean that
baseline is faster, positive comparison. Results that do not exhibit more than a
±5% change in mean bytes_written_per_cpu_second are
discarded. An experiment is erratic if its coefficient of variation is greater
than 0.1. The abbreviated table will be
omitted if no interesting changes are observed.
No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.
Fine details of change detection per experiment.
| experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| syslog_loki | 152.06KiB/CPU-s | 1.73 | 100.00% | 8.58MiB/CPU-s | 339.11KiB/CPU-s | 4.38KiB/CPU-s | 0.0 | 0.038609 | 8.72MiB/CPU-s | 226.59KiB/CPU-s | 2.93KiB/CPU-s | 0.0 | 0.02536 | False | False |
| datadog_agent_remap_datadog_logs_acks | 442.93KiB/CPU-s | 1.34 | 100.00% | 32.29MiB/CPU-s | 1.73MiB/CPU-s | 22.86KiB/CPU-s | 0.0 | 0.053583 | 32.72MiB/CPU-s | 1.28MiB/CPU-s | 16.95KiB/CPU-s | 0.0 | 0.039199 | False | False |
| http_text_to_http_json | 291.42KiB/CPU-s | 1.23 | 100.00% | 23.17MiB/CPU-s | 801.48KiB/CPU-s | 10.34KiB/CPU-s | 0.0 | 0.033772 | 23.46MiB/CPU-s | 734.49KiB/CPU-s | 9.48KiB/CPU-s | 0.0 | 0.030573 | False | False |
| syslog_humio_logs | 34.66KiB/CPU-s | 0.37 | 100.00% | 9.12MiB/CPU-s | 183.27KiB/CPU-s | 2.37KiB/CPU-s | 0.0 | 0.019629 | 9.15MiB/CPU-s | 139.52KiB/CPU-s | 1.8KiB/CPU-s | 0.0 | 0.014887 | False | False |
| socket_to_socket_blackhole | 34.44KiB/CPU-s | 0.27 | 100.00% | 12.68MiB/CPU-s | 469.32KiB/CPU-s | 6.06KiB/CPU-s | 0.0 | 0.036128 | 12.72MiB/CPU-s | 454.26KiB/CPU-s | 5.87KiB/CPU-s | 0.0 | 0.034877 | False | False |
| syslog_log2metric_splunk_hec_metrics | 17.11KiB/CPU-s | 0.18 | 99.97% | 9.05MiB/CPU-s | 214.59KiB/CPU-s | 2.77KiB/CPU-s | 0.0 | 0.023162 | 9.06MiB/CPU-s | 299.55KiB/CPU-s | 3.87KiB/CPU-s | 0.0 | 0.032273 | False | False |
| splunk_hec_to_splunk_hec_logs_noack | 9.37KiB/CPU-s | 0.07 | 93.76% | 13.62MiB/CPU-s | 336.21KiB/CPU-s | 4.34KiB/CPU-s | 0.0 | 0.024111 | 13.63MiB/CPU-s | 196.99KiB/CPU-s | 2.54KiB/CPU-s | 0.0 | 0.014118 | False | False |
| enterprise_http_to_http | 6.13KiB/CPU-s | 0.04 | 88.81% | 13.62MiB/CPU-s | 258.6KiB/CPU-s | 3.34KiB/CPU-s | 0.0 | 0.018543 | 13.62MiB/CPU-s | 149.55KiB/CPU-s | 1.93KiB/CPU-s | 0.0 | 0.010719 | False | False |
| file_to_blackhole | 5.22KiB/CPU-s | 0.01 | 19.61% | 54.5MiB/CPU-s | 1.06MiB/CPU-s | 14.02KiB/CPU-s | 0.0 | 0.019477 | 54.5MiB/CPU-s | 1.19MiB/CPU-s | 15.68KiB/CPU-s | 0.0 | 0.021798 | False | False |
| http_to_http_json | 646.21B/CPU-s | 0.0 | 10.70% | 13.6MiB/CPU-s | 263.91KiB/CPU-s | 3.41KiB/CPU-s | 0.0 | 0.018949 | 13.6MiB/CPU-s | 250.23KiB/CPU-s | 3.23KiB/CPU-s | 0.0 | 0.017966 | False | False |
| fluent_elasticsearch | -46.58B/CPU-s | -0.0 | 6.73% | 45.41MiB/CPU-s | 29.72KiB/CPU-s | 388.53B/CPU-s | 0.0 | 0.000639 | 45.41MiB/CPU-s | 29.94KiB/CPU-s | 391.49B/CPU-s | 0.0 | 0.000644 | False | False |
| splunk_hec_indexer_ack_blackhole | -1.8KiB/CPU-s | -0.01 | 30.38% | 13.62MiB/CPU-s | 246.97KiB/CPU-s | 3.19KiB/CPU-s | 0.0 | 0.017709 | 13.62MiB/CPU-s | 257.01KiB/CPU-s | 3.32KiB/CPU-s | 0.0 | 0.018431 | False | False |
| splunk_hec_to_splunk_hec_logs_acks | -1.39KiB/CPU-s | -0.01 | 16.19% | 13.61MiB/CPU-s | 364.53KiB/CPU-s | 4.7KiB/CPU-s | 0.0 | 0.026145 | 13.61MiB/CPU-s | 379.31KiB/CPU-s | 4.89KiB/CPU-s | 0.0 | 0.027208 | False | False |
| http_to_http_noack | -6.47KiB/CPU-s | -0.05 | 80.84% | 13.62MiB/CPU-s | 227.5KiB/CPU-s | 2.94KiB/CPU-s | 0.0 | 0.01631 | 13.61MiB/CPU-s | 309.53KiB/CPU-s | 3.99KiB/CPU-s | 0.0 | 0.022202 | False | False |
| http_to_http_acks | -4.23KiB/CPU-s | -0.08 | 6.47% | 5.27MiB/CPU-s | 2.81MiB/CPU-s | 37.14KiB/CPU-s | 0.0 | 0.533064 | 5.27MiB/CPU-s | 2.76MiB/CPU-s | 36.47KiB/CPU-s | 0.0 | 0.523907 | True | False |
| datadog_agent_remap_datadog_logs | -102.15KiB/CPU-s | -0.3 | 99.87% | 33.1MiB/CPU-s | 1.82MiB/CPU-s | 24.03KiB/CPU-s | 0.0 | 0.054955 | 33.01MiB/CPU-s | 1.56MiB/CPU-s | 20.61KiB/CPU-s | 0.0 | 0.047275 | False | False |
| otlp_grpc_to_blackhole | -3.44KiB/CPU-s | -0.32 | 100.00% | 1.04MiB/CPU-s | 38.26KiB/CPU-s | 505.69B/CPU-s | 0.0 | 0.036026 | 1.03MiB/CPU-s | 44.35KiB/CPU-s | 586.17B/CPU-s | 0.0 | 0.041898 | False | False |
| datadog_agent_remap_blackhole | -124.62KiB/CPU-s | -0.39 | 100.00% | 31.17MiB/CPU-s | 808.13KiB/CPU-s | 10.44KiB/CPU-s | 0.0 | 0.025318 | 31.05MiB/CPU-s | 983.74KiB/CPU-s | 12.7KiB/CPU-s | 0.0 | 0.03094 | False | False |
| otlp_http_to_blackhole | -9.6KiB/CPU-s | -0.62 | 100.00% | 1.5MiB/CPU-s | 121.63KiB/CPU-s | 1.57KiB/CPU-s | 0.0 | 0.078951 | 1.49MiB/CPU-s | 129.49KiB/CPU-s | 1.67KiB/CPU-s | 0.0 | 0.08458 | False | False |
| syslog_splunk_hec_logs | -89.35KiB/CPU-s | -0.97 | 100.00% | 8.99MiB/CPU-s | 257.8KiB/CPU-s | 3.33KiB/CPU-s | 0.0 | 0.027997 | 8.9MiB/CPU-s | 269.07KiB/CPU-s | 3.47KiB/CPU-s | 0.0 | 0.029506 | False | False |
| syslog_regex_logs2metric_ddmetrics | -38.01KiB/CPU-s | -1.0 | 100.00% | 3.71MiB/CPU-s | 422.63KiB/CPU-s | 5.45KiB/CPU-s | 0.0 | 0.111329 | 3.67MiB/CPU-s | 440.44KiB/CPU-s | 5.69KiB/CPU-s | 0.0 | 0.117192 | True | False |
| datadog_agent_remap_blackhole_acks | -381.0KiB/CPU-s | -1.18 | 100.00% | 31.47MiB/CPU-s | 777.93KiB/CPU-s | 10.05KiB/CPU-s | 0.0 | 0.024142 | 31.09MiB/CPU-s | 1.36MiB/CPU-s | 18.0KiB/CPU-s | 0.0 | 0.04378 | False | False |
| splunk_hec_route_s3 | -190.44KiB/CPU-s | -1.6 | 100.00% | 11.64MiB/CPU-s | 608.19KiB/CPU-s | 7.85KiB/CPU-s | 0.0 | 0.051043 | 11.45MiB/CPU-s | 662.44KiB/CPU-s | 8.55KiB/CPU-s | 0.0 | 0.056499 | False | False |
| syslog_log2metric_humio_metrics | -321.59KiB/CPU-s | -4.96 | 100.00% | 6.34MiB/CPU-s | 76.72KiB/CPU-s | 1014.26B/CPU-s | 0.0 | 0.011825 | 6.02MiB/CPU-s | 299.27KiB/CPU-s | 3.86KiB/CPU-s | 0.0 | 0.048533 | False | False |
Thanks for all this @jsoref! I think we'll need a bit of discussion and consensus for adding a CI check for this, was that an issue you already opened?
Regression Test Results
Run ID: b2186599-c1f8-49dc-8dbd-e39964731233
Baseline: 3f6d32f05d1bb919b39048922f5d053e3f17dd5a
Comparison: f05a73d3f1a5ee9d5a7dfb5fbc1548e810427b09
Total vector CPUs: 7
Explanation
A regression test is an integrated performance test for vector in
a repeatable rig, with varying configuration for vector. What
follows is a statistical summary of a brief vector run for each
configuration across SHAs given above. The goal of these tests are to determine,
quickly, if vector performance is changed and to what degree by a
pull request. Where appropriate units are scaled per-core.
The table below, if present, lists those experiments that have experienced a
statistically significant change in their bytes_written_per_cpu_second performance
between baseline and comparison SHAs, with 90.0%
confidence OR have been detected as newly erratic. Negative values mean that
baseline is faster, positive comparison. Results that do not exhibit more than a
±5% change in mean bytes_written_per_cpu_second are
discarded. An experiment is erratic if its coefficient of variation is greater
than 0.1. The abbreviated table will be
omitted if no interesting changes are observed.
No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.
Fine details of change detection per experiment.
| experiment | Δ mean | Δ mean % | confidence | baseline mean | baseline stdev | baseline stderr | baseline outlier % | baseline CoV | comparison mean | comparison stdev | comparison stderr | comparison outlier % | comparison CoV | erratic | declared erratic |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| datadog_agent_remap_blackhole | 506.08KiB/CPU-s | 1.61 | 100.00% | 30.79MiB/CPU-s | 1.71MiB/CPU-s | 22.58KiB/CPU-s | 0.0 | 0.055473 | 31.28MiB/CPU-s | 1.09MiB/CPU-s | 14.4KiB/CPU-s | 0.0 | 0.034823 | False | False |
| syslog_log2metric_humio_metrics | 52.89KiB/CPU-s | 0.83 | 100.00% | 6.21MiB/CPU-s | 96.7KiB/CPU-s | 1.25KiB/CPU-s | 0.0 | 0.015198 | 6.26MiB/CPU-s | 132.49KiB/CPU-s | 1.71KiB/CPU-s | 0.0 | 0.020652 | False | False |
| datadog_agent_remap_datadog_logs_acks | 131.03KiB/CPU-s | 0.38 | 100.00% | 33.83MiB/CPU-s | 964.26KiB/CPU-s | 12.44KiB/CPU-s | 0.0 | 0.027834 | 33.96MiB/CPU-s | 911.81KiB/CPU-s | 11.77KiB/CPU-s | 0.0 | 0.026221 | False | False |
| syslog_loki | 32.41KiB/CPU-s | 0.36 | 100.00% | 8.79MiB/CPU-s | 332.71KiB/CPU-s | 4.29KiB/CPU-s | 0.0 | 0.036956 | 8.82MiB/CPU-s | 137.53KiB/CPU-s | 1.87KiB/CPU-s | 0.0 | 0.015221 | False | False |
| http_to_http_json | 38.0KiB/CPU-s | 0.27 | 100.00% | 13.58MiB/CPU-s | 277.86KiB/CPU-s | 3.59KiB/CPU-s | 0.0 | 0.019982 | 13.62MiB/CPU-s | 229.31KiB/CPU-s | 2.96KiB/CPU-s | 0.0 | 0.016446 | False | False |
| syslog_regex_logs2metric_ddmetrics | 8.65KiB/CPU-s | 0.23 | 71.21% | 3.64MiB/CPU-s | 396.86KiB/CPU-s | 5.12KiB/CPU-s | 0.0 | 0.10647 | 3.65MiB/CPU-s | 490.06KiB/CPU-s | 6.33KiB/CPU-s | 0.0 | 0.131169 | True | False |
| enterprise_http_to_http | 12.35KiB/CPU-s | 0.09 | 99.17% | 13.61MiB/CPU-s | 330.21KiB/CPU-s | 4.26KiB/CPU-s | 0.0 | 0.023688 | 13.62MiB/CPU-s | 149.5KiB/CPU-s | 1.93KiB/CPU-s | 0.0 | 0.010715 | False | False |
| splunk_hec_to_splunk_hec_logs_noack | 5.91KiB/CPU-s | 0.04 | 86.67% | 13.62MiB/CPU-s | 237.08KiB/CPU-s | 3.06KiB/CPU-s | 0.0 | 0.016998 | 13.63MiB/CPU-s | 191.63KiB/CPU-s | 2.47KiB/CPU-s | 0.0 | 0.013734 | False | False |
| datadog_agent_remap_datadog_logs | 6.19KiB/CPU-s | 0.02 | 15.74% | 33.44MiB/CPU-s | 1.76MiB/CPU-s | 23.27KiB/CPU-s | 0.0 | 0.052682 | 33.45MiB/CPU-s | 1.57MiB/CPU-s | 20.71KiB/CPU-s | 0.0 | 0.046879 | False | False |
| file_to_blackhole | -1.3KiB/CPU-s | -0.0 | 4.86% | 54.49MiB/CPU-s | 1.12MiB/CPU-s | 14.79KiB/CPU-s | 0.0 | 0.020549 | 54.49MiB/CPU-s | 1.17MiB/CPU-s | 15.42KiB/CPU-s | 0.0 | 0.021428 | False | False |
| fluent_elasticsearch | -73.35B/CPU-s | -0.0 | 10.46% | 45.41MiB/CPU-s | 30.06KiB/CPU-s | 392.91B/CPU-s | 0.0 | 0.000646 | 45.41MiB/CPU-s | 30.34KiB/CPU-s | 396.45B/CPU-s | 0.0 | 0.000652 | False | False |
| splunk_hec_indexer_ack_blackhole | -2.94KiB/CPU-s | -0.02 | 48.46% | 13.62MiB/CPU-s | 239.58KiB/CPU-s | 3.09KiB/CPU-s | 0.0 | 0.017178 | 13.62MiB/CPU-s | 256.04KiB/CPU-s | 3.3KiB/CPU-s | 0.0 | 0.018362 | False | False |
| splunk_hec_to_splunk_hec_logs_acks | -3.61KiB/CPU-s | -0.03 | 43.85% | 13.62MiB/CPU-s | 332.71KiB/CPU-s | 4.29KiB/CPU-s | 0.0 | 0.023855 | 13.62MiB/CPU-s | 349.42KiB/CPU-s | 4.51KiB/CPU-s | 0.0 | 0.02506 | False | False |
| http_to_http_noack | -4.16KiB/CPU-s | -0.03 | 57.80% | 13.62MiB/CPU-s | 257.35KiB/CPU-s | 3.32KiB/CPU-s | 0.0 | 0.018454 | 13.61MiB/CPU-s | 308.38KiB/CPU-s | 3.98KiB/CPU-s | 0.0 | 0.022119 | False | False |
| syslog_log2metric_splunk_hec_metrics | -25.18KiB/CPU-s | -0.26 | 100.00% | 9.29MiB/CPU-s | 297.85KiB/CPU-s | 3.85KiB/CPU-s | 0.0 | 0.031291 | 9.27MiB/CPU-s | 257.8KiB/CPU-s | 3.33KiB/CPU-s | 0.0 | 0.027155 | False | False |
| syslog_splunk_hec_logs | -35.41KiB/CPU-s | -0.38 | 100.00% | 9.08MiB/CPU-s | 159.89KiB/CPU-s | 2.06KiB/CPU-s | 0.0 | 0.0172 | 9.04MiB/CPU-s | 220.21KiB/CPU-s | 2.84KiB/CPU-s | 0.0 | 0.023779 | False | False |
| splunk_hec_route_s3 | -48.85KiB/CPU-s | -0.4 | 100.00% | 11.8MiB/CPU-s | 522.45KiB/CPU-s | 6.74KiB/CPU-s | 0.0 | 0.043241 | 11.75MiB/CPU-s | 566.73KiB/CPU-s | 7.31KiB/CPU-s | 0.0 | 0.047097 | False | False |
| datadog_agent_remap_blackhole_acks | -190.53KiB/CPU-s | -0.58 | 100.00% | 32.02MiB/CPU-s | 471.62KiB/CPU-s | 6.09KiB/CPU-s | 0.0 | 0.014381 | 31.84MiB/CPU-s | 927.47KiB/CPU-s | 11.97KiB/CPU-s | 0.0 | 0.028445 | False | False |
| syslog_humio_logs | -67.46KiB/CPU-s | -0.72 | 100.00% | 9.2MiB/CPU-s | 133.66KiB/CPU-s | 1.73KiB/CPU-s | 0.0 | 0.014188 | 9.13MiB/CPU-s | 178.13KiB/CPU-s | 2.3KiB/CPU-s | 0.0 | 0.019045 | False | False |
| http_to_http_acks | -49.29KiB/CPU-s | -0.91 | 65.18% | 5.29MiB/CPU-s | 2.8MiB/CPU-s | 37.05KiB/CPU-s | 0.0 | 0.530031 | 5.24MiB/CPU-s | 2.82MiB/CPU-s | 37.24KiB/CPU-s | 0.0 | 0.537752 | True | False |
| otlp_grpc_to_blackhole | -17.31KiB/CPU-s | -1.61 | 100.00% | 1.05MiB/CPU-s | 40.67KiB/CPU-s | 537.66B/CPU-s | 0.0 | 0.037786 | 1.03MiB/CPU-s | 47.15KiB/CPU-s | 623.13B/CPU-s | 0.0 | 0.044516 | False | False |
| socket_to_socket_blackhole | -278.39KiB/CPU-s | -2.03 | 100.00% | 13.4MiB/CPU-s | 173.51KiB/CPU-s | 2.24KiB/CPU-s | 0.0 | 0.012643 | 13.13MiB/CPU-s | 69.48KiB/CPU-s | 918.8B/CPU-s | 0.0 | 0.005167 | False | False |
| http_text_to_http_json | -515.36KiB/CPU-s | -2.1 | 100.00% | 23.97MiB/CPU-s | 588.44KiB/CPU-s | 7.6KiB/CPU-s | 0.0 | 0.023968 | 23.47MiB/CPU-s | 800.01KiB/CPU-s | 10.32KiB/CPU-s | 0.0 | 0.033285 | False | False |
| otlp_http_to_blackhole | -38.55KiB/CPU-s | -2.43 | 100.00% | 1.55MiB/CPU-s | 107.16KiB/CPU-s | 1.38KiB/CPU-s | 0.0 | 0.067632 | 1.51MiB/CPU-s | 119.93KiB/CPU-s | 1.55KiB/CPU-s | 0.0 | 0.077579 | False | False |