feat(prometheus): add auth implementation to Prometheus Exporter

[zamazan4ik]

Resolves https://github.com/vectordotdev/vector/issues/11041

In this PR I introduce auth functionality to Prometheus exporter. It can be used for protecting prometheus metrics from unauthorized access.

Implementation details

Auth implementation

Current HTTP server for Prometheus Exporter is implemented on Hyper. Hyper is a low-level library for HTTP so it doesn't have built-in auth middleware. There are two ways: implement auth in Vector codebase from the scratch or use more advanced HTTP server library (like tower-http, axum, warp.). I have implemented more simple way with implementing auth just a part of already existing HTTP handler for hyper.

However, I suggest you later in all data source migrate to something more high-level abstraction. It allows us to reuse more existing functionality (like built-in auth middlewares).

Testing

I have tested locally the following scenarios:

• Default behaviour without auth - works as before
• Specify auth in config file and try to get access without credentials - Unauthorized result
• Access with Basic creds - Ok
• Access with Bearer token - Ok

[netlify[bot]]

✅ Deploy Preview for vector-project canceled.

Name	Link
🔨 Latest commit	b498b114260e1991232be340753a1e3a9076240e
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/632cf088cbccea0008b1f5c3

[zamazan4ik]

@jszwedko would be great, if you review it :)

[github-actions[bot]]

Soak Test Results

Baseline: d498040a770ae2bb5c9d25efce62acadcb17ee57 Comparison: 927f1bb8bcca19acb8010fd3d1eba491aa360fd6 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
syslog_splunk_hec_logs	204.68KiB	1.25	100.00%	16.01MiB	767.13KiB	15.6KiB	0	0.0467913	16.21MiB	633.68KiB	12.92KiB	0	0.0381745	False	False
syslog_log2metric_splunk_hec_metrics	210.57KiB	1.2	100.00%	17.11MiB	672.92KiB	13.7KiB	0	0.03839	17.32MiB	802.79KiB	16.34KiB	0	0.0452552	False	False
syslog_loki	155.45KiB	1.07	100.00%	14.15MiB	580.78KiB	11.88KiB	0	0.040086	14.3MiB	764.35KiB	15.54KiB	0	0.052196	False	False
http_pipelines_blackhole_acks	8.79KiB	0.71	99.97%	1.22MiB	103.37KiB	2.1KiB	0	0.0829717	1.23MiB	60.59KiB	1.24KiB	0	0.0482932	False	False
syslog_humio_logs	99.84KiB	0.6	100.00%	16.22MiB	137.33KiB	2.8KiB	0	0.00826426	16.32MiB	235.53KiB	4.82KiB	0	0.014089	False	False
splunk_hec_indexer_ack_blackhole	8.14KiB	0.03	25.48%	23.75MiB	886.14KiB	18.02KiB	0	0.036429	23.76MiB	855.85KiB	17.42KiB	0	0.0351722	False	False
splunk_hec_to_splunk_hec_logs_noack	6.72KiB	0.03	48.14%	23.83MiB	381.07KiB	7.79KiB	0	0.015612	23.84MiB	338.01KiB	6.9KiB	0	0.0138442	False	False
enterprise_http_to_http	-1.38KiB	-0.01	15.08%	23.85MiB	251.5KiB	5.13KiB	0	0.0102973	23.85MiB	251.72KiB	5.15KiB	0	0.0103068	False	False
splunk_hec_to_splunk_hec_logs_acks	-1.29KiB	-0.01	4.48%	23.77MiB	794.91KiB	16.18KiB	0	0.0326546	23.77MiB	799.59KiB	16.27KiB	0	0.0328484	False	False
splunk_hec_route_s3	-8.51KiB	-0.05	9.96%	17.7MiB	2.33MiB	48.55KiB	0	0.131599	17.69MiB	2.28MiB	47.6KiB	0	0.128664	False	False
file_to_blackhole	-75.98KiB	-0.08	55.71%	95.34MiB	3.1MiB	64.31KiB	0	0.0325318	95.27MiB	3.62MiB	75.29KiB	0	0.0380304	False	False
http_pipelines_no_grok_blackhole	-14.95KiB	-0.14	52.75%	10.48MiB	74.25KiB	1.52KiB	0	0.00692023	10.46MiB	1020.44KiB	20.76KiB	0	0.0952377	False	False
http_to_http_json	-46.66KiB	-0.19	99.95%	23.85MiB	329.85KiB	6.73KiB	0	0.0135046	23.8MiB	569.91KiB	11.63KiB	0	0.0233778	False	False
fluent_elasticsearch	-188.03KiB	-0.23	100.00%	79.47MiB	52.88KiB	1.07KiB	0	0.000649671	79.29MiB	1.71MiB	35.11KiB	0	0.0215246	False	False
http_to_http_noack	-70.53KiB	-0.29	99.88%	23.84MiB	402.21KiB	8.22KiB	0	0.0164741	23.77MiB	985.48KiB	20.08KiB	0	0.0404808	False	False
datadog_agent_remap_blackhole	-261.76KiB	-0.42	98.22%	61.12MiB	4.27MiB	88.94KiB	0	0.0698714	60.87MiB	3.14MiB	65.46KiB	0	0.0515492	False	False
syslog_regex_logs2metric_ddmetrics	-58.25KiB	-0.45	99.96%	12.76MiB	579.7KiB	11.81KiB	0	0.0443426	12.71MiB	564.24KiB	11.5KiB	0	0.0433531	False	False
http_to_http_acks	-92.09KiB	-0.52	30.46%	17.37MiB	7.87MiB	164.53KiB	0	0.453012	17.28MiB	8.06MiB	168.09KiB	0	0.466381	True	True
http_text_to_http_json	-259.23KiB	-0.67	100.00%	37.96MiB	1.05MiB	21.89KiB	0	0.0275739	37.71MiB	1.07MiB	22.4KiB	0	0.0284055	False	False
datadog_agent_remap_datadog_logs_acks	-592.23KiB	-0.94	100.00%	61.36MiB	2.94MiB	61.49KiB	0	0.0479191	60.78MiB	4.3MiB	89.43KiB	0	0.0706773	False	False
datadog_agent_remap_blackhole_acks	-627.99KiB	-1.06	100.00%	57.96MiB	3.95MiB	82.28KiB	0	0.0681439	57.35MiB	3.53MiB	73.74KiB	0	0.0615721	False	False
http_pipelines_blackhole	-35.14KiB	-2.19	100.00%	1.56MiB	178.82KiB	3.65KiB	0	0.111575	1.53MiB	217.04KiB	4.43KiB	0	0.138461	False	False
socket_to_socket_blackhole	-681.59KiB	-2.8	100.00%	23.75MiB	297.75KiB	6.08KiB	0	0.0122385	23.09MiB	453.88KiB	9.27KiB	0	0.0191937	False	False
syslog_log2metric_humio_metrics	-364.7KiB	-2.94	100.00%	12.11MiB	577.17KiB	11.78KiB	0	0.0465474	11.75MiB	948.47KiB	19.3KiB	0	0.0788103	False	False
datadog_agent_remap_datadog_logs	-2.65MiB	-4.47	100.00%	59.41MiB	3.26MiB	68.39KiB	0	0.0548992	56.76MiB	6.11MiB	127.31KiB	0	0.107666	False	False

[github-actions[bot]]

Soak Test Results

Baseline: d498040a770ae2bb5c9d25efce62acadcb17ee57 Comparison: 3ea743080d495bc4cb76dc96c1a19ce6273fa0e9 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
http_pipelines_blackhole_acks	9.02KiB	0.73	99.71%	1.21MiB	119.85KiB	2.44KiB	0	0.0968383	1.22MiB	87.85KiB	1.79KiB	0	0.0704658	False	False
syslog_splunk_hec_logs	82.98KiB	0.5	99.99%	16.25MiB	747.53KiB	15.21KiB	0	0.0449183	16.33MiB	671.46KiB	13.68KiB	0	0.0401472	False	False
syslog_log2metric_humio_metrics	29.92KiB	0.23	99.43%	12.81MiB	203.94KiB	4.17KiB	0	0.0155387	12.84MiB	489.54KiB	9.97KiB	0	0.0372147	False	False
http_pipelines_blackhole	2.88KiB	0.18	77.41%	1.6MiB	26.2KiB	547.99B	0	0.0159904	1.6MiB	113.53KiB	2.31KiB	0	0.0691656	False	False
splunk_hec_to_splunk_hec_logs_noack	26.11KiB	0.11	95.20%	23.81MiB	552.34KiB	11.26KiB	0	0.0226492	23.84MiB	336.93KiB	6.88KiB	0	0.0138014	False	False
splunk_hec_to_splunk_hec_logs_acks	2.56KiB	0.01	8.55%	23.76MiB	837.33KiB	17.04KiB	0	0.0344123	23.76MiB	822.23KiB	16.73KiB	0	0.0337884	False	False
enterprise_http_to_http	719.85B	0	7.61%	23.85MiB	253.35KiB	5.17KiB	0	0.0103737	23.85MiB	256.24KiB	5.24KiB	0	0.0104913	False	False
splunk_hec_indexer_ack_blackhole	-3.36KiB	-0.01	10.51%	23.75MiB	881.43KiB	17.93KiB	0	0.0362368	23.75MiB	885.37KiB	18.01KiB	0	0.0364035	False	False
file_to_blackhole	-33.56KiB	-0.03	28.83%	95.34MiB	2.98MiB	61.85KiB	0	0.0312861	95.31MiB	3.19MiB	66.48KiB	0	0.0335147	False	False
http_to_http_json	-42.55KiB	-0.17	99.88%	23.85MiB	331.4KiB	6.77KiB	0	0.0135681	23.81MiB	553.0KiB	11.29KiB	0	0.0226804	False	False
fluent_elasticsearch	-188.15KiB	-0.23	100.00%	79.47MiB	53.89KiB	1.09KiB	0	0.000662054	79.29MiB	1.72MiB	35.36KiB	0	0.0216513	False	False
syslog_loki	-45.91KiB	-0.31	99.42%	14.44MiB	367.15KiB	7.51KiB	0	0.0248197	14.4MiB	730.65KiB	14.85KiB	0	0.0495469	False	False
syslog_humio_logs	-57.31KiB	-0.34	100.00%	16.68MiB	264.71KiB	5.4KiB	0	0.0154971	16.62MiB	164.66KiB	3.37KiB	0	0.00967212	False	False
http_to_http_noack	-88.22KiB	-0.36	99.89%	23.82MiB	610.82KiB	12.48KiB	0	0.0250368	23.73MiB	1.15MiB	24.03KiB	0	0.0485552	False	False
splunk_hec_route_s3	-108.23KiB	-0.56	90.68%	18.9MiB	2.21MiB	45.94KiB	0	0.116646	18.8MiB	2.16MiB	45.21KiB	0	0.114874	False	False
syslog_regex_logs2metric_ddmetrics	-84.9KiB	-0.66	100.00%	12.66MiB	612.53KiB	12.48KiB	0	0.0472526	12.57MiB	555.72KiB	11.33KiB	0	0.0431531	False	False
datadog_agent_remap_datadog_logs_acks	-423.88KiB	-0.67	99.98%	61.59MiB	3.25MiB	67.91KiB	0	0.0527642	61.17MiB	4.32MiB	90.0KiB	0	0.0706648	False	False
http_text_to_http_json	-298.48KiB	-0.75	100.00%	38.63MiB	817.07KiB	16.68KiB	0	0.0206492	38.34MiB	907.96KiB	18.53KiB	0	0.0231207	False	False
syslog_log2metric_splunk_hec_metrics	-195.67KiB	-1.09	100.00%	17.48MiB	581.9KiB	11.86KiB	0	0.0324993	17.29MiB	746.82KiB	15.22KiB	0	0.0421712	False	False
datadog_agent_remap_blackhole	-715.7KiB	-1.2	100.00%	58.36MiB	4.08MiB	84.85KiB	0	0.0698147	57.66MiB	2.81MiB	58.59KiB	0	0.0486898	False	False
http_pipelines_no_grok_blackhole	-136.75KiB	-1.21	100.00%	10.99MiB	46.04KiB	963.03B	0	0.00408998	10.86MiB	1.11MiB	23.05KiB	0	0.101974	False	False
http_to_http_acks	-298.7KiB	-1.65	79.97%	17.7MiB	7.89MiB	164.92KiB	0	0.445588	17.41MiB	7.9MiB	164.87KiB	0	0.453944	True	True
datadog_agent_remap_datadog_logs	-1.04MiB	-1.7	100.00%	61.24MiB	811.92KiB	16.63KiB	0	0.0129439	60.2MiB	4.18MiB	87.12KiB	0	0.0694986	False	False
datadog_agent_remap_blackhole_acks	-2.51MiB	-4.12	100.00%	60.98MiB	4.23MiB	88.01KiB	0	0.0692792	58.47MiB	3.3MiB	68.88KiB	0	0.0564009	False	False
socket_to_socket_blackhole	-981.58KiB	-4.21	100.00%	22.79MiB	1.5MiB	31.36KiB	0	0.0658181	21.83MiB	1.7MiB	35.58KiB	0	0.0779531	False	False

[zamazan4ik]

I left a couple of small notes, but, also, do you think we could add some tests for this behavior? You should be able to copy and modify the ones already in src/sinks/prometheus/exporter.rs.

Yes, I definitely want add some tests. I just waited for the feedback from your side. Right now seems that in general you agree with this implementation, so I can start work on tests.

[jszwedko]

I left a couple of small notes, but, also, do you think we could add some tests for this behavior? You should be able to copy and modify the ones already in src/sinks/prometheus/exporter.rs.

Yes, I definitely want add some tests. I just waited for the feedback from your side. Right now seems that in general you agree with this implementation, so I can start work on tests.

👍 I agree with your assessment in the PR description. It would be nice to rely on middleware for this in the future, but what you have makes sense for this sink which is not currently using a HTTP framework like axum or warp.

[zamazan4ik]

@jszwedko by the way, CI says that for user/password I need to use special type instead of usual String. Just want to clarify - should I fix it too or it is just a false positive?

[github-actions[bot]]

Soak Test Results

Baseline: 50f37f5b1a5910d51d698773d23026270b07b7e8 Comparison: a7067a1b8b296c30e8cfe110c429278dc711cf87 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
syslog_log2metric_splunk_hec_metrics	684.3KiB	3.87	100.00%	17.29MiB	947.09KiB	19.3KiB	0	0.053492	17.95MiB	767.6KiB	15.63KiB	0	0.0417408	False	False
syslog_splunk_hec_logs	574.92KiB	3.52	100.00%	15.94MiB	825.03KiB	16.78KiB	0	0.0505468	16.5MiB	669.4KiB	13.64KiB	0	0.0396156	False	False
syslog_humio_logs	506.45KiB	3.08	100.00%	16.03MiB	740.8KiB	15.12KiB	0	0.0451093	16.53MiB	569.55KiB	11.66KiB	0	0.0336434	False	False
syslog_regex_logs2metric_ddmetrics	274.07KiB	2.17	100.00%	12.33MiB	555.18KiB	11.31KiB	0	0.0439455	12.6MiB	599.01KiB	12.2KiB	0	0.0464081	False	False
datadog_agent_remap_blackhole	1.06MiB	1.85	100.00%	57.27MiB	4.8MiB	100.06KiB	0	0.0838156	58.33MiB	4.16MiB	86.7KiB	0	0.0712761	False	False
socket_to_socket_blackhole	266.22KiB	1.17	100.00%	22.3MiB	975.33KiB	19.91KiB	0	0.0426947	22.56MiB	1016.95KiB	20.76KiB	0	0.0440038	False	False
http_pipelines_blackhole_acks	12.39KiB	0.99	100.00%	1.22MiB	99.17KiB	2.02KiB	0	0.0794603	1.23MiB	66.73KiB	1.36KiB	0	0.0529454	False	False
splunk_hec_indexer_ack_blackhole	16.66KiB	0.07	47.54%	23.74MiB	939.66KiB	19.11KiB	0	0.0386423	23.76MiB	879.07KiB	17.89KiB	0	0.0361261	False	False
splunk_hec_to_splunk_hec_logs_noack	10.49KiB	0.04	65.74%	23.83MiB	425.04KiB	8.68KiB	0	0.0174159	23.84MiB	335.0KiB	6.84KiB	0	0.0137209	False	False
enterprise_http_to_http	1.48KiB	0.01	16.04%	23.84MiB	252.51KiB	5.15KiB	0	0.0103399	23.85MiB	252.05KiB	5.16KiB	0	0.0103201	False	False
splunk_hec_to_splunk_hec_logs_acks	-18.76KiB	-0.08	55.04%	23.77MiB	830.57KiB	16.9KiB	0	0.0341209	23.75MiB	892.66KiB	18.15KiB	0	0.0366999	False	False
datadog_agent_remap_blackhole_acks	-65.03KiB	-0.11	48.60%	58.2MiB	3.98MiB	82.87KiB	0	0.0683796	58.13MiB	2.65MiB	55.34KiB	0	0.0455499	False	False
http_to_http_json	-35.47KiB	-0.15	99.34%	23.84MiB	349.34KiB	7.13KiB	0	0.0143058	23.81MiB	535.21KiB	10.92KiB	0	0.021949	False	False
file_to_blackhole	-162.04KiB	-0.17	77.24%	95.31MiB	4.01MiB	83.01KiB	0	0.0420468	95.15MiB	5.08MiB	105.54KiB	0	0.0534068	False	False
http_pipelines_no_grok_blackhole	-21.32KiB	-0.2	64.44%	10.33MiB	403.41KiB	8.23KiB	0	0.0381142	10.31MiB	1.04MiB	21.56KiB	0	0.100351	False	False
fluent_elasticsearch	-173.23KiB	-0.21	99.99%	79.47MiB	54.13KiB	1.1KiB	0	0.000664978	79.3MiB	2.1MiB	43.14KiB	0	0.0264413	False	False
http_pipelines_blackhole	-3.53KiB	-0.21	82.27%	1.66MiB	20.38KiB	426.54B	0	0.0120079	1.65MiB	126.89KiB	2.58KiB	0	0.0749306	False	False
http_to_http_noack	-57.9KiB	-0.24	99.46%	23.84MiB	416.9KiB	8.53KiB	0	0.0170773	23.78MiB	931.8KiB	18.99KiB	0	0.0382596	False	False
splunk_hec_route_s3	-144.75KiB	-0.76	97.09%	18.7MiB	2.31MiB	48.01KiB	0	0.123286	18.56MiB	2.18MiB	45.73KiB	0	0.117709	False	False
datadog_agent_remap_datadog_logs_acks	-515.01KiB	-0.81	100.00%	62.02MiB	3.42MiB	71.34KiB	0	0.055065	61.52MiB	4.38MiB	91.2KiB	0	0.071203	False	False
http_to_http_acks	-155.76KiB	-0.88	48.70%	17.2MiB	8.12MiB	169.71KiB	0	0.471921	17.05MiB	8.01MiB	166.97KiB	0	0.469707	True	True
syslog_loki	-159.9KiB	-1.08	100.00%	14.39MiB	554.6KiB	11.34KiB	0	0.0376169	14.24MiB	791.91KiB	16.1KiB	0	0.0543018	False	False
datadog_agent_remap_datadog_logs	-1.33MiB	-2.19	100.00%	60.66MiB	2.21MiB	46.42KiB	0	0.0364745	59.34MiB	4.79MiB	99.63KiB	0	0.0806296	False	False
syslog_log2metric_humio_metrics	-321.89KiB	-2.51	100.00%	12.54MiB	626.66KiB	12.79KiB	0	0.0487973	12.22MiB	730.27KiB	14.88KiB	0	0.058328	False	False
http_text_to_http_json	-1.13MiB	-2.92	100.00%	38.68MiB	1.12MiB	23.46KiB	0	0.0290106	37.55MiB	1.11MiB	23.27KiB	0	0.0296475	False	False

[github-actions[bot]]

Soak Test Results

Baseline: 7b26f65c6bde9fe824ea15948fa0147cf026db5e Comparison: 9dc93b67f9a2bcffb9480431f53639ee62376b0d Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
splunk_hec_route_s3	341.67KiB	1.83	100.00%	18.27MiB	2.39MiB	49.71KiB	0	0.130665	18.6MiB	2.28MiB	47.55KiB	0	0.122322	False	False
syslog_splunk_hec_logs	267.98KiB	1.63	100.00%	16.05MiB	812.37KiB	16.54KiB	0	0.0494125	16.31MiB	747.42KiB	15.23KiB	0	0.0447324	False	False
syslog_log2metric_splunk_hec_metrics	255.6KiB	1.45	100.00%	17.26MiB	833.52KiB	16.98KiB	0	0.0471592	17.51MiB	934.77KiB	19.03KiB	0	0.0521333	False	False
syslog_humio_logs	225.21KiB	1.35	100.00%	16.32MiB	371.53KiB	7.58KiB	0	0.0222264	16.54MiB	310.7KiB	6.36KiB	0	0.0183399	False	False
syslog_loki	198.76KiB	1.35	100.00%	14.43MiB	511.36KiB	10.47KiB	0	0.0346011	14.62MiB	780.31KiB	15.86KiB	0	0.0520995	False	False
syslog_regex_logs2metric_ddmetrics	169.17KiB	1.32	100.00%	12.53MiB	681.35KiB	13.86KiB	0	0.0530732	12.7MiB	466.71KiB	9.52KiB	0	0.0358809	False	False
http_pipelines_blackhole_acks	13.88KiB	1.17	100.00%	1.16MiB	121.92KiB	2.48KiB	0	0.103003	1.17MiB	107.51KiB	2.19KiB	0	0.0897745	False	False
syslog_log2metric_humio_metrics	133.28KiB	1.06	100.00%	12.33MiB	791.42KiB	16.15KiB	0	0.0626436	12.47MiB	877.04KiB	17.85KiB	0	0.0686964	False	False
splunk_hec_to_splunk_hec_logs_acks	23.9KiB	0.1	69.60%	23.75MiB	855.49KiB	17.4KiB	0	0.0351665	23.78MiB	757.4KiB	15.42KiB	0	0.0311035	False	False
splunk_hec_to_splunk_hec_logs_noack	11.38KiB	0.05	69.71%	23.83MiB	426.28KiB	8.7KiB	0	0.0174681	23.84MiB	333.56KiB	6.81KiB	0	0.0136625	False	False
splunk_hec_indexer_ack_blackhole	1.49KiB	0.01	4.24%	23.73MiB	980.28KiB	19.93KiB	0	0.0403269	23.74MiB	972.55KiB	19.77KiB	0	0.0400061	False	False
enterprise_http_to_http	-632.94B	-0	6.63%	23.85MiB	253.34KiB	5.17KiB	0	0.0103728	23.85MiB	260.98KiB	5.34KiB	0	0.0106857	False	False
file_to_blackhole	-67.79KiB	-0.07	49.74%	95.34MiB	3.32MiB	68.85KiB	0	0.0348346	95.27MiB	3.56MiB	74.04KiB	0	0.0373582	False	False
http_to_http_json	-30.2KiB	-0.12	98.69%	23.85MiB	333.54KiB	6.81KiB	0	0.0136566	23.82MiB	493.3KiB	10.08KiB	0	0.0202226	False	False
datadog_agent_remap_blackhole	-78.19KiB	-0.13	53.54%	60.5MiB	4.27MiB	88.92KiB	0	0.0705669	60.42MiB	2.84MiB	59.36KiB	0	0.047071	False	False
http_to_http_acks	-42.47KiB	-0.24	14.52%	17.25MiB	7.88MiB	164.71KiB	0	0.456723	17.21MiB	7.82MiB	163.36KiB	0	0.454556	True	True
fluent_elasticsearch	-192.11KiB	-0.24	100.00%	79.47MiB	54.19KiB	1.1KiB	0	0.000665705	79.29MiB	1.71MiB	35.22KiB	0	0.0215749	False	False
http_to_http_noack	-76.11KiB	-0.31	99.83%	23.83MiB	511.31KiB	10.45KiB	0	0.0209516	23.75MiB	1.05MiB	21.83KiB	0	0.0440672	False	False
datadog_agent_remap_blackhole_acks	-473.53KiB	-0.75	100.00%	61.7MiB	3.94MiB	82.13KiB	0	0.0638734	61.24MiB	2.98MiB	62.37KiB	0	0.0486987	False	False
http_pipelines_no_grok_blackhole	-85.25KiB	-0.78	99.91%	10.66MiB	507.01KiB	10.35KiB	0	0.046418	10.58MiB	1.12MiB	23.36KiB	0	0.105986	False	False
socket_to_socket_blackhole	-425.3KiB	-1.84	100.00%	22.58MiB	1.3MiB	27.25KiB	0	0.0577108	22.17MiB	1.12MiB	23.49KiB	0	0.0506595	False	False
datadog_agent_remap_datadog_logs_acks	-1.5MiB	-2.37	100.00%	63.17MiB	2.9MiB	60.59KiB	0	0.0458374	61.67MiB	4.38MiB	91.16KiB	0	0.0709946	False	False
http_pipelines_blackhole	-49.18KiB	-2.92	100.00%	1.65MiB	98.48KiB	2.01KiB	0	0.0583823	1.6MiB	151.05KiB	3.08KiB	0	0.0922439	False	False
http_text_to_http_json	-1.17MiB	-3.03	100.00%	38.58MiB	1.63MiB	34.14KiB	0	0.0423224	37.41MiB	1.36MiB	28.35KiB	0	0.0362322	False	False
datadog_agent_remap_datadog_logs	-2.14MiB	-3.5	100.00%	61.3MiB	2.0MiB	41.92KiB	0	0.0326272	59.15MiB	4.13MiB	86.04KiB	0	0.0698194	False	False

[zamazan4ik]

I have synced my branch with the upstream, added SensitiveString using and added const HeaderValue usage.

[github-actions[bot]]

Soak Test Results

Baseline: ab3ca33efecdc91eff89fb9c0bb3a1ab81cd2762 Comparison: 46c0b5e190f908b3fb27424fe57b2e0f4e7ec504 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
http_to_http_acks	157.02KiB	0.89	51.05%	17.28MiB	7.75MiB	162.15KiB	0	0.448533	17.43MiB	7.63MiB	159.15KiB	0	0.437366	True	True
syslog_regex_logs2metric_ddmetrics	47.11KiB	0.38	99.95%	11.95MiB	479.35KiB	9.76KiB	0	0.0391573	12.0MiB	452.56KiB	9.22KiB	0	0.0368264	False	False
syslog_humio_logs	57.83KiB	0.35	100.00%	16.11MiB	204.51KiB	4.18KiB	0	0.0123931	16.17MiB	206.33KiB	4.22KiB	0	0.0124594	False	False
http_pipelines_blackhole_acks	1.94KiB	0.15	47.00%	1.23MiB	114.67KiB	2.33KiB	0	0.0912648	1.23MiB	98.8KiB	2.01KiB	0	0.0785142	False	False
http_text_to_http_json	48.02KiB	0.12	84.00%	39.24MiB	1.1MiB	22.93KiB	0	0.027948	39.28MiB	1.21MiB	25.34KiB	0	0.0308534	False	False
splunk_hec_to_splunk_hec_logs_noack	26.12KiB	0.11	95.68%	23.81MiB	543.2KiB	11.08KiB	0	0.0222716	23.84MiB	325.18KiB	6.64KiB	0	0.0133184	False	False
splunk_hec_indexer_ack_blackhole	17.2KiB	0.07	51.06%	23.75MiB	895.32KiB	18.22KiB	0	0.0368024	23.77MiB	832.78KiB	16.95KiB	0	0.0342074	False	False
enterprise_http_to_http	-2.38KiB	-0.01	25.57%	23.85MiB	249.81KiB	5.1KiB	0	0.0102278	23.84MiB	255.76KiB	5.23KiB	0	0.0104726	False	False
splunk_hec_to_splunk_hec_logs_acks	-7.72KiB	-0.03	23.59%	23.75MiB	868.4KiB	17.66KiB	0	0.0356976	23.74MiB	919.0KiB	18.69KiB	0	0.0377896	False	False
splunk_hec_route_s3	-26.64KiB	-0.14	30.79%	18.56MiB	2.32MiB	48.21KiB	0	0.124713	18.53MiB	2.24MiB	46.94KiB	0	0.121092	False	False
datadog_agent_remap_blackhole_acks	-89.61KiB	-0.14	61.80%	62.04MiB	4.15MiB	86.36KiB	0	0.0668161	61.95MiB	2.64MiB	55.18KiB	0	0.042553	False	False
file_to_blackhole	-192.53KiB	-0.2	82.68%	95.32MiB	4.13MiB	85.45KiB	0	0.04328	95.13MiB	5.43MiB	112.6KiB	0	0.0570173	False	False
http_to_http_json	-54.47KiB	-0.22	99.99%	23.85MiB	329.28KiB	6.72KiB	0	0.0134813	23.79MiB	603.28KiB	12.3KiB	0	0.0247545	False	False
socket_to_socket_blackhole	-53.45KiB	-0.22	99.57%	23.23MiB	600.22KiB	12.25KiB	0	0.0252303	23.17MiB	693.26KiB	14.15KiB	0	0.0292073	False	False
datadog_agent_remap_datadog_logs_acks	-190.47KiB	-0.31	93.59%	60.02MiB	2.59MiB	54.22KiB	0	0.0431601	59.83MiB	4.2MiB	87.39KiB	0	0.0701513	False	False
http_to_http_noack	-114.53KiB	-0.47	100.00%	23.85MiB	255.55KiB	5.22KiB	0	0.0104633	23.73MiB	1.15MiB	24.02KiB	0	0.0485348	False	False
syslog_splunk_hec_logs	-81.02KiB	-0.49	99.99%	16.13MiB	791.83KiB	16.1KiB	0	0.0479298	16.05MiB	661.7KiB	13.5KiB	0	0.0402499	False	False
datadog_agent_remap_datadog_logs	-390.85KiB	-0.62	100.00%	61.93MiB	1.26MiB	26.42KiB	0	0.0203531	61.55MiB	4.33MiB	90.09KiB	0	0.0702771	False	False
fluent_elasticsearch	-536.66KiB	-0.66	100.00%	79.47MiB	52.67KiB	1.06KiB	0	0.000647103	78.95MiB	5.24MiB	107.6KiB	0	0.0663603	False	False
syslog_log2metric_humio_metrics	-108.96KiB	-0.84	100.00%	12.66MiB	319.54KiB	6.52KiB	0	0.0246477	12.55MiB	511.0KiB	10.4KiB	0	0.0397509	False	False
syslog_loki	-157.78KiB	-1.09	100.00%	14.15MiB	673.63KiB	13.8KiB	0	0.0464889	13.99MiB	1.01MiB	20.96KiB	0	0.0719365	False	False
http_pipelines_no_grok_blackhole	-142.86KiB	-1.31	100.00%	10.62MiB	162.86KiB	3.33KiB	0	0.0149664	10.49MiB	1.05MiB	21.9KiB	0	0.100293	False	False
http_pipelines_blackhole	-26.61KiB	-1.55	100.00%	1.67MiB	56.35KiB	1.15KiB	0	0.0329107	1.65MiB	122.36KiB	2.49KiB	0	0.0725932	False	False
syslog_log2metric_splunk_hec_metrics	-553.07KiB	-3.14	100.00%	17.22MiB	1005.01KiB	20.49KiB	0	0.0569691	16.68MiB	1.48MiB	30.83KiB	0	0.0885956	False	False
datadog_agent_remap_blackhole	-2.29MiB	-3.75	100.00%	60.91MiB	4.59MiB	95.67KiB	0	0.0754113	58.62MiB	4.92MiB	102.57KiB	0	0.0838777	False	False

[zamazan4ik]

I got one more warning regarding using lazy_static instead of const. @jszwedko according to use Vector guidelines, should I use here something like once_cell::sync::Lazy or just left it as a const will be fine?

[zamazan4ik]

I guess explicit use base64; should resolve the issue :)

[zamazan4ik]

Argh. @jszwedko could you please help me resolve the issue with base64 error? I canot understand what I am doing wrong. I have already checked other base64 usages in Vector codebase and didn't find any difference with my case. I can successfully build Vector binary locally (that is how I test it locally). But why it fails on CI... I do not know. As you see, explicit use base64; didn't help. Could you locally checkout the branch and just try to build it?

[jszwedko]

Argh. @jszwedko could you please help me resolve the issue with base64 error? I canot understand what I am doing wrong. I have already checked other base64 usages in Vector codebase and didn't find any difference with my case. I can successfully build Vector binary locally (that is how I test it locally). But why it fails on CI... I do not know. As you see, explicit use base64; didn't help. Could you locally checkout the branch and just try to build it?

Doh, I'm sorry, I led you in the wrong direction. The issue is that the base64 crate is marked as optional in Cargo.toml so must be specifically included for components that need it. I pushed db023d2ca to remedy this. This explains why you didn't see the issue locally though since it would only show up when building subsets of Vector that don't result in base64 being included.

[zamazan4ik]

Ah, that one was kinda tricky :) Thank you for the help!

[zamazan4ik]

@jszwedko I have fixed your commit: you mistakenly added dep:base64 to sources-prometheus, not sinks-prometheus. Now it builds fine but soak tests are failed for some reason. I am completely sure that is not caused by my changes. So maybe fix them somehow on your side or just suppress them and merge without soak tests.

[github-actions[bot]]

Soak Test Results

Baseline: 0d94ae2b80c80e57deea9af62388766577f4b7b1 Comparison: db023d2ca75bc404502d31007bd7309c510074cb Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
http_text_to_http_json	1.15MiB	3.06	100.00%	37.59MiB	1.02MiB	21.25KiB	0	0.0270455	38.74MiB	1.36MiB	28.48KiB	0	0.0351638	False	False
syslog_log2metric_humio_metrics	220.57KiB	1.76	100.00%	12.23MiB	446.94KiB	9.13KiB	0	0.0356881	12.44MiB	599.87KiB	12.21KiB	0	0.0470706	False	False
datadog_agent_remap_datadog_logs_acks	796.83KiB	1.26	100.00%	61.57MiB	3.47MiB	72.52KiB	0	0.0563876	62.35MiB	4.48MiB	93.32KiB	0	0.0718889	False	False
http_pipelines_blackhole_acks	11.74KiB	0.94	100.00%	1.23MiB	120.79KiB	2.46KiB	0	0.0962411	1.24MiB	54.69KiB	1.12KiB	0	0.0431724	False	False
syslog_splunk_hec_logs	63.48KiB	0.39	99.89%	16.06MiB	760.5KiB	15.47KiB	0	0.0462288	16.12MiB	577.71KiB	11.78KiB	0	0.0349824	False	False
syslog_regex_logs2metric_ddmetrics	44.36KiB	0.35	99.39%	12.21MiB	572.24KiB	11.65KiB	0	0.0457564	12.25MiB	550.83KiB	11.23KiB	0	0.0438888	False	False
datadog_agent_remap_datadog_logs	211.18KiB	0.34	97.30%	61.42MiB	1.03MiB	21.68KiB	0	0.016828	61.63MiB	4.47MiB	92.97KiB	0	0.0724439	False	False
splunk_hec_to_splunk_hec_logs_noack	38.98KiB	0.16	99.43%	23.8MiB	604.28KiB	12.33KiB	0	0.0247878	23.84MiB	334.67KiB	6.83KiB	0	0.0137062	False	False
enterprise_http_to_http	3.4KiB	0.01	35.69%	23.84MiB	253.84KiB	5.18KiB	0	0.010395	23.85MiB	253.84KiB	5.19KiB	0	0.0103937	False	False
socket_to_socket_blackhole	-789.38B	-0	1.97%	22.63MiB	1.0MiB	20.93KiB	0	0.0442299	22.63MiB	1.11MiB	23.22KiB	0	0.0490481	False	False
file_to_blackhole	-52.37KiB	-0.05	36.88%	95.34MiB	3.39MiB	70.18KiB	0	0.0355014	95.29MiB	4.01MiB	83.5KiB	0	0.0421245	False	False
splunk_hec_to_splunk_hec_logs_acks	-21.08KiB	-0.09	58.69%	23.78MiB	890.3KiB	18.12KiB	0	0.0365563	23.76MiB	900.12KiB	18.31KiB	0	0.0369917	False	False
splunk_hec_indexer_ack_blackhole	-26.14KiB	-0.11	69.54%	23.77MiB	828.47KiB	16.87KiB	0	0.0340296	23.74MiB	937.63KiB	19.07KiB	0	0.0385551	False	False
http_to_http_json	-32.43KiB	-0.13	98.95%	23.84MiB	348.28KiB	7.11KiB	0	0.0142619	23.81MiB	513.47KiB	10.49KiB	0	0.0210547	False	False
syslog_log2metric_splunk_hec_metrics	-31.97KiB	-0.18	90.47%	17.58MiB	590.5KiB	12.04KiB	0	0.0327907	17.55MiB	732.22KiB	14.91KiB	0	0.0407332	False	False
http_pipelines_blackhole	-4.58KiB	-0.27	99.02%	1.68MiB	16.73KiB	350.37B	0	0.00971081	1.68MiB	85.23KiB	1.74KiB	0	0.0496095	False	False
http_to_http_acks	-49.77KiB	-0.28	16.85%	17.17MiB	7.82MiB	163.58KiB	0	0.455576	17.12MiB	8.02MiB	167.29KiB	0	0.468203	True	True
fluent_elasticsearch	-241.88KiB	-0.3	100.00%	79.47MiB	51.88KiB	1.05KiB	0	0.000637324	79.24MiB	2.5MiB	51.49KiB	0	0.0316048	False	False
datadog_agent_remap_blackhole_acks	-231.25KiB	-0.37	94.59%	61.4MiB	4.67MiB	97.23KiB	0	0.0760639	61.17MiB	3.37MiB	70.37KiB	0	0.0549982	False	False
http_to_http_noack	-95.55KiB	-0.39	99.99%	23.84MiB	404.65KiB	8.27KiB	0	0.0165745	23.74MiB	1.1MiB	22.93KiB	0	0.0463036	False	False
syslog_humio_logs	-68.48KiB	-0.42	100.00%	16.07MiB	577.45KiB	11.79KiB	0	0.0350782	16.01MiB	556.66KiB	11.4KiB	0	0.0339564	False	False
splunk_hec_route_s3	-151.29KiB	-0.79	97.55%	18.6MiB	2.35MiB	48.84KiB	0	0.126154	18.45MiB	2.21MiB	46.21KiB	0	0.119782	False	False
http_pipelines_no_grok_blackhole	-194.87KiB	-1.75	100.00%	10.88MiB	40.41KiB	844.93B	0	0.00362743	10.69MiB	1.03MiB	21.5KiB	0	0.0966169	False	False
datadog_agent_remap_blackhole	-1.29MiB	-2.18	100.00%	59.22MiB	4.83MiB	100.53KiB	0	0.0814703	57.93MiB	4.6MiB	96.02KiB	0	0.0794651	False	False
syslog_loki	-373.5KiB	-2.53	100.00%	14.44MiB	587.07KiB	12.02KiB	0	0.0396947	14.08MiB	941.1KiB	19.13KiB	0	0.0652816	False	False

[jszwedko]

Doh, thanks for fixing that @zamazan4ik . I think the soak test failure was due to one of the CI runners going away. I restarted them.

[github-actions[bot]]

Soak Test Results

Baseline: 18aead8d070976a64313939191fb9635623fc69e Comparison: b498b114260e1991232be340753a1e3a9076240e Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
http_text_to_http_json	1.36MiB	3.63	100.00%	37.55MiB	1.08MiB	22.51KiB	0	0.0286783	38.91MiB	1.24MiB	26.01KiB	0	0.0319675	False	False
syslog_log2metric_humio_metrics	165.31KiB	1.29	100.00%	12.52MiB	436.52KiB	8.91KiB	0	0.0340332	12.68MiB	535.4KiB	10.91KiB	0	0.0412111	False	False
datadog_agent_remap_datadog_logs_acks	744.09KiB	1.27	100.00%	57.29MiB	3.8MiB	79.43KiB	0	0.0663154	58.02MiB	5.01MiB	104.33KiB	0	0.0863508	False	False
socket_to_socket_blackhole	210.51KiB	0.88	100.00%	23.24MiB	325.99KiB	6.65KiB	0	0.0136979	23.44MiB	644.74KiB	13.17KiB	0	0.0268544	False	False
http_pipelines_blackhole_acks	5.88KiB	0.47	94.59%	1.22MiB	120.28KiB	2.45KiB	0	0.0965276	1.22MiB	89.49KiB	1.82KiB	0	0.0714797	False	False
syslog_regex_logs2metric_ddmetrics	51.55KiB	0.42	99.99%	11.96MiB	529.24KiB	10.77KiB	0	0.0432118	12.01MiB	388.07KiB	7.91KiB	0	0.0315528	False	False
datadog_agent_remap_datadog_logs	256.17KiB	0.41	99.75%	61.29MiB	412.69KiB	8.45KiB	0	0.00657405	61.54MiB	4.05MiB	84.27KiB	0	0.0657258	False	False
splunk_hec_indexer_ack_blackhole	24.39KiB	0.1	64.13%	23.74MiB	968.77KiB	19.7KiB	0	0.0398505	23.76MiB	876.14KiB	17.83KiB	0	0.0360039	False	False
syslog_humio_logs	14.18KiB	0.09	90.73%	16.22MiB	293.38KiB	5.99KiB	0	0.0176601	16.23MiB	289.96KiB	5.94KiB	0	0.0174392	False	False
splunk_hec_to_splunk_hec_logs_acks	14.62KiB	0.06	43.19%	23.74MiB	915.45KiB	18.61KiB	0	0.0376569	23.75MiB	865.02KiB	17.59KiB	0	0.0355609	False	False
splunk_hec_to_splunk_hec_logs_noack	15.03KiB	0.06	79.48%	23.82MiB	472.86KiB	9.65KiB	0	0.0193809	23.84MiB	337.5KiB	6.89KiB	0	0.0138243	False	False
enterprise_http_to_http	-3.73KiB	-0.02	38.34%	23.85MiB	257.27KiB	5.25KiB	0	0.0105333	23.84MiB	258.49KiB	5.29KiB	0	0.0105846	False	False
file_to_blackhole	-37.43KiB	-0.04	30.47%	95.34MiB	3.01MiB	62.32KiB	0	0.0315231	95.3MiB	3.48MiB	72.42KiB	0	0.0365047	False	False
http_to_http_json	-33.76KiB	-0.14	99.33%	23.85MiB	333.54KiB	6.81KiB	0	0.013656	23.81MiB	509.39KiB	10.4KiB	0	0.0208849	False	False
datadog_agent_remap_blackhole_acks	-151.8KiB	-0.26	72.75%	57.98MiB	5.18MiB	107.83KiB	0	0.0892917	57.83MiB	4.14MiB	86.63KiB	0	0.0715175	False	False
syslog_log2metric_splunk_hec_metrics	-54.28KiB	-0.31	99.17%	17.34MiB	554.6KiB	11.31KiB	0	0.0312328	17.28MiB	843.37KiB	17.16KiB	0	0.0476402	False	False
fluent_elasticsearch	-357.26KiB	-0.44	100.00%	79.47MiB	54.15KiB	1.09KiB	0	0.000665287	79.12MiB	4.12MiB	84.62KiB	0	0.0520132	False	False
http_to_http_noack	-110.5KiB	-0.45	100.00%	23.84MiB	397.57KiB	8.13KiB	0	0.0162849	23.73MiB	1.17MiB	24.49KiB	0	0.0495047	False	False
datadog_agent_remap_blackhole	-345.32KiB	-0.57	99.88%	59.31MiB	3.73MiB	77.77KiB	0	0.0628459	58.98MiB	3.47MiB	72.45KiB	0	0.0588958	False	False
http_to_http_acks	-158.76KiB	-0.88	50.13%	17.58MiB	7.81MiB	163.31KiB	0	0.444113	17.43MiB	8.07MiB	168.47KiB	0	0.462903	True	True
http_pipelines_blackhole	-17.23KiB	-1	100.00%	1.69MiB	47.81KiB	999.64B	0	0.0276374	1.67MiB	129.41KiB	2.64KiB	0	0.0755547	False	False
http_pipelines_no_grok_blackhole	-154.91KiB	-1.41	100.00%	10.73MiB	289.23KiB	5.9KiB	0	0.0263147	10.58MiB	1.0MiB	20.89KiB	0	0.0947739	False	False
splunk_hec_route_s3	-289.95KiB	-1.5	100.00%	18.85MiB	2.26MiB	46.95KiB	0	0.11966	18.56MiB	2.21MiB	46.29KiB	0	0.119247	False	False
syslog_loki	-353.34KiB	-2.43	100.00%	14.17MiB	717.88KiB	14.68KiB	0	0.0494483	13.83MiB	916.65KiB	18.63KiB	0	0.064715	False	False
syslog_splunk_hec_logs	-538.04KiB	-3.35	100.00%	15.69MiB	1.16MiB	24.15KiB	0	0.0738282	15.16MiB	1.35MiB	28.13KiB	0	0.0888735	False	False

[jszwedko]

Merged! Thanks for this contribution!