Closes #3532. Closes: #3702.

This is an implementation of a generic HTTP scraping source.

Much of the scraping logic was leveraged from the existing prometheus_scrape source. Or rather, the common/generic logic was extracted and now both prometheus_scrape and http_scrape leverage the common logic.

NOTES

I suspect I might be asked to move the common logic out of the src/sources/http_scrape/mod.rs , and into somewhere in the vector common area. Thinking about the effort to extract components into crates, the prometheus_scrape crate would have a dependency on the http_scrape crate . But I wasn't exactly sure where to put it.

TODO

[x] The placement of sending the HttpScrapeEventsSent seems wrong to me, but I was not sure how to get the data needed to send the event, after the stream was written.
[x] (int test) TLS test case using certs for the dufs container
[x] (int test) add a test case to validate headers applied
[x] (int test) add a shutdown signal test
[x] (int test) validate that a failure occurs when invalid auth is passed or auth is omitted but required. (ie, I see provisions for happy path testing but need to assert the failure in this case).
[x] (int test) same as above but for invalid endpoint
[x] Multi-value header support

Un-implemented features tracked in the below issues

#13889 (OAuth support) #13888 (Compression support)

Aug 01 '22 21:08 neuronull

Deploy Preview for vector-project ready!

Name	Link
Latest commit	33c236419b6c65f23d143556aca131cf7811e165
Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/6308d8ec5ffdce0008889239
Deploy Preview	https://deploy-preview-13793--vector-project.netlify.app
Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

Aug 01 '22 21:08 netlify[bot]

Soak Test Results

Baseline: 3fd07543f5c2db8d7243753458cc088340daf18e Comparison: 09dbf431b0690ef73beaeb942a62e50be5d7adf6 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
http_pipelines_blackhole	60.93KiB	3.84	100.00%	1.55MiB	94.29KiB	1.93KiB	0.0593724	1.61MiB	99.48KiB	2.03KiB	0.0603244	False	False
splunk_hec_route_s3	423.78KiB	2.28	100.00%	18.18MiB	2.49MiB	51.77KiB	0.136742	18.59MiB	2.38MiB	49.79KiB	0.127962	False	False
syslog_loki	190.09KiB	1.25	100.00%	14.85MiB	118.43KiB	2.43KiB	0.00778741	15.03MiB	747.1KiB	15.19KiB	0.048518	False	False
syslog_splunk_hec_logs	122.81KiB	0.72	100.00%	16.57MiB	848.94KiB	17.27KiB	0.0500277	16.69MiB	809.04KiB	16.47KiB	0.047334	False	False
http_text_to_http_json	237.09KiB	0.58	100.00%	39.62MiB	729.26KiB	14.89KiB	0.0179712	39.85MiB	707.56KiB	14.44KiB	0.0173352	False	False
http_pipelines_blackhole_acks	4.71KiB	0.39	92.87%	1.18MiB	109.56KiB	2.23KiB	0.0905557	1.19MiB	66.67KiB	1.36KiB	0.0548965	False	False
datadog_agent_remap_datadog_logs_acks	108.74KiB	0.17	63.69%	63.77MiB	3.52MiB	73.49KiB	0.0551646	63.88MiB	4.53MiB	94.29KiB	0.0708977	False	False
syslog_regex_logs2metric_ddmetrics	19.73KiB	0.15	71.85%	12.8MiB	676.61KiB	13.77KiB	0.051596	12.82MiB	592.45KiB	12.08KiB	0.0451108	False	False
syslog_log2metric_splunk_hec_metrics	25.11KiB	0.13	77.63%	18.54MiB	624.71KiB	12.73KiB	0.0328948	18.57MiB	798.0KiB	16.24KiB	0.041964	False	False
splunk_hec_to_splunk_hec_logs_noack	26.99KiB	0.11	96.04%	23.81MiB	548.78KiB	11.2KiB	0.0225029	23.84MiB	334.08KiB	6.82KiB	0.013684	False	False
syslog_humio_logs	17.48KiB	0.1	100.00%	17.09MiB	113.0KiB	2.31KiB	0.00645615	17.11MiB	127.92KiB	2.62KiB	0.00730181	False	False
splunk_hec_indexer_ack_blackhole	7.02KiB	0.03	23.24%	23.75MiB	854.79KiB	17.39KiB	0.0351354	23.76MiB	795.19KiB	16.18KiB	0.0326762	False	False
splunk_hec_to_splunk_hec_logs_acks	2.34KiB	0.01	7.32%	23.75MiB	923.07KiB	18.78KiB	0.0379455	23.75MiB	845.15KiB	17.19KiB	0.0347388	False	False
enterprise_http_to_http	-1.08KiB	-0	11.94%	23.85MiB	249.63KiB	5.1KiB	0.0102207	23.85MiB	248.91KiB	5.1KiB	0.0101917	False	False
http_to_http_json	999.55B	0	5.28%	23.82MiB	496.92KiB	10.14KiB	0.0203709	23.82MiB	523.28KiB	10.69KiB	0.0214503	False	False
file_to_blackhole	-56.26KiB	-0.06	50.45%	95.35MiB	2.72MiB	56.37KiB	0.0285144	95.29MiB	2.9MiB	60.29KiB	0.0304333	False	False
fluent_elasticsearch	-194.22KiB	-0.24	100.00%	79.47MiB	53.67KiB	1.09KiB	0.000659328	79.28MiB	1.69MiB	34.87KiB	0.0213673	False	False
http_to_http_noack	-119.46KiB	-0.49	100.00%	23.84MiB	256.87KiB	5.25KiB	0.0105179	23.73MiB	1.16MiB	24.2KiB	0.0489307	False	False
datadog_agent_remap_datadog_logs	-329.87KiB	-0.5	99.95%	64.13MiB	355.28KiB	7.28KiB	0.00540869	63.81MiB	4.52MiB	94.03KiB	0.0707739	False	False
http_pipelines_no_grok_blackhole	-57.07KiB	-0.5	98.53%	11.05MiB	280.3KiB	5.72KiB	0.0247605	11.0MiB	1.09MiB	22.66KiB	0.098912	False	False
syslog_log2metric_humio_metrics	-97.87KiB	-0.74	100.00%	12.97MiB	206.64KiB	4.22KiB	0.0155537	12.88MiB	528.59KiB	10.76KiB	0.040082	False	False
datadog_agent_remap_blackhole	-633.77KiB	-1	100.00%	62.11MiB	4.23MiB	88.08KiB	0.0680075	61.5MiB	4.42MiB	92.1KiB	0.071889	False	False
datadog_agent_remap_blackhole_acks	-658.85KiB	-1.03	100.00%	62.22MiB	4.43MiB	92.35KiB	0.0712333	61.57MiB	3.5MiB	73.22KiB	0.0568046	False	False
http_to_http_acks	-243.7KiB	-1.32	65.95%	18.04MiB	8.85MiB	185.09KiB	0.490603	17.8MiB	8.45MiB	176.36KiB	0.474587	True	True
socket_to_socket_blackhole	-524.52KiB	-2.16	100.00%	23.69MiB	291.81KiB	5.96KiB	0.0120269	23.18MiB	211.33KiB	4.31KiB	0.00890238	False	False

Aug 02 '22 00:08 github-actions[bot]

Quick notes - this is missing the markdown file in website/content/en/docs/reference/configuration/sources/ that provides the documentation scaffolding on the site right now.

Ohh I saw #13753 , and thought that it was intentionally left out of the initial changes, and the "publishing" was something that was done at a later point for some reason.

Aug 02 '22 14:08 neuronull

Quick notes - this is missing the markdown file in website/content/en/docs/reference/configuration/sources/ that provides the documentation scaffolding on the site right now.

Ohh I saw #13753 , and thought that it was intentionally left out of the initial changes, and the "publishing" was something that was done at a later point for some reason.

🤔 maybe it's a matter of preference - I like to have it included in the same PR so I can review how the CUE gets rendered. Either way it won't be on vector.dev until we do an actual release, but it will be on master.vector.dev

Aug 02 '22 17:08 spencergilbert

Soak Test Results

Baseline: 2a889d14fe92b1eb5c85209aa891ab52af06f770 Comparison: 32acc20f385ee0dcfbd3f122b51a44dd4ba793f8 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
syslog_log2metric_splunk_hec_metrics	125.21KiB	0.67	100.00%	18.17MiB	629.25KiB	12.82KiB	0.0338174	18.29MiB	785.9KiB	16.0KiB	0.0419537	False	False
syslog_splunk_hec_logs	79.82KiB	0.47	99.96%	16.46MiB	795.58KiB	16.22KiB	0.0471788	16.54MiB	759.52KiB	15.49KiB	0.0448286	False	False
http_pipelines_blackhole_acks	4.44KiB	0.37	90.58%	1.18MiB	106.49KiB	2.17KiB	0.0879589	1.19MiB	75.17KiB	1.53KiB	0.0618624	False	False
syslog_regex_logs2metric_ddmetrics	38.27KiB	0.3	96.47%	12.46MiB	615.95KiB	12.55KiB	0.0482695	12.5MiB	645.19KiB	13.15KiB	0.0504093	False	False
splunk_hec_to_splunk_hec_logs_noack	22.18KiB	0.09	92.69%	23.82MiB	505.78KiB	10.32KiB	0.0207348	23.84MiB	334.45KiB	6.83KiB	0.0136987	False	False
splunk_hec_indexer_ack_blackhole	9.12KiB	0.04	28.38%	23.75MiB	877.93KiB	17.85KiB	0.0360993	23.75MiB	865.28KiB	17.6KiB	0.0355657	False	False
splunk_hec_route_s3	5.54KiB	0.03	6.81%	19.11MiB	2.24MiB	46.73KiB	0.117405	19.11MiB	2.14MiB	44.91KiB	0.112184	False	False
http_to_http_acks	2.23KiB	0.01	0.74%	18.47MiB	8.31MiB	173.68KiB	0.449671	18.47MiB	7.87MiB	164.33KiB	0.425954	True	True
enterprise_http_to_http	-677.55B	-0	7.24%	23.85MiB	251.88KiB	5.14KiB	0.0103131	23.85MiB	252.13KiB	5.16KiB	0.0103235	False	False
splunk_hec_to_splunk_hec_logs_acks	-5.04KiB	-0.02	17.31%	23.77MiB	787.34KiB	16.02KiB	0.0323464	23.76MiB	812.16KiB	16.53KiB	0.0333731	False	False
datadog_agent_remap_blackhole_acks	-39.14KiB	-0.06	19.55%	61.43MiB	5.42MiB	112.91KiB	0.0882027	61.39MiB	5.29MiB	110.63KiB	0.0861329	False	False
file_to_blackhole	-145.98KiB	-0.15	72.41%	95.32MiB	3.99MiB	82.69KiB	0.0418579	95.18MiB	5.08MiB	105.39KiB	0.0533182	False	False
http_to_http_json	-38.89KiB	-0.16	99.69%	23.84MiB	347.06KiB	7.09KiB	0.0142111	23.81MiB	542.51KiB	11.07KiB	0.0222495	False	False
http_to_http_noack	-68.3KiB	-0.28	99.67%	23.83MiB	525.17KiB	10.73KiB	0.0215176	23.76MiB	1012.88KiB	20.63KiB	0.0416168	False	False
datadog_agent_remap_datadog_logs_acks	-189.16KiB	-0.29	89.88%	63.23MiB	3.22MiB	67.28KiB	0.0508785	63.04MiB	4.5MiB	93.71KiB	0.0713832	False	False
http_text_to_http_json	-129.75KiB	-0.33	100.00%	38.5MiB	885.97KiB	18.08KiB	0.0224674	38.37MiB	949.65KiB	19.38KiB	0.0241617	False	False
datadog_agent_remap_datadog_logs	-235.86KiB	-0.36	99.15%	64.03MiB	340.12KiB	6.96KiB	0.00518614	63.8MiB	4.29MiB	89.27KiB	0.0671767	False	False
fluent_elasticsearch	-311.96KiB	-0.38	100.00%	79.47MiB	53.78KiB	1.09KiB	0.000660725	79.17MiB	3.12MiB	64.04KiB	0.0393783	False	False
http_pipelines_no_grok_blackhole	-206.56KiB	-1.82	100.00%	11.07MiB	289.21KiB	5.91KiB	0.0255026	10.87MiB	1.11MiB	23.19KiB	0.102435	False	False
syslog_humio_logs	-327.41KiB	-1.88	100.00%	17.04MiB	247.05KiB	5.04KiB	0.0141539	16.72MiB	601.54KiB	12.31KiB	0.0351226	False	False
http_pipelines_blackhole	-32.48KiB	-2.01	100.00%	1.58MiB	110.5KiB	2.26KiB	0.0681959	1.55MiB	149.04KiB	3.04KiB	0.0938604	False	False
datadog_agent_remap_blackhole	-1.25MiB	-2.04	100.00%	61.17MiB	6.09MiB	126.98KiB	0.099609	59.92MiB	5.2MiB	108.59KiB	0.0867955	False	False
syslog_loki	-339.06KiB	-2.26	100.00%	14.65MiB	330.42KiB	6.76KiB	0.0220195	14.32MiB	728.79KiB	14.81KiB	0.0496899	False	False
socket_to_socket_blackhole	-590.93KiB	-2.44	100.00%	23.63MiB	285.15KiB	5.82KiB	0.0117827	23.05MiB	135.1KiB	2.76KiB	0.00572249	False	False
syslog_log2metric_humio_metrics	-363.11KiB	-2.87	100.00%	12.36MiB	635.55KiB	12.96KiB	0.0502095	12.0MiB	687.63KiB	14.0KiB	0.0559287	False	False

Aug 02 '22 20:08 github-actions[bot]

Soak Test Results

Baseline: 47d2a0eb005f84952c674c78e90f4aaf326d6465 Comparison: 36761c6008d5e11cdf3c57b6f1c01c8c0b82b76f Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
http_text_to_http_json	2.6MiB	6.92	100.00%	37.47MiB	848.17KiB	17.31KiB	0.0220981	40.07MiB	773.59KiB	15.79KiB	0.0188497	False	False
socket_to_socket_blackhole	807.01KiB	3.54	100.00%	22.23MiB	818.31KiB	16.7KiB	0.0359351	23.02MiB	793.14KiB	16.19KiB	0.0336375	False	False
datadog_agent_remap_blackhole_acks	722.71KiB	1.1	100.00%	64.32MiB	4.51MiB	93.82KiB	0.0700471	65.02MiB	3.23MiB	67.48KiB	0.0496243	False	False
datadog_agent_remap_blackhole	570.73KiB	0.88	100.00%	63.34MiB	4.18MiB	87.18KiB	0.0660523	63.9MiB	3.52MiB	73.46KiB	0.0551303	False	False
splunk_hec_route_s3	101.47KiB	0.53	87.53%	18.6MiB	2.27MiB	47.22KiB	0.121893	18.7MiB	2.21MiB	46.22KiB	0.118132	False	False
syslog_regex_logs2metric_ddmetrics	37.03KiB	0.28	96.34%	12.71MiB	643.04KiB	13.1KiB	0.0493874	12.75MiB	584.83KiB	11.92KiB	0.0447892	False	False
splunk_hec_indexer_ack_blackhole	24.1KiB	0.1	63.99%	23.74MiB	962.25KiB	19.57KiB	0.0395748	23.76MiB	865.34KiB	17.61KiB	0.0355539	False	False
splunk_hec_to_splunk_hec_logs_noack	22.1KiB	0.09	92.60%	23.82MiB	504.63KiB	10.3KiB	0.0206883	23.84MiB	335.1KiB	6.84KiB	0.0137257	False	False
splunk_hec_to_splunk_hec_logs_acks	2.65KiB	0.01	9.01%	23.76MiB	818.68KiB	16.66KiB	0.033646	23.76MiB	805.61KiB	16.39KiB	0.0331053	False	False
enterprise_http_to_http	-52.22B	-0	0.57%	23.85MiB	246.64KiB	5.03KiB	0.0100987	23.85MiB	251.59KiB	5.15KiB	0.0103012	False	False
file_to_blackhole	311.08B	0	0.25%	95.3MiB	3.67MiB	76.05KiB	0.0384895	95.3MiB	2.95MiB	61.3KiB	0.0309139	False	False
http_to_http_acks	-3.73KiB	-0.02	1.20%	17.91MiB	8.23MiB	172.16KiB	0.459492	17.91MiB	8.49MiB	177.28KiB	0.473662	True	True
datadog_agent_remap_datadog_logs_acks	-60.86KiB	-0.1	39.94%	62.11MiB	3.45MiB	71.98KiB	0.0554786	62.05MiB	4.39MiB	91.3KiB	0.0706748	False	False
http_to_http_json	-40.43KiB	-0.17	99.78%	23.85MiB	352.52KiB	7.2KiB	0.0144334	23.81MiB	543.08KiB	11.09KiB	0.0222725	False	False
datadog_agent_remap_datadog_logs	-136.6KiB	-0.21	87.36%	63.67MiB	424.81KiB	8.7KiB	0.00651457	63.53MiB	4.27MiB	88.92KiB	0.0671941	False	False
fluent_elasticsearch	-201.2KiB	-0.25	100.00%	79.47MiB	53.07KiB	1.07KiB	0.00065205	79.28MiB	1.78MiB	36.54KiB	0.0223872	False	False
http_to_http_noack	-95.11KiB	-0.39	99.97%	23.83MiB	522.39KiB	10.68KiB	0.021406	23.73MiB	1.15MiB	24.02KiB	0.0485406	False	False
syslog_humio_logs	-83.83KiB	-0.48	100.00%	17.11MiB	231.88KiB	4.73KiB	0.0132312	17.03MiB	173.32KiB	3.55KiB	0.00993729	False	False
syslog_loki	-73.85KiB	-0.49	99.86%	14.63MiB	717.31KiB	14.68KiB	0.0478728	14.56MiB	879.51KiB	17.88KiB	0.0589891	False	False
syslog_splunk_hec_logs	-110.86KiB	-0.65	100.00%	16.67MiB	796.76KiB	16.21KiB	0.0466563	16.57MiB	700.17KiB	14.26KiB	0.0412684	False	False
http_pipelines_no_grok_blackhole	-80.95KiB	-0.73	99.45%	10.76MiB	744.86KiB	15.21KiB	0.0675988	10.68MiB	1.19MiB	24.84KiB	0.111607	False	False
syslog_log2metric_splunk_hec_metrics	-216.75KiB	-1.15	100.00%	18.41MiB	498.04KiB	10.16KiB	0.0264142	18.2MiB	851.47KiB	17.32KiB	0.0456838	False	False
http_pipelines_blackhole_acks	-15.93KiB	-1.35	100.00%	1.15MiB	106.46KiB	2.17KiB	0.0902659	1.14MiB	99.31KiB	2.02KiB	0.0853621	False	False
http_pipelines_blackhole	-24.44KiB	-1.51	100.00%	1.58MiB	95.84KiB	1.96KiB	0.0592748	1.55MiB	142.16KiB	2.9KiB	0.0892772	False	False
syslog_log2metric_humio_metrics	-358.65KiB	-2.71	100.00%	12.92MiB	534.89KiB	10.92KiB	0.0404327	12.57MiB	784.86KiB	15.97KiB	0.0609823	False	False

Aug 03 '22 00:08 github-actions[bot]

Soak Test Results

Baseline: c2a0d5a39fd32680890392c252c7c3e5e865b5e3 Comparison: 3d6906e8c5d450b5fdcf23537c9fecb173b94401 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
socket_to_socket_blackhole	928.57KiB	4.12	100.00%	22.02MiB	556.81KiB	11.37KiB	0.0246868	22.93MiB	568.09KiB	11.6KiB	0.0241909	False	False
http_text_to_http_json	1.45MiB	3.76	100.00%	38.44MiB	937.44KiB	19.14KiB	0.0238124	39.88MiB	907.33KiB	18.52KiB	0.0222116	False	False
http_pipelines_blackhole_acks	26.39KiB	2.36	100.00%	1.09MiB	117.22KiB	2.39KiB	0.104628	1.12MiB	91.46KiB	1.86KiB	0.0797559	False	False
syslog_regex_logs2metric_ddmetrics	278.54KiB	2.18	100.00%	12.49MiB	543.91KiB	11.09KiB	0.0425063	12.77MiB	567.97KiB	11.58KiB	0.0434411	False	False
syslog_splunk_hec_logs	318.92KiB	1.95	100.00%	15.97MiB	932.87KiB	18.97KiB	0.057041	16.28MiB	824.85KiB	16.83KiB	0.0494711	False	False
datadog_agent_remap_blackhole_acks	923.78KiB	1.57	100.00%	57.4MiB	4.78MiB	99.56KiB	0.0832509	58.31MiB	4.32MiB	90.4KiB	0.0740697	False	False
syslog_loki	214.05KiB	1.43	100.00%	14.57MiB	312.3KiB	6.39KiB	0.020921	14.78MiB	744.98KiB	15.14KiB	0.0492013	False	False
http_to_http_acks	251.6KiB	1.39	67.60%	17.69MiB	8.66MiB	181.05KiB	0.489219	17.94MiB	8.61MiB	179.67KiB	0.479739	True	True
splunk_hec_route_s3	192.3KiB	1.01	99.53%	18.57MiB	2.39MiB	49.8KiB	0.128722	18.76MiB	2.22MiB	46.36KiB	0.118075	False	False
datadog_agent_remap_blackhole	456.41KiB	0.72	100.00%	62.25MiB	3.98MiB	83.1KiB	0.0639938	62.7MiB	3.52MiB	73.33KiB	0.0560694	False	False
syslog_log2metric_splunk_hec_metrics	74.83KiB	0.42	96.96%	17.31MiB	1.1MiB	22.93KiB	0.0634674	17.38MiB	1.24MiB	25.85KiB	0.0713573	False	False
splunk_hec_to_splunk_hec_logs_noack	31.43KiB	0.13	98.23%	23.81MiB	559.79KiB	11.41KiB	0.0229569	23.84MiB	328.84KiB	6.71KiB	0.0134684	False	False
splunk_hec_indexer_ack_blackhole	11.87KiB	0.05	37.12%	23.75MiB	877.67KiB	17.85KiB	0.0360807	23.76MiB	828.69KiB	16.86KiB	0.0340502	False	False
enterprise_http_to_http	-2.37KiB	-0.01	25.22%	23.85MiB	256.5KiB	5.24KiB	0.0105016	23.84MiB	252.72KiB	5.17KiB	0.0103477	False	False
splunk_hec_to_splunk_hec_logs_acks	-7.8KiB	-0.03	23.06%	23.79MiB	914.4KiB	18.61KiB	0.037527	23.78MiB	934.69KiB	19.02KiB	0.0383724	False	False
syslog_humio_logs	-5.53KiB	-0.03	55.79%	16.96MiB	261.3KiB	5.33KiB	0.0150428	16.95MiB	236.12KiB	4.83KiB	0.0135977	False	False
http_to_http_json	-27.25KiB	-0.11	97.00%	23.84MiB	363.67KiB	7.42KiB	0.0148922	23.82MiB	494.94KiB	10.12KiB	0.02029	False	False
file_to_blackhole	-167.97KiB	-0.17	85.93%	95.33MiB	2.99MiB	61.92KiB	0.0313789	95.16MiB	4.61MiB	95.7KiB	0.0484239	False	False
datadog_agent_remap_datadog_logs_acks	-209.84KiB	-0.33	90.49%	61.3MiB	3.86MiB	80.53KiB	0.0628874	61.1MiB	4.64MiB	96.5KiB	0.0758642	False	False
fluent_elasticsearch	-333.83KiB	-0.41	100.00%	79.47MiB	52.45KiB	1.06KiB	0.000644357	79.15MiB	3.34MiB	68.66KiB	0.042206	False	False
http_to_http_noack	-129.28KiB	-0.53	100.00%	23.84MiB	411.21KiB	8.4KiB	0.0168425	23.71MiB	1.24MiB	25.93KiB	0.0524721	False	False
http_pipelines_blackhole	-13.55KiB	-0.8	100.00%	1.65MiB	42.16KiB	881.9B	0.0250079	1.63MiB	105.6KiB	2.15KiB	0.0631383	False	False
http_pipelines_no_grok_blackhole	-103.74KiB	-0.93	100.00%	10.92MiB	382.36KiB	7.8KiB	0.0341977	10.82MiB	1.11MiB	23.08KiB	0.10247	False	False
syslog_log2metric_humio_metrics	-121.04KiB	-0.94	100.00%	12.63MiB	308.41KiB	6.29KiB	0.0238484	12.51MiB	543.6KiB	11.06KiB	0.0424325	False	False
datadog_agent_remap_datadog_logs	-952.52KiB	-1.51	100.00%	61.56MiB	3.56MiB	74.52KiB	0.0577761	60.63MiB	5.4MiB	112.38KiB	0.0889751	False	False

Aug 03 '22 18:08 github-actions[bot]

Soak Test Results

Baseline: f64bceffe36cf987b23bd541bebc5ef8498d8171 Comparison: 18f74482443a54c84d53f34af2e4f3ed8e112133 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
http_pipelines_blackhole_acks	7.66KiB	0.64	99.00%	1.17MiB	108.08KiB	2.2KiB	0.0902102	1.18MiB	98.31KiB	2.0KiB	0.081535	False	False
splunk_hec_to_splunk_hec_logs_noack	21.4KiB	0.09	91.84%	23.82MiB	503.18KiB	10.27KiB	0.0206287	23.84MiB	330.21KiB	6.74KiB	0.0135255	False	False
enterprise_http_to_http	-2.01KiB	-0.01	21.89%	23.85MiB	245.36KiB	5.01KiB	0.0100458	23.84MiB	255.08KiB	5.22KiB	0.0104447	False	False
syslog_log2metric_splunk_hec_metrics	-8.04KiB	-0.04	25.94%	17.79MiB	733.95KiB	14.97KiB	0.0402857	17.78MiB	940.06KiB	19.13KiB	0.0516213	False	False
splunk_hec_to_splunk_hec_logs_acks	-13.24KiB	-0.05	41.99%	23.76MiB	801.56KiB	16.31KiB	0.0329366	23.75MiB	860.13KiB	17.5KiB	0.0353623	False	False
syslog_regex_logs2metric_ddmetrics	-7.66KiB	-0.06	33.70%	12.59MiB	622.94KiB	12.69KiB	0.0483035	12.58MiB	596.07KiB	12.15KiB	0.0462475	False	False
file_to_blackhole	-58.55KiB	-0.06	45.77%	95.37MiB	3.14MiB	65.03KiB	0.032885	95.31MiB	3.4MiB	70.75KiB	0.0356932	False	False
datadog_agent_remap_blackhole	-45.32KiB	-0.07	21.80%	60.15MiB	5.97MiB	124.45KiB	0.0992831	60.11MiB	5.11MiB	106.47KiB	0.0849185	False	False
splunk_hec_indexer_ack_blackhole	-18.53KiB	-0.08	51.08%	23.75MiB	893.85KiB	18.19KiB	0.0367418	23.73MiB	967.38KiB	19.67KiB	0.0397943	False	False
http_to_http_json	-31.02KiB	-0.13	98.46%	23.84MiB	358.21KiB	7.31KiB	0.0146688	23.81MiB	514.63KiB	10.51KiB	0.0211013	False	False
fluent_elasticsearch	-184.88KiB	-0.23	100.00%	79.47MiB	52.98KiB	1.07KiB	0.0006509	79.29MiB	1.67MiB	34.45KiB	0.021097	False	False
syslog_humio_logs	-39.86KiB	-0.23	100.00%	17.27MiB	126.77KiB	2.59KiB	0.00716769	17.23MiB	126.62KiB	2.59KiB	0.00717538	False	False
http_to_http_noack	-113.45KiB	-0.46	100.00%	23.84MiB	260.09KiB	5.32KiB	0.0106499	23.73MiB	1.15MiB	24.01KiB	0.0485167	False	False
http_pipelines_blackhole	-8.23KiB	-0.49	100.00%	1.64MiB	11.59KiB	242.45B	0.00691323	1.63MiB	87.13KiB	1.78KiB	0.0522319	False	False
http_pipelines_no_grok_blackhole	-69.0KiB	-0.61	99.70%	11.12MiB	150.97KiB	3.08KiB	0.0132521	11.06MiB	1.11MiB	23.02KiB	0.100005	False	False
http_text_to_http_json	-357.44KiB	-0.86	100.00%	40.44MiB	671.32KiB	13.7KiB	0.0162085	40.09MiB	744.63KiB	15.21KiB	0.018135	False	False
splunk_hec_route_s3	-199.46KiB	-1.03	99.74%	18.9MiB	2.26MiB	47.05KiB	0.119567	18.7MiB	2.22MiB	46.39KiB	0.118702	False	False
datadog_agent_remap_datadog_logs_acks	-774.93KiB	-1.19	100.00%	63.36MiB	3.16MiB	66.08KiB	0.0498485	62.6MiB	4.49MiB	93.46KiB	0.0716961	False	False
datadog_agent_remap_blackhole_acks	-885.08KiB	-1.36	100.00%	63.41MiB	4.9MiB	102.09KiB	0.0772953	62.54MiB	3.6MiB	75.44KiB	0.0576072	False	False
socket_to_socket_blackhole	-393.0KiB	-1.59	100.00%	24.15MiB	147.16KiB	3.0KiB	0.00594991	23.77MiB	271.87KiB	5.55KiB	0.0111693	False	False
http_to_http_acks	-297.09KiB	-1.6	76.02%	18.09MiB	8.44MiB	176.32KiB	0.466171	17.8MiB	8.67MiB	181.06KiB	0.487017	True	True
datadog_agent_remap_datadog_logs	-1.27MiB	-1.97	100.00%	64.7MiB	456.84KiB	9.34KiB	0.00689443	63.42MiB	4.38MiB	91.29KiB	0.0691198	False	False
syslog_splunk_hec_logs	-411.04KiB	-2.37	100.00%	16.93MiB	799.8KiB	16.27KiB	0.0461287	16.53MiB	865.41KiB	17.65KiB	0.0511249	False	False
syslog_log2metric_humio_metrics	-390.17KiB	-2.94	100.00%	12.96MiB	202.43KiB	4.13KiB	0.0152513	12.58MiB	478.29KiB	9.74KiB	0.037127	False	False
syslog_loki	-630.05KiB	-4.03	100.00%	15.25MiB	327.62KiB	6.71KiB	0.0209733	14.64MiB	732.75KiB	14.9KiB	0.0488809	False	False

Aug 04 '22 02:08 github-actions[bot]

Soak Test Results

Baseline: c402e6ed94400959dbc298d4b342dcb0842089d4 Comparison: aaeac7482c81db4f72755164e977bdef61fa35b5 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
socket_to_socket_blackhole	1.42MiB	6.48	100.00%	21.86MiB	836.22KiB	17.07KiB	0.0373419	23.28MiB	826.12KiB	16.86KiB	0.0346461	False	False
http_text_to_http_json	1.94MiB	5.17	100.00%	37.58MiB	1016.22KiB	20.74KiB	0.0264004	39.53MiB	1.12MiB	23.41KiB	0.0283252	False	False
http_pipelines_blackhole_acks	20.34KiB	1.73	100.00%	1.15MiB	123.84KiB	2.52KiB	0.105355	1.17MiB	86.67KiB	1.77KiB	0.0724762	False	False
splunk_hec_route_s3	269.11KiB	1.39	100.00%	18.85MiB	2.19MiB	45.63KiB	0.116185	19.11MiB	2.15MiB	44.95KiB	0.112471	False	False
datadog_agent_remap_blackhole	636.47KiB	1.02	99.99%	60.98MiB	6.06MiB	126.12KiB	0.0992908	61.61MiB	4.93MiB	102.97KiB	0.0800667	False	False
syslog_regex_logs2metric_ddmetrics	118.01KiB	0.91	100.00%	12.66MiB	646.58KiB	13.17KiB	0.0498818	12.77MiB	606.81KiB	12.37KiB	0.046391	False	False
http_to_http_acks	82.75KiB	0.45	26.17%	17.88MiB	8.46MiB	176.85KiB	0.472997	17.96MiB	8.31MiB	173.44KiB	0.462478	True	True
datadog_agent_remap_blackhole_acks	169.17KiB	0.26	92.94%	64.66MiB	4.02MiB	83.79KiB	0.0621741	64.83MiB	1.98MiB	41.59KiB	0.03061	False	False
datadog_agent_remap_datadog_logs_acks	144.94KiB	0.22	80.22%	63.28MiB	2.97MiB	62.02KiB	0.0468627	63.42MiB	4.51MiB	93.9KiB	0.071113	False	False
splunk_hec_to_splunk_hec_logs_noack	11.0KiB	0.05	67.74%	23.83MiB	431.31KiB	8.81KiB	0.0176735	23.84MiB	332.3KiB	6.78KiB	0.0136103	False	False
syslog_humio_logs	6.1KiB	0.03	88.05%	17.15MiB	132.91KiB	2.71KiB	0.00756724	17.16MiB	137.9KiB	2.82KiB	0.0078486	False	False
splunk_hec_indexer_ack_blackhole	2.08KiB	0.01	6.63%	23.75MiB	867.31KiB	17.64KiB	0.0356527	23.75MiB	869.44KiB	17.69KiB	0.035737	False	False
splunk_hec_to_splunk_hec_logs_acks	-897.76B	-0	2.83%	23.75MiB	860.84KiB	17.51KiB	0.0353899	23.75MiB	856.75KiB	17.42KiB	0.035223	False	False
enterprise_http_to_http	-1.09KiB	-0	12.18%	23.85MiB	243.14KiB	4.96KiB	0.00995496	23.85MiB	246.73KiB	5.05KiB	0.0101024	False	False
file_to_blackhole	-9.5KiB	-0.01	6.60%	95.31MiB	3.87MiB	80.12KiB	0.0405834	95.3MiB	3.95MiB	82.03KiB	0.0414263	False	False
http_to_http_json	-42.16KiB	-0.17	99.81%	23.84MiB	357.39KiB	7.3KiB	0.0146359	23.8MiB	560.5KiB	11.42KiB	0.0229931	False	False
syslog_log2metric_splunk_hec_metrics	-30.92KiB	-0.17	76.47%	18.06MiB	852.52KiB	17.36KiB	0.0460803	18.03MiB	954.17KiB	19.42KiB	0.0516606	False	False
syslog_splunk_hec_logs	-33.28KiB	-0.19	89.16%	16.92MiB	777.42KiB	15.82KiB	0.0448624	16.89MiB	656.93KiB	13.4KiB	0.0379821	False	False
fluent_elasticsearch	-177.66KiB	-0.22	100.00%	79.47MiB	54.32KiB	1.1KiB	0.000667358	79.3MiB	1.51MiB	31.16KiB	0.019093	False	False
http_pipelines_blackhole	-5.43KiB	-0.33	99.40%	1.61MiB	38.95KiB	815.42B	0.0235914	1.61MiB	88.71KiB	1.81KiB	0.0539143	False	False
http_to_http_noack	-109.08KiB	-0.45	100.00%	23.84MiB	404.58KiB	8.27KiB	0.0165722	23.73MiB	1.16MiB	24.1KiB	0.0487158	False	False
datadog_agent_remap_datadog_logs	-678.59KiB	-1.07	100.00%	61.82MiB	2.6MiB	54.42KiB	0.0419946	61.16MiB	4.7MiB	97.85KiB	0.076783	False	False
syslog_log2metric_humio_metrics	-197.22KiB	-1.48	100.00%	13.01MiB	212.85KiB	4.35KiB	0.0159782	12.81MiB	498.71KiB	10.15KiB	0.0379994	False	False
http_pipelines_no_grok_blackhole	-182.55KiB	-1.6	100.00%	11.12MiB	388.96KiB	7.94KiB	0.0341432	10.94MiB	1.14MiB	23.76KiB	0.104255	False	False
syslog_loki	-530.38KiB	-3.43	100.00%	15.08MiB	281.18KiB	5.76KiB	0.018204	14.56MiB	729.41KiB	14.83KiB	0.0489021	False	False

Aug 04 '22 23:08 github-actions[bot]

Soak Test Results

Baseline: 5707371b031a1d888a892452aec2cb52ac24caba Comparison: fe4dbd0eace0f12a7926493f9235bbd19192ea4d Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
syslog_log2metric_humio_metrics	124.51KiB	0.94	100.00%	12.91MiB	141.64KiB	2.89KiB	0.010712	13.03MiB	548.74KiB	11.17KiB	0.0411129	False	False
socket_to_socket_blackhole	196.73KiB	0.83	100.00%	23.08MiB	736.98KiB	15.04KiB	0.0311745	23.27MiB	764.62KiB	15.61KiB	0.0320765	False	False
http_pipelines_blackhole_acks	9.82KiB	0.81	99.99%	1.18MiB	100.29KiB	2.04KiB	0.0830837	1.19MiB	72.41KiB	1.48KiB	0.0595052	False	False
splunk_hec_route_s3	97.78KiB	0.5	86.61%	18.95MiB	2.26MiB	46.96KiB	0.119032	19.05MiB	2.16MiB	45.26KiB	0.113554	False	False
datadog_agent_remap_blackhole_acks	178.05KiB	0.27	91.11%	64.81MiB	4.4MiB	91.54KiB	0.0678054	64.99MiB	2.42MiB	50.66KiB	0.0372315	False	False
datadog_agent_remap_datadog_logs_acks	110.46KiB	0.17	65.59%	62.03MiB	3.51MiB	73.34KiB	0.0566006	62.13MiB	4.36MiB	90.82KiB	0.0701954	False	False
splunk_hec_to_splunk_hec_logs_noack	11.29KiB	0.05	68.87%	23.83MiB	433.94KiB	8.86KiB	0.0177825	23.84MiB	331.74KiB	6.77KiB	0.0135883	False	False
splunk_hec_indexer_ack_blackhole	10.64KiB	0.04	35.82%	23.76MiB	814.75KiB	16.58KiB	0.0334787	23.77MiB	773.8KiB	15.75KiB	0.0317821	False	False
enterprise_http_to_http	764.58B	0	8.17%	23.85MiB	251.32KiB	5.13KiB	0.0102901	23.85MiB	252.4KiB	5.17KiB	0.010334	False	False
file_to_blackhole	-40.79KiB	-0.04	27.35%	95.33MiB	3.78MiB	78.28KiB	0.0396015	95.29MiB	4.15MiB	86.42KiB	0.0435572	False	False
splunk_hec_to_splunk_hec_logs_acks	-17.66KiB	-0.07	54.70%	23.77MiB	788.07KiB	16.04KiB	0.0323712	23.75MiB	846.55KiB	17.22KiB	0.0347989	False	False
http_to_http_json	-41.49KiB	-0.17	99.77%	23.84MiB	353.35KiB	7.21KiB	0.0144694	23.8MiB	566.49KiB	11.56KiB	0.0232368	False	False
datadog_agent_remap_blackhole	-167.02KiB	-0.26	93.50%	63.21MiB	3.59MiB	74.95KiB	0.0567741	63.05MiB	2.43MiB	50.71KiB	0.0385101	False	False
syslog_splunk_hec_logs	-49.18KiB	-0.3	98.10%	16.12MiB	756.14KiB	15.4KiB	0.0457923	16.07MiB	697.87KiB	14.22KiB	0.0423893	False	False
http_to_http_noack	-95.97KiB	-0.39	99.98%	23.83MiB	514.73KiB	10.53KiB	0.0210912	23.73MiB	1.15MiB	23.9KiB	0.0482876	False	False
syslog_humio_logs	-79.47KiB	-0.47	100.00%	16.47MiB	239.78KiB	4.9KiB	0.0142179	16.39MiB	232.02KiB	4.75KiB	0.013823	False	False
fluent_elasticsearch	-406.98KiB	-0.5	100.00%	79.47MiB	53.43KiB	1.08KiB	0.000656353	79.08MiB	4.33MiB	89.09KiB	0.0548044	False	False
http_pipelines_blackhole	-9.79KiB	-0.58	100.00%	1.66MiB	9.88KiB	206.81B	0.00582218	1.65MiB	108.79KiB	2.22KiB	0.0644542	False	False
datadog_agent_remap_datadog_logs	-542.72KiB	-0.84	100.00%	62.78MiB	526.1KiB	10.77KiB	0.00818262	62.25MiB	4.41MiB	91.88KiB	0.0708527	False	False
http_to_http_acks	-183.1KiB	-1.02	56.36%	17.5MiB	7.82MiB	163.46KiB	0.446673	17.33MiB	8.12MiB	169.17KiB	0.46859	True	True
http_text_to_http_json	-423.12KiB	-1.04	100.00%	39.87MiB	700.02KiB	14.29KiB	0.0171438	39.45MiB	715.61KiB	14.61KiB	0.0177092	False	False
syslog_loki	-171.41KiB	-1.16	100.00%	14.41MiB	474.73KiB	9.71KiB	0.0321601	14.25MiB	795.73KiB	16.18KiB	0.0545398	False	False
http_pipelines_no_grok_blackhole	-136.94KiB	-1.22	100.00%	10.97MiB	263.13KiB	5.37KiB	0.0234137	10.84MiB	1.1MiB	22.86KiB	0.101282	False	False
syslog_regex_logs2metric_ddmetrics	-152.72KiB	-1.22	100.00%	12.19MiB	434.28KiB	8.86KiB	0.0347831	12.04MiB	442.93KiB	9.03KiB	0.0359154	False	False
syslog_log2metric_splunk_hec_metrics	-255.26KiB	-1.39	100.00%	17.95MiB	444.2KiB	9.06KiB	0.0241565	17.7MiB	734.65KiB	14.95KiB	0.0405148	False	False

Aug 05 '22 23:08 github-actions[bot]

Soak Test Results

Baseline: a101266cadfbe7e548dd12dee1af0f3b315ea682 Comparison: 6772cd0f75f4a7606afb18484fd66a0e4281519f Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
socket_to_socket_blackhole	530.52KiB	2.29	100.00%	22.67MiB	267.33KiB	5.46KiB	0.0115156	23.18MiB	367.97KiB	7.51KiB	0.0154967	False	False
splunk_hec_route_s3	320.75KiB	1.75	100.00%	17.91MiB	2.32MiB	48.36KiB	0.129646	18.22MiB	2.19MiB	45.92KiB	0.120373	False	False
syslog_loki	208.48KiB	1.42	100.00%	14.3MiB	521.39KiB	10.69KiB	0.0356038	14.5MiB	873.97KiB	17.77KiB	0.0588429	False	False
splunk_hec_indexer_ack_blackhole	13.24KiB	0.05	40.65%	23.76MiB	890.61KiB	18.12KiB	0.0366032	23.77MiB	832.0KiB	16.94KiB	0.0341758	False	False
splunk_hec_to_splunk_hec_logs_noack	10.41KiB	0.04	65.40%	23.83MiB	428.62KiB	8.75KiB	0.0175624	23.84MiB	330.08KiB	6.74KiB	0.0135192	False	False
http_to_http_acks	6.95KiB	0.04	2.34%	17.14MiB	7.88MiB	164.65KiB	0.45936	17.15MiB	8.13MiB	169.76KiB	0.473716	True	True
enterprise_http_to_http	-1.64KiB	-0.01	18.01%	23.85MiB	246.64KiB	5.03KiB	0.0100983	23.85MiB	250.5KiB	5.12KiB	0.0102567	False	False
splunk_hec_to_splunk_hec_logs_acks	-1.26KiB	-0.01	3.97%	23.75MiB	871.58KiB	17.72KiB	0.035837	23.74MiB	881.38KiB	17.92KiB	0.0362418	False	False
file_to_blackhole	-47.17KiB	-0.05	45.85%	95.36MiB	2.49MiB	51.66KiB	0.0261336	95.31MiB	2.76MiB	57.45KiB	0.0289982	False	False
http_to_http_json	-30.01KiB	-0.12	98.10%	23.84MiB	360.25KiB	7.35KiB	0.0147521	23.81MiB	512.48KiB	10.47KiB	0.0210115	False	False
fluent_elasticsearch	-182.68KiB	-0.22	99.98%	79.47MiB	53.92KiB	1.09KiB	0.000662409	79.29MiB	2.38MiB	48.89KiB	0.0299698	False	False
syslog_log2metric_humio_metrics	-43.83KiB	-0.34	99.45%	12.76MiB	436.54KiB	8.91KiB	0.0334127	12.71MiB	639.63KiB	13.01KiB	0.0491223	False	False
http_to_http_noack	-95.46KiB	-0.39	99.98%	23.83MiB	507.11KiB	10.37KiB	0.0207774	23.74MiB	1.14MiB	23.67KiB	0.0478286	False	False
http_pipelines_blackhole	-6.71KiB	-0.42	98.38%	1.57MiB	67.33KiB	1.38KiB	0.0417832	1.57MiB	119.01KiB	2.43KiB	0.0741657	False	False
http_pipelines_blackhole_acks	-4.76KiB	-0.42	93.86%	1.11MiB	96.4KiB	1.96KiB	0.0847143	1.11MiB	79.46KiB	1.62KiB	0.0701208	False	False
datadog_agent_remap_blackhole	-313.79KiB	-0.48	99.65%	63.97MiB	4.12MiB	85.77KiB	0.0643347	63.66MiB	3.1MiB	64.57KiB	0.0486077	False	False
http_text_to_http_json	-225.84KiB	-0.56	100.00%	39.51MiB	757.58KiB	15.46KiB	0.0187188	39.29MiB	739.37KiB	15.1KiB	0.0183715	False	False
datadog_agent_remap_datadog_logs_acks	-484.79KiB	-0.75	100.00%	63.17MiB	3.2MiB	66.82KiB	0.0505919	62.7MiB	4.47MiB	92.95KiB	0.0712039	False	False
datadog_agent_remap_blackhole_acks	-507.78KiB	-0.79	99.98%	62.52MiB	5.03MiB	104.83KiB	0.0804951	62.03MiB	4.28MiB	89.2KiB	0.0689165	False	False
http_pipelines_no_grok_blackhole	-100.26KiB	-0.89	100.00%	10.94MiB	173.02KiB	3.53KiB	0.0154354	10.85MiB	1.09MiB	22.65KiB	0.100291	False	False
syslog_log2metric_splunk_hec_metrics	-159.04KiB	-0.94	100.00%	16.49MiB	530.14KiB	10.81KiB	0.0313862	16.34MiB	663.38KiB	13.51KiB	0.0396484	False	False
syslog_regex_logs2metric_ddmetrics	-122.52KiB	-0.96	100.00%	12.4MiB	548.29KiB	11.18KiB	0.043168	12.28MiB	519.62KiB	10.59KiB	0.0413093	False	False
syslog_splunk_hec_logs	-163.27KiB	-0.99	100.00%	16.14MiB	792.54KiB	16.15KiB	0.047936	15.98MiB	724.49KiB	14.77KiB	0.0442572	False	False
datadog_agent_remap_datadog_logs	-648.47KiB	-1.01	100.00%	63.0MiB	862.01KiB	17.64KiB	0.0133594	62.37MiB	4.25MiB	88.52KiB	0.068142	False	False
syslog_humio_logs	-179.77KiB	-1.06	100.00%	16.64MiB	142.51KiB	2.91KiB	0.00836233	16.46MiB	120.26KiB	2.46KiB	0.00713201	False	False

Aug 08 '22 18:08 github-actions[bot]

Soak Test Results

Baseline: 44cfd901ec18913d461a62458f5cd777baef6fd1 Comparison: ae11fa37ed65027d54e61dbde56d345943ff07a5 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
syslog_loki	465.23KiB	3.26	100.00%	13.94MiB	260.76KiB	5.34KiB	0.0182588	14.4MiB	763.25KiB	15.52KiB	0.051757	False	False
datadog_agent_remap_blackhole_acks	1.42MiB	2.35	100.00%	60.45MiB	5.5MiB	114.54KiB	0.0909492	61.87MiB	3.93MiB	82.27KiB	0.0635173	False	False
http_text_to_http_json	691.19KiB	1.71	100.00%	39.5MiB	791.59KiB	16.16KiB	0.0195663	40.18MiB	829.39KiB	16.93KiB	0.0201561	False	False
datadog_agent_remap_blackhole	1005.74KiB	1.69	100.00%	58.23MiB	4.89MiB	101.87KiB	0.0839043	59.21MiB	4.45MiB	92.91KiB	0.0751922	False	False
socket_to_socket_blackhole	164.76KiB	0.71	100.00%	22.54MiB	258.13KiB	5.27KiB	0.0111792	22.71MiB	438.92KiB	8.96KiB	0.018874	False	False
splunk_hec_route_s3	121.74KiB	0.63	93.83%	19.01MiB	2.29MiB	47.6KiB	0.120197	19.13MiB	2.13MiB	44.48KiB	0.111157	False	False
syslog_humio_logs	67.85KiB	0.41	100.00%	16.33MiB	311.87KiB	6.37KiB	0.0186512	16.39MiB	304.25KiB	6.23KiB	0.0181221	False	False
http_pipelines_blackhole_acks	4.23KiB	0.35	91.24%	1.19MiB	102.38KiB	2.09KiB	0.0841578	1.19MiB	65.69KiB	1.34KiB	0.0538115	False	False
syslog_log2metric_splunk_hec_metrics	51.33KiB	0.29	96.10%	17.26MiB	707.36KiB	14.43KiB	0.0400133	17.31MiB	994.61KiB	20.24KiB	0.0560991	False	False
syslog_regex_logs2metric_ddmetrics	19.61KiB	0.16	84.62%	12.05MiB	449.23KiB	9.15KiB	0.036397	12.07MiB	503.48KiB	10.26KiB	0.0407277	False	False
splunk_hec_to_splunk_hec_logs_noack	15.28KiB	0.06	80.69%	23.82MiB	466.9KiB	9.53KiB	0.0191356	23.84MiB	336.12KiB	6.86KiB	0.0137669	False	False
enterprise_http_to_http	538.61B	0	5.74%	23.85MiB	254.66KiB	5.2KiB	0.0104273	23.85MiB	250.79KiB	5.14KiB	0.0102686	False	False
splunk_hec_indexer_ack_blackhole	-2.16KiB	-0.01	6.98%	23.75MiB	849.03KiB	17.27KiB	0.0349057	23.75MiB	867.47KiB	17.64KiB	0.0356669	False	False
splunk_hec_to_splunk_hec_logs_acks	-4.91KiB	-0.02	15.72%	23.75MiB	856.14KiB	17.41KiB	0.0351931	23.75MiB	865.82KiB	17.61KiB	0.0355981	False	False
file_to_blackhole	-56.35KiB	-0.06	42.72%	95.34MiB	3.15MiB	65.38KiB	0.0330742	95.29MiB	3.63MiB	75.55KiB	0.0381001	False	False
syslog_splunk_hec_logs	-16.54KiB	-0.1	61.05%	16.45MiB	732.57KiB	14.91KiB	0.0434702	16.44MiB	594.65KiB	12.13KiB	0.0353213	False	False
fluent_elasticsearch	-160.08KiB	-0.2	100.00%	79.47MiB	53.07KiB	1.07KiB	0.000652027	79.32MiB	1.5MiB	30.95KiB	0.0189484	False	False
http_to_http_json	-57.14KiB	-0.23	99.99%	23.85MiB	348.16KiB	7.11KiB	0.0142546	23.79MiB	620.69KiB	12.65KiB	0.0254721	False	False
datadog_agent_remap_datadog_logs_acks	-181.33KiB	-0.29	92.24%	61.12MiB	2.64MiB	55.17KiB	0.0431218	60.94MiB	4.16MiB	86.67KiB	0.0682945	False	False
http_to_http_noack	-127.83KiB	-0.52	100.00%	23.85MiB	255.86KiB	5.23KiB	0.010476	23.72MiB	1.2MiB	25.1KiB	0.0507791	False	False
http_to_http_acks	-111.37KiB	-0.62	36.71%	17.52MiB	7.98MiB	166.77KiB	0.455415	17.41MiB	7.81MiB	162.91KiB	0.448314	True	True
syslog_log2metric_humio_metrics	-98.6KiB	-0.75	100.00%	12.9MiB	270.0KiB	5.51KiB	0.0204336	12.81MiB	475.25KiB	9.68KiB	0.0362366	False	False
http_pipelines_blackhole	-14.66KiB	-0.88	100.00%	1.63MiB	20.95KiB	438.41B	0.0125168	1.62MiB	104.44KiB	2.13KiB	0.062956	False	False
http_pipelines_no_grok_blackhole	-144.19KiB	-1.25	100.00%	11.25MiB	49.43KiB	1.01KiB	0.00429099	11.11MiB	1.07MiB	22.36KiB	0.0966511	False	False
datadog_agent_remap_datadog_logs	-845.64KiB	-1.31	100.00%	62.82MiB	532.0KiB	10.89KiB	0.00826841	61.99MiB	4.06MiB	84.51KiB	0.0654331	False	False

Aug 08 '22 21:08 github-actions[bot]

Soak Test Results

Baseline: 3bab7aed9471ba8ace2a95a04178f26783433c85 Comparison: a2a1ddb670152ebc417dd749fa0c568ed6911969 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
syslog_log2metric_splunk_hec_metrics	625.67KiB	3.76	100.00%	16.23MiB	1010.77KiB	20.6KiB	0.0608035	16.84MiB	1.26MiB	26.16KiB	0.0745071	False	False
syslog_loki	455.12KiB	3.22	100.00%	13.82MiB	511.32KiB	10.47KiB	0.0361292	14.26MiB	835.18KiB	16.98KiB	0.0571745	False	False
http_text_to_http_json	706.23KiB	1.79	100.00%	38.64MiB	1.07MiB	22.28KiB	0.0275873	39.33MiB	1.1MiB	22.97KiB	0.0279279	False	False
syslog_splunk_hec_logs	247.9KiB	1.56	100.00%	15.5MiB	765.36KiB	15.57KiB	0.0481962	15.75MiB	732.24KiB	14.94KiB	0.0454014	False	False
datadog_agent_remap_blackhole_acks	706.32KiB	1.19	100.00%	57.99MiB	5.78MiB	120.29KiB	0.0995926	58.68MiB	5.24MiB	109.56KiB	0.0893071	False	False
socket_to_socket_blackhole	223.35KiB	0.93	100.00%	23.43MiB	325.68KiB	6.65KiB	0.0135718	23.65MiB	481.07KiB	9.82KiB	0.0198625	False	False
syslog_humio_logs	127.67KiB	0.78	100.00%	15.93MiB	683.83KiB	13.96KiB	0.0419217	16.05MiB	634.4KiB	12.99KiB	0.0385893	False	False
datadog_agent_remap_blackhole	446.73KiB	0.7	99.97%	62.14MiB	4.48MiB	93.54KiB	0.0721549	62.57MiB	3.79MiB	79.18KiB	0.0606264	False	False
splunk_hec_route_s3	72.39KiB	0.4	71.61%	17.55MiB	2.33MiB	48.42KiB	0.132443	17.62MiB	2.25MiB	47.09KiB	0.127643	False	False
http_to_http_acks	58.75KiB	0.33	20.25%	17.49MiB	8.01MiB	167.41KiB	0.457781	17.55MiB	7.49MiB	156.12KiB	0.426568	True	True
http_pipelines_blackhole_acks	3.08KiB	0.26	69.96%	1.15MiB	110.04KiB	2.24KiB	0.0932356	1.16MiB	96.05KiB	1.96KiB	0.0811735	False	False
splunk_hec_to_splunk_hec_logs_noack	5.5KiB	0.02	40.51%	23.83MiB	380.39KiB	7.76KiB	0.0155836	23.84MiB	334.38KiB	6.83KiB	0.0136957	False	False
enterprise_http_to_http	-799.47B	-0	8.70%	23.85MiB	244.55KiB	4.99KiB	0.010013	23.85MiB	249.85KiB	5.11KiB	0.0102303	False	False
splunk_hec_indexer_ack_blackhole	-5.57KiB	-0.02	17.19%	23.76MiB	873.17KiB	17.77KiB	0.0358851	23.75MiB	908.68KiB	18.49KiB	0.0373531	False	False
splunk_hec_to_splunk_hec_logs_acks	-9.51KiB	-0.04	30.66%	23.76MiB	819.69KiB	16.68KiB	0.0336828	23.75MiB	857.42KiB	17.44KiB	0.035247	False	False
file_to_blackhole	-81.74KiB	-0.08	63.85%	95.37MiB	2.97MiB	61.55KiB	0.0311273	95.29MiB	3.13MiB	65.06KiB	0.0328273	False	False
http_to_http_json	-45.22KiB	-0.19	99.91%	23.85MiB	342.37KiB	6.99KiB	0.0140177	23.8MiB	574.47KiB	11.72KiB	0.0235644	False	False
http_to_http_noack	-75.49KiB	-0.31	99.92%	23.84MiB	415.22KiB	8.49KiB	0.0170078	23.76MiB	1.0MiB	20.89KiB	0.0421383	False	False
syslog_regex_logs2metric_ddmetrics	-81.21KiB	-0.64	100.00%	12.32MiB	600.45KiB	12.24KiB	0.0475909	12.24MiB	583.96KiB	11.9KiB	0.0465838	False	False
http_pipelines_blackhole	-10.25KiB	-0.65	100.00%	1.54MiB	23.13KiB	484.27B	0.0147038	1.53MiB	101.2KiB	2.06KiB	0.0647573	False	False
fluent_elasticsearch	-673.02KiB	-0.83	100.00%	79.47MiB	53.29KiB	1.07KiB	0.000654703	78.82MiB	6.32MiB	129.64KiB	0.0801702	False	False
datadog_agent_remap_datadog_logs_acks	-589.27KiB	-0.9	100.00%	63.93MiB	3.13MiB	65.43KiB	0.0489571	63.35MiB	4.49MiB	93.45KiB	0.0708509	False	False
http_pipelines_no_grok_blackhole	-165.83KiB	-1.45	100.00%	11.17MiB	87.57KiB	1.79KiB	0.00765678	11.0MiB	1.12MiB	23.39KiB	0.102081	False	False
syslog_log2metric_humio_metrics	-194.83KiB	-1.48	100.00%	12.82MiB	761.45KiB	15.56KiB	0.0579801	12.63MiB	941.55KiB	19.16KiB	0.0727737	False	False
datadog_agent_remap_datadog_logs	-1.11MiB	-1.76	100.00%	63.45MiB	1.04MiB	21.71KiB	0.0163239	62.33MiB	4.35MiB	90.57KiB	0.0697762	False	False

Aug 08 '22 23:08 github-actions[bot]

Soak Test Results

Baseline: a47dc196134e1f19b4c712a38541437879b33a08 Comparison: 1e806aee634d15dc77a5b58d9336e4ccc0ea99e1 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
splunk_hec_route_s3	734.12KiB	4.12	100.00%	17.4MiB	2.5MiB	52.05KiB	0.143471	18.12MiB	2.44MiB	51.0KiB	0.13457	False	False
http_pipelines_blackhole_acks	41.74KiB	3.76	100.00%	1.08MiB	128.54KiB	2.62KiB	0.115757	1.12MiB	120.29KiB	2.45KiB	0.104405	False	False
syslog_loki	393.73KiB	2.76	100.00%	13.91MiB	555.98KiB	11.37KiB	0.0390187	14.3MiB	829.02KiB	16.85KiB	0.0566158	False	False
http_text_to_http_json	894.01KiB	2.29	100.00%	38.06MiB	829.51KiB	16.93KiB	0.0212818	38.93MiB	785.8KiB	16.04KiB	0.0197084	False	False
socket_to_socket_blackhole	509.59KiB	2.21	100.00%	22.53MiB	137.23KiB	2.8KiB	0.00594807	23.02MiB	143.94KiB	2.94KiB	0.00610385	False	False
datadog_agent_remap_datadog_logs_acks	707.99KiB	1.15	100.00%	59.94MiB	3.67MiB	76.79KiB	0.0612761	60.63MiB	4.79MiB	99.61KiB	0.0789128	False	False
syslog_log2metric_humio_metrics	146.26KiB	1.14	100.00%	12.56MiB	435.11KiB	8.88KiB	0.0338216	12.7MiB	645.84KiB	13.14KiB	0.0496382	False	False
datadog_agent_remap_datadog_logs	653.02KiB	1.02	100.00%	62.61MiB	1.07MiB	22.38KiB	0.0170586	63.25MiB	3.89MiB	81.12KiB	0.0615413	False	False
datadog_agent_remap_blackhole_acks	549.93KiB	0.85	100.00%	62.97MiB	4.38MiB	91.18KiB	0.069532	63.51MiB	3.28MiB	68.51KiB	0.0516191	False	False
datadog_agent_remap_blackhole	496.84KiB	0.76	100.00%	63.55MiB	4.19MiB	87.23KiB	0.0658639	64.03MiB	3.27MiB	68.25KiB	0.0510903	False	False
http_to_http_acks	14.55KiB	0.08	4.95%	17.16MiB	7.94MiB	165.91KiB	0.462416	17.18MiB	7.92MiB	165.38KiB	0.461023	True	True
splunk_hec_to_splunk_hec_logs_acks	18.7KiB	0.08	54.98%	23.75MiB	896.34KiB	18.22KiB	0.0368495	23.77MiB	824.13KiB	16.77KiB	0.0338547	False	False
splunk_hec_to_splunk_hec_logs_noack	16.61KiB	0.07	84.30%	23.82MiB	463.8KiB	9.47KiB	0.0190092	23.84MiB	339.2KiB	6.93KiB	0.0138929	False	False
splunk_hec_indexer_ack_blackhole	-1.72KiB	-0.01	5.09%	23.75MiB	926.16KiB	18.84KiB	0.0380799	23.75MiB	944.67KiB	19.22KiB	0.0388435	False	False
enterprise_http_to_http	-2.08KiB	-0.01	22.59%	23.85MiB	246.86KiB	5.04KiB	0.0101073	23.84MiB	254.62KiB	5.21KiB	0.0104256	False	False
file_to_blackhole	-38.39KiB	-0.04	33.40%	95.34MiB	2.77MiB	57.39KiB	0.0290318	95.31MiB	3.27MiB	67.94KiB	0.0342515	False	False
http_to_http_json	-25.21KiB	-0.1	95.98%	23.84MiB	354.39KiB	7.23KiB	0.0145116	23.82MiB	485.63KiB	9.92KiB	0.0199062	False	False
http_to_http_noack	-79.11KiB	-0.32	99.98%	23.85MiB	255.81KiB	5.23KiB	0.010474	23.77MiB	992.86KiB	20.23KiB	0.0407841	False	False
fluent_elasticsearch	-477.49KiB	-0.59	100.00%	79.47MiB	53.91KiB	1.09KiB	0.000662313	79.01MiB	4.73MiB	97.05KiB	0.0598333	False	False
http_pipelines_blackhole	-11.32KiB	-0.7	99.79%	1.57MiB	97.79KiB	2.0KiB	0.0607437	1.56MiB	151.76KiB	3.09KiB	0.0949329	False	False
syslog_humio_logs	-118.08KiB	-0.7	100.00%	16.55MiB	124.46KiB	2.54KiB	0.0073411	16.44MiB	139.8KiB	2.86KiB	0.00830353	False	False
syslog_regex_logs2metric_ddmetrics	-104.66KiB	-0.82	100.00%	12.42MiB	533.0KiB	10.86KiB	0.0419082	12.32MiB	518.11KiB	10.56KiB	0.0410758	False	False
syslog_log2metric_splunk_hec_metrics	-176.4KiB	-0.99	100.00%	17.45MiB	603.35KiB	12.3KiB	0.0337567	17.28MiB	833.08KiB	16.96KiB	0.0470748	False	False
syslog_splunk_hec_logs	-180.64KiB	-1.11	100.00%	15.9MiB	928.26KiB	18.91KiB	0.057014	15.72MiB	926.38KiB	18.89KiB	0.0575371	False	False
http_pipelines_no_grok_blackhole	-338.27KiB	-3.05	100.00%	10.84MiB	519.48KiB	10.61KiB	0.0467803	10.51MiB	1.15MiB	24.04KiB	0.109801	False	False

Aug 09 '22 21:08 github-actions[bot]

Soak Test Results

Baseline: a47dc196134e1f19b4c712a38541437879b33a08 Comparison: 074489d35bdad5ace2847d2a724d3073b4838b6c Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
http_text_to_http_json	1.16MiB	3.1	100.00%	37.45MiB	978.75KiB	19.97KiB	0.0255147	38.61MiB	1.0MiB	20.99KiB	0.0259907	False	False
socket_to_socket_blackhole	670.41KiB	2.9	100.00%	22.61MiB	128.34KiB	2.62KiB	0.00554149	23.27MiB	140.88KiB	2.88KiB	0.00591207	False	False
syslog_log2metric_humio_metrics	232.63KiB	1.9	100.00%	11.94MiB	692.4KiB	14.13KiB	0.0566213	12.17MiB	890.32KiB	18.12KiB	0.0714463	False	False
datadog_agent_remap_datadog_logs_acks	997.87KiB	1.58	100.00%	61.7MiB	3.18MiB	66.41KiB	0.0514835	62.68MiB	4.48MiB	93.35KiB	0.0715394	False	False
splunk_hec_route_s3	261.28KiB	1.37	99.99%	18.68MiB	2.32MiB	48.28KiB	0.124167	18.93MiB	2.14MiB	44.78KiB	0.113085	False	False
datadog_agent_remap_datadog_logs	762.66KiB	1.24	100.00%	60.2MiB	1.21MiB	25.31KiB	0.0200418	60.95MiB	4.17MiB	86.8KiB	0.0683768	False	False
http_pipelines_blackhole_acks	12.63KiB	1.05	100.00%	1.18MiB	109.39KiB	2.23KiB	0.0905386	1.19MiB	81.61KiB	1.66KiB	0.0668452	False	False
syslog_log2metric_splunk_hec_metrics	99.05KiB	0.57	99.97%	16.99MiB	945.47KiB	19.26KiB	0.0543397	17.08MiB	948.42KiB	19.3KiB	0.0542006	False	False
syslog_humio_logs	77.83KiB	0.46	100.00%	16.62MiB	144.5KiB	2.95KiB	0.00849043	16.69MiB	165.43KiB	3.39KiB	0.00967607	False	False
http_pipelines_blackhole	5.4KiB	0.32	99.41%	1.63MiB	10.52KiB	219.93B	0.00631256	1.63MiB	95.38KiB	1.94KiB	0.0570738	False	False
splunk_hec_to_splunk_hec_logs_noack	16.45KiB	0.07	83.56%	23.82MiB	471.74KiB	9.63KiB	0.0193348	23.84MiB	336.64KiB	6.88KiB	0.013788	False	False
splunk_hec_indexer_ack_blackhole	11.68KiB	0.05	34.38%	23.74MiB	932.04KiB	18.95KiB	0.0383346	23.75MiB	892.39KiB	18.15KiB	0.0366865	False	False
splunk_hec_to_splunk_hec_logs_acks	603.71B	0	1.83%	23.76MiB	860.07KiB	17.5KiB	0.0353429	23.76MiB	922.25KiB	18.76KiB	0.0378972	False	False
enterprise_http_to_http	974.41B	0	10.34%	23.85MiB	253.64KiB	5.18KiB	0.0103852	23.85MiB	252.82KiB	5.17KiB	0.0103513	False	False
file_to_blackhole	-57.87KiB	-0.06	44.53%	95.34MiB	3.08MiB	63.78KiB	0.0322641	95.28MiB	3.58MiB	74.34KiB	0.0375137	False	False
syslog_regex_logs2metric_ddmetrics	-14.57KiB	-0.11	64.00%	12.47MiB	513.39KiB	10.47KiB	0.0401932	12.46MiB	588.55KiB	11.99KiB	0.0461302	False	False
datadog_agent_remap_blackhole	-91.54KiB	-0.14	65.33%	64.05MiB	3.74MiB	78.01KiB	0.0583854	63.96MiB	2.78MiB	58.1KiB	0.0435079	False	False
http_to_http_json	-33.21KiB	-0.14	98.93%	23.84MiB	360.83KiB	7.37KiB	0.0147761	23.81MiB	524.71KiB	10.71KiB	0.0215161	False	False
syslog_loki	-36.24KiB	-0.26	93.04%	13.6MiB	586.38KiB	12.0KiB	0.0420818	13.57MiB	785.38KiB	15.97KiB	0.0565104	False	False
datadog_agent_remap_blackhole_acks	-204.02KiB	-0.31	94.71%	65.24MiB	4.43MiB	92.23KiB	0.0678761	65.04MiB	2.43MiB	50.93KiB	0.037384	False	False
fluent_elasticsearch	-309.69KiB	-0.38	100.00%	79.47MiB	53.61KiB	1.08KiB	0.000658574	79.17MiB	3.0MiB	61.57KiB	0.037859	False	False
http_to_http_noack	-111.35KiB	-0.46	100.00%	23.84MiB	416.07KiB	8.5KiB	0.0170417	23.73MiB	1.16MiB	24.15KiB	0.0488235	False	False
syslog_splunk_hec_logs	-201.8KiB	-1.22	100.00%	16.17MiB	915.15KiB	18.65KiB	0.0552608	15.97MiB	1.05MiB	21.96KiB	0.0658336	False	False
http_pipelines_no_grok_blackhole	-147.57KiB	-1.28	100.00%	11.24MiB	58.35KiB	1.19KiB	0.00506885	11.09MiB	1.1MiB	22.95KiB	0.0993388	False	False
http_to_http_acks	-276.05KiB	-1.53	76.86%	17.58MiB	7.5MiB	156.89KiB	0.426651	17.31MiB	8.1MiB	169.03KiB	0.467699	True	True

Aug 09 '22 23:08 github-actions[bot]

Soak Test Results

Baseline: d4f0afa8aad2b32026939df97cddce7272811d9c Comparison: 29f763a1014582a08bcd65540475f5eff3a7893f Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
http_to_http_acks	186.11KiB	1.06	58.60%	17.21MiB	7.74MiB	161.75KiB	0.44957	17.39MiB	7.67MiB	160.42KiB	0.441298	True	True
datadog_agent_remap_blackhole_acks	668.83KiB	1.04	100.00%	62.72MiB	4.56MiB	94.84KiB	0.0726138	63.37MiB	2.99MiB	62.53KiB	0.0471495	False	False
syslog_splunk_hec_logs	149.08KiB	0.88	100.00%	16.55MiB	637.94KiB	13.0KiB	0.0376335	16.7MiB	593.04KiB	12.1KiB	0.0346795	False	False
splunk_hec_route_s3	150.37KiB	0.82	96.32%	17.82MiB	2.47MiB	51.47KiB	0.138559	17.97MiB	2.41MiB	50.33KiB	0.133917	False	False
socket_to_socket_blackhole	184.94KiB	0.77	100.00%	23.34MiB	186.49KiB	3.81KiB	0.0078014	23.52MiB	169.05KiB	3.45KiB	0.00701743	False	False
syslog_regex_logs2metric_ddmetrics	91.7KiB	0.73	100.00%	12.26MiB	643.32KiB	13.1KiB	0.0512347	12.35MiB	634.12KiB	12.92KiB	0.0501356	False	False
syslog_humio_logs	117.72KiB	0.69	100.00%	16.69MiB	105.02KiB	2.14KiB	0.00614526	16.8MiB	107.49KiB	2.2KiB	0.00624655	False	False
datadog_agent_remap_blackhole	427.16KiB	0.69	99.39%	60.37MiB	5.59MiB	116.67KiB	0.0926454	60.79MiB	4.94MiB	103.06KiB	0.081224	False	False
syslog_log2metric_splunk_hec_metrics	63.54KiB	0.35	99.91%	17.86MiB	562.14KiB	11.46KiB	0.0307385	17.92MiB	747.87KiB	15.23KiB	0.0407524	False	False
http_pipelines_blackhole_acks	3.89KiB	0.31	85.26%	1.23MiB	114.62KiB	2.33KiB	0.0911243	1.23MiB	64.99KiB	1.33KiB	0.0515067	False	False
syslog_loki	37.82KiB	0.25	97.66%	14.81MiB	353.55KiB	7.23KiB	0.0233111	14.85MiB	739.23KiB	15.03KiB	0.0486193	False	False
syslog_log2metric_humio_metrics	24.66KiB	0.19	92.49%	12.77MiB	381.19KiB	7.78KiB	0.0291516	12.79MiB	563.13KiB	11.46KiB	0.0429846	False	False
splunk_hec_to_splunk_hec_logs_noack	10.2KiB	0.04	64.25%	23.83MiB	430.54KiB	8.79KiB	0.0176422	23.84MiB	330.7KiB	6.75KiB	0.0135454	False	False
enterprise_http_to_http	-551.2B	-0	5.93%	23.85MiB	250.93KiB	5.12KiB	0.0102741	23.85MiB	249.77KiB	5.11KiB	0.0102268	False	False
splunk_hec_indexer_ack_blackhole	-924.21B	-0	2.60%	23.74MiB	956.29KiB	19.45KiB	0.0393299	23.74MiB	966.91KiB	19.66KiB	0.0397682	False	False
splunk_hec_to_splunk_hec_logs_acks	-2.61KiB	-0.01	8.18%	23.75MiB	880.75KiB	17.91KiB	0.0362148	23.74MiB	888.37KiB	18.07KiB	0.0365321	False	False
file_to_blackhole	-41.38KiB	-0.04	31.22%	95.34MiB	3.23MiB	67.0KiB	0.0338916	95.3MiB	3.76MiB	78.22KiB	0.0394404	False	False
http_to_http_json	-30.12KiB	-0.12	98.65%	23.85MiB	332.87KiB	6.8KiB	0.013629	23.82MiB	494.73KiB	10.11KiB	0.020281	False	False
http_to_http_noack	-113.23KiB	-0.46	100.00%	23.85MiB	245.86KiB	5.03KiB	0.0100669	23.73MiB	1.15MiB	24.0KiB	0.0485017	False	False
fluent_elasticsearch	-409.62KiB	-0.5	100.00%	79.47MiB	53.08KiB	1.07KiB	0.000652167	79.07MiB	4.3MiB	88.44KiB	0.0544174	False	False
http_pipelines_blackhole	-9.59KiB	-0.55	99.99%	1.7MiB	46.83KiB	979.96B	0.026947	1.69MiB	109.03KiB	2.22KiB	0.063083	False	False
datadog_agent_remap_datadog_logs_acks	-560.32KiB	-0.87	100.00%	62.92MiB	3.11MiB	65.04KiB	0.049445	62.37MiB	4.32MiB	89.96KiB	0.069261	False	False
http_pipelines_no_grok_blackhole	-100.99KiB	-0.91	100.00%	10.84MiB	198.28KiB	4.05KiB	0.0178607	10.74MiB	1.03MiB	21.36KiB	0.095465	False	False
datadog_agent_remap_datadog_logs	-785.68KiB	-1.22	100.00%	62.9MiB	415.95KiB	8.52KiB	0.0064567	62.13MiB	4.3MiB	89.55KiB	0.0692116	False	False
http_text_to_http_json	-989.91KiB	-2.49	100.00%	38.84MiB	944.64KiB	19.28KiB	0.0237464	37.87MiB	966.55KiB	19.73KiB	0.0249174	False	False

Aug 16 '22 21:08 github-actions[bot]

Soak Test Results

Baseline: 8e2201bcfd7b7ec460d783ed8b8489cafef55902 Comparison: 1193263ece1ceae83a62d613a07856568f2b6518 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
syslog_regex_logs2metric_ddmetrics	280.38KiB	2.31	100.00%	11.85MiB	535.18KiB	10.91KiB	0.0440826	12.13MiB	527.04KiB	10.75KiB	0.0424327	False	False
socket_to_socket_blackhole	318.36KiB	1.4	100.00%	22.2MiB	293.73KiB	6.0KiB	0.0129196	22.51MiB	239.97KiB	4.9KiB	0.0104092	False	False
datadog_agent_remap_blackhole	769.57KiB	1.22	100.00%	61.49MiB	4.14MiB	86.38KiB	0.0673914	62.24MiB	3.27MiB	68.16KiB	0.0524813	False	False
syslog_splunk_hec_logs	181.92KiB	1.09	100.00%	16.27MiB	704.8KiB	14.36KiB	0.0422974	16.45MiB	450.7KiB	9.2KiB	0.0267557	False	False
syslog_log2metric_splunk_hec_metrics	176.8KiB	1	100.00%	17.29MiB	1.07MiB	22.27KiB	0.0616926	17.46MiB	1019.49KiB	20.78KiB	0.0570013	False	False
syslog_humio_logs	117.2KiB	0.71	100.00%	16.08MiB	454.4KiB	9.28KiB	0.027586	16.2MiB	516.68KiB	10.57KiB	0.0311457	False	False
http_pipelines_blackhole_acks	7.31KiB	0.59	99.16%	1.21MiB	109.85KiB	2.24KiB	0.0885494	1.22MiB	80.33KiB	1.64KiB	0.0643701	False	False
syslog_log2metric_humio_metrics	33.01KiB	0.26	82.04%	12.36MiB	791.82KiB	16.16KiB	0.0625585	12.39MiB	910.08KiB	18.53KiB	0.0717144	False	False
splunk_hec_to_splunk_hec_logs_noack	25.42KiB	0.1	95.50%	23.81MiB	518.82KiB	10.58KiB	0.0212714	23.84MiB	341.93KiB	6.98KiB	0.0140044	False	False
http_pipelines_no_grok_blackhole	8.23KiB	0.08	23.40%	10.65MiB	586.47KiB	11.97KiB	0.0537483	10.66MiB	1.2MiB	24.91KiB	0.112237	False	False
enterprise_http_to_http	-1.09KiB	-0	12.08%	23.85MiB	246.33KiB	5.03KiB	0.0100856	23.85MiB	251.81KiB	5.15KiB	0.0103104	False	False
http_to_http_acks	-262.47B	-0	0.09%	17.17MiB	7.97MiB	166.66KiB	0.464213	17.17MiB	7.94MiB	165.84KiB	0.462113	True	True
splunk_hec_to_splunk_hec_logs_acks	-4.87KiB	-0.02	15.87%	23.76MiB	839.6KiB	17.08KiB	0.0345069	23.75MiB	851.69KiB	17.32KiB	0.035011	False	False
splunk_hec_indexer_ack_blackhole	-6.88KiB	-0.03	22.24%	23.76MiB	829.35KiB	16.87KiB	0.0340817	23.75MiB	864.97KiB	17.59KiB	0.0355555	False	False
file_to_blackhole	-58.21KiB	-0.06	59.77%	95.37MiB	2.29MiB	47.42KiB	0.023979	95.31MiB	2.45MiB	50.8KiB	0.0256474	False	False
http_to_http_json	-36.11KiB	-0.15	99.48%	23.84MiB	340.67KiB	6.96KiB	0.0139503	23.81MiB	532.81KiB	10.87KiB	0.0218509	False	False
fluent_elasticsearch	-140.71KiB	-0.17	100.00%	79.47MiB	54.84KiB	1.11KiB	0.000673725	79.34MiB	1.32MiB	27.24KiB	0.0166737	False	False
http_to_http_noack	-72.62KiB	-0.3	99.61%	23.82MiB	601.71KiB	12.3KiB	0.0246637	23.75MiB	1.05MiB	21.96KiB	0.0443431	False	False
datadog_agent_remap_blackhole_acks	-275.77KiB	-0.44	97.88%	61.86MiB	4.72MiB	98.37KiB	0.0763619	61.59MiB	3.26MiB	68.08KiB	0.0528498	False	False
splunk_hec_route_s3	-150.9KiB	-0.79	97.73%	18.7MiB	2.28MiB	47.51KiB	0.12195	18.55MiB	2.21MiB	46.13KiB	0.118994	False	False
syslog_loki	-168.52KiB	-1.18	100.00%	13.99MiB	893.8KiB	18.28KiB	0.0623786	13.83MiB	1013.83KiB	20.61KiB	0.0715982	False	False
datadog_agent_remap_datadog_logs_acks	-981.48KiB	-1.51	100.00%	63.29MiB	3.9MiB	81.31KiB	0.0615806	62.33MiB	4.43MiB	92.21KiB	0.0710536	False	False
datadog_agent_remap_datadog_logs	-1.04MiB	-1.63	100.00%	63.38MiB	727.17KiB	14.88KiB	0.0112023	62.34MiB	4.06MiB	84.59KiB	0.0651313	False	False
http_pipelines_blackhole	-32.59KiB	-1.94	100.00%	1.64MiB	66.57KiB	1.36KiB	0.0395844	1.61MiB	136.08KiB	2.77KiB	0.0825109	False	False
http_text_to_http_json	-1.33MiB	-3.36	100.00%	39.52MiB	1.05MiB	21.88KiB	0.0264914	38.19MiB	1.08MiB	22.62KiB	0.0283259	False	False

Aug 22 '22 18:08 github-actions[bot]

Soak Test Results

Baseline: 8426cb2ade475df17e42a1d87e563c335ef3c7fb Comparison: 75bec101cfa5f5d6d79236ff30f0037698699410 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
datadog_agent_remap_blackhole_acks	1.01MiB	1.6	100.00%	63.3MiB	4.28MiB	89.04KiB	0.0675335	64.32MiB	2.93MiB	61.3KiB	0.0455706	False	False
syslog_regex_logs2metric_ddmetrics	159.41KiB	1.27	100.00%	12.21MiB	589.02KiB	12.0KiB	0.0470862	12.37MiB	589.82KiB	12.02KiB	0.0465568	False	False
datadog_agent_remap_blackhole	756.71KiB	1.18	100.00%	62.52MiB	4.1MiB	85.5KiB	0.0656175	63.26MiB	2.7MiB	56.33KiB	0.0426533	False	False
syslog_log2metric_splunk_hec_metrics	197.19KiB	1.12	100.00%	17.21MiB	1.12MiB	23.3KiB	0.064878	17.4MiB	1.14MiB	23.71KiB	0.0653761	False	False
http_pipelines_blackhole_acks	13.02KiB	1.08	99.99%	1.18MiB	138.93KiB	2.83KiB	0.115315	1.19MiB	85.92KiB	1.75KiB	0.0705506	False	False
syslog_log2metric_humio_metrics	96.82KiB	0.73	100.00%	12.98MiB	188.88KiB	3.86KiB	0.0142055	13.08MiB	473.37KiB	9.64KiB	0.0353433	False	False
splunk_hec_route_s3	124.11KiB	0.66	93.09%	18.23MiB	2.35MiB	48.86KiB	0.128755	18.35MiB	2.28MiB	47.66KiB	0.124094	False	False
http_to_http_acks	95.99KiB	0.54	31.53%	17.23MiB	8.05MiB	168.27KiB	0.467005	17.32MiB	7.97MiB	166.0KiB	0.45967	True	True
datadog_agent_remap_datadog_logs_acks	159.53KiB	0.25	84.94%	61.97MiB	2.98MiB	62.41KiB	0.0481561	62.12MiB	4.41MiB	91.74KiB	0.0709284	False	False
syslog_splunk_hec_logs	29.85KiB	0.18	87.70%	16.5MiB	740.13KiB	15.06KiB	0.0437853	16.53MiB	595.51KiB	12.15KiB	0.0351676	False	False
datadog_agent_remap_datadog_logs	92.44KiB	0.15	77.23%	61.29MiB	865.3KiB	17.71KiB	0.0137834	61.38MiB	3.58MiB	74.53KiB	0.0582346	False	False
splunk_hec_indexer_ack_blackhole	22.75KiB	0.09	62.82%	23.75MiB	928.36KiB	18.88KiB	0.03817	23.77MiB	839.99KiB	17.1KiB	0.0345043	False	False
splunk_hec_to_splunk_hec_logs_acks	8.57KiB	0.04	29.68%	23.76MiB	794.16KiB	16.16KiB	0.0326276	23.77MiB	769.64KiB	15.67KiB	0.031609	False	False
splunk_hec_to_splunk_hec_logs_noack	965.22B	0	7.71%	23.84MiB	336.37KiB	6.87KiB	0.0137782	23.84MiB	337.73KiB	6.9KiB	0.0138335	False	False
enterprise_http_to_http	-712.28B	-0	7.78%	23.85MiB	247.91KiB	5.06KiB	0.0101503	23.85MiB	244.97KiB	5.01KiB	0.01003	False	False
file_to_blackhole	-38.25KiB	-0.04	28.00%	95.34MiB	3.4MiB	70.43KiB	0.0356268	95.3MiB	3.85MiB	80.16KiB	0.0404058	False	False
syslog_humio_logs	-12.75KiB	-0.07	99.93%	16.64MiB	122.34KiB	2.5KiB	0.00717907	16.63MiB	138.32KiB	2.83KiB	0.00812265	False	False
fluent_elasticsearch	-124.75KiB	-0.15	99.95%	79.47MiB	52.78KiB	1.07KiB	0.000648406	79.35MiB	1.74MiB	35.89KiB	0.0219807	False	False
http_to_http_json	-39.79KiB	-0.16	99.76%	23.84MiB	344.47KiB	7.03KiB	0.014106	23.8MiB	542.15KiB	11.06KiB	0.0222372	False	False
http_to_http_noack	-95.17KiB	-0.39	99.99%	23.84MiB	407.65KiB	8.33KiB	0.0166971	23.74MiB	1.1MiB	22.96KiB	0.0463705	False	False
http_pipelines_blackhole	-8.64KiB	-0.52	99.96%	1.62MiB	39.72KiB	831.46B	0.0239571	1.61MiB	112.53KiB	2.29KiB	0.0682273	False	False
http_pipelines_no_grok_blackhole	-94.61KiB	-0.85	100.00%	10.81MiB	202.3KiB	4.13KiB	0.0182659	10.72MiB	1016.58KiB	20.68KiB	0.0925793	False	False
http_text_to_http_json	-335.6KiB	-0.85	100.00%	38.62MiB	1.35MiB	28.18KiB	0.0348975	38.29MiB	1.15MiB	24.03KiB	0.0300164	False	False
syslog_loki	-166.76KiB	-1.18	100.00%	13.85MiB	304.47KiB	6.24KiB	0.0214582	13.69MiB	675.33KiB	13.73KiB	0.0481619	False	False
socket_to_socket_blackhole	-1.36MiB	-5.68	100.00%	23.84MiB	282.99KiB	5.78KiB	0.0115902	22.48MiB	761.07KiB	15.54KiB	0.0330493	False	False

Aug 24 '22 22:08 github-actions[bot]

Soak Test Results

Baseline: 6572842f77efb10317de10babc056416d93f7b82 Comparison: db122fd6f8ec6aa6f411bcec5e61cc5f50b4f82b Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
http_to_http_acks	290.71KiB	1.64	81.30%	17.28MiB	7.37MiB	154.02KiB	0.426022	17.57MiB	7.55MiB	157.51KiB	0.429885	True	True
syslog_regex_logs2metric_ddmetrics	111.64KiB	0.89	100.00%	12.31MiB	660.09KiB	13.43KiB	0.0523628	12.42MiB	498.54KiB	10.17KiB	0.0392005	False	False
http_pipelines_blackhole_acks	8.14KiB	0.67	99.15%	1.19MiB	123.1KiB	2.5KiB	0.101108	1.2MiB	89.32KiB	1.82KiB	0.0728758	False	False
splunk_hec_route_s3	68.55KiB	0.37	67.39%	18.09MiB	2.41MiB	50.07KiB	0.132944	18.16MiB	2.33MiB	48.65KiB	0.128063	False	False
syslog_log2metric_humio_metrics	37.01KiB	0.28	99.94%	12.88MiB	274.89KiB	5.61KiB	0.0208315	12.92MiB	452.26KiB	9.21KiB	0.0341775	False	False
splunk_hec_to_splunk_hec_logs_noack	18.95KiB	0.08	87.62%	23.82MiB	502.65KiB	10.25KiB	0.0206039	23.84MiB	333.45KiB	6.81KiB	0.0136574	False	False
splunk_hec_to_splunk_hec_logs_acks	11.51KiB	0.05	37.28%	23.75MiB	846.09KiB	17.21KiB	0.0347786	23.76MiB	800.08KiB	16.28KiB	0.0328721	False	False
splunk_hec_indexer_ack_blackhole	12.51KiB	0.05	38.50%	23.76MiB	889.58KiB	18.1KiB	0.0365589	23.77MiB	838.14KiB	17.06KiB	0.0344273	False	False
syslog_log2metric_splunk_hec_metrics	1.41KiB	0.01	6.51%	17.79MiB	455.29KiB	9.28KiB	0.0249835	17.79MiB	711.91KiB	14.49KiB	0.0390622	False	False
enterprise_http_to_http	-3.25KiB	-0.01	33.66%	23.85MiB	260.41KiB	5.32KiB	0.0106615	23.84MiB	256.7KiB	5.25KiB	0.010511	False	False
http_pipelines_no_grok_blackhole	-2.08KiB	-0.02	7.03%	10.96MiB	174.16KiB	3.55KiB	0.0155179	10.96MiB	1.12MiB	23.24KiB	0.101895	False	False
file_to_blackhole	-59.96KiB	-0.06	47.56%	95.36MiB	3.0MiB	62.11KiB	0.0314123	95.3MiB	3.4MiB	70.81KiB	0.0357085	False	False
syslog_humio_logs	-21.21KiB	-0.13	99.89%	16.39MiB	223.18KiB	4.56KiB	0.0132979	16.37MiB	224.57KiB	4.6KiB	0.0133976	False	False
http_to_http_json	-32.31KiB	-0.13	98.95%	23.84MiB	346.25KiB	7.07KiB	0.014179	23.81MiB	512.03KiB	10.45KiB	0.0209955	False	False
datadog_agent_remap_blackhole_acks	-125.5KiB	-0.19	76.76%	63.72MiB	4.23MiB	88.02KiB	0.0663088	63.59MiB	2.75MiB	57.4KiB	0.043163	False	False
syslog_splunk_hec_logs	-32.97KiB	-0.2	94.13%	16.41MiB	627.62KiB	12.79KiB	0.0373524	16.37MiB	580.89KiB	11.85KiB	0.0346392	False	False
fluent_elasticsearch	-201.54KiB	-0.25	100.00%	79.47MiB	52.69KiB	1.07KiB	0.000647305	79.28MiB	1.68MiB	34.57KiB	0.0211981	False	False
http_to_http_noack	-87.21KiB	-0.36	99.98%	23.84MiB	407.45KiB	8.33KiB	0.0166891	23.75MiB	1.06MiB	22.15KiB	0.0447141	False	False
http_pipelines_blackhole	-14.87KiB	-0.88	100.00%	1.64MiB	29.15KiB	609.99B	0.0173203	1.63MiB	112.43KiB	2.29KiB	0.0674056	False	False
http_text_to_http_json	-799.68KiB	-2	100.00%	39.0MiB	831.75KiB	16.98KiB	0.0208226	38.22MiB	886.11KiB	18.1KiB	0.0226366	False	False
datadog_agent_remap_blackhole	-1.29MiB	-2.2	100.00%	58.44MiB	6.83MiB	142.35KiB	0.116922	57.15MiB	6.64MiB	138.49KiB	0.116122	False	False
datadog_agent_remap_datadog_logs_acks	-1.73MiB	-2.74	100.00%	63.3MiB	2.27MiB	47.45KiB	0.0357794	61.57MiB	4.34MiB	90.44KiB	0.0705541	False	False
datadog_agent_remap_datadog_logs	-1.89MiB	-3.06	100.00%	61.85MiB	1.66MiB	34.73KiB	0.0267914	59.96MiB	4.39MiB	91.4KiB	0.0731887	False	False
socket_to_socket_blackhole	-851.88KiB	-3.46	100.00%	24.06MiB	279.2KiB	5.7KiB	0.011332	23.22MiB	150.05KiB	3.06KiB	0.00630854	False	False
syslog_loki	-666.48KiB	-4.47	100.00%	14.56MiB	419.61KiB	8.59KiB	0.0281303	13.91MiB	728.6KiB	14.81KiB	0.0511301	False	False

Aug 25 '22 21:08 github-actions[bot]

Hey @hhromic , just checking if you have any further thoughts on this before it gets merged in!

Aug 26 '22 14:08 neuronull

Hey @hhromic , just checking if you have any further thoughts on this before it gets merged in!

Hi @neuronull, appreciate the consideration! From my side I think there are no more comments at this time! When this feature gets released for sure we will test it more with real-world use cases and will probably be able to give more feedback if necessary!

Thanks for this fantastic addition!

Aug 26 '22 14:08 hhromic

Soak Test Results

Baseline: a00a3f914a7fc75bb57373514d4b3c7f7f7325af Comparison: 33c236419b6c65f23d143556aca131cf7811e165 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison CoV	erratic	declared erratic
datadog_agent_remap_blackhole	831.06KiB	1.33	100.00%	61.16MiB	4.45MiB	92.6KiB	0.072701	61.97MiB	3.68MiB	76.8KiB	0.0594234	False	False
syslog_splunk_hec_logs	143.17KiB	0.87	100.00%	15.99MiB	1004.99KiB	20.46KiB	0.0613738	16.13MiB	938.84KiB	19.12KiB	0.0568374	False	False
http_pipelines_blackhole_acks	6.49KiB	0.55	98.91%	1.15MiB	102.53KiB	2.09KiB	0.0872622	1.15MiB	71.68KiB	1.46KiB	0.06067	False	False
datadog_agent_remap_datadog_logs_acks	165.63KiB	0.27	78.41%	60.69MiB	4.19MiB	87.51KiB	0.0690046	60.85MiB	4.86MiB	101.26KiB	0.0799263	False	False
datadog_agent_remap_blackhole_acks	156.63KiB	0.25	74.06%	62.39MiB	4.88MiB	101.58KiB	0.078149	62.55MiB	4.53MiB	94.68KiB	0.0724523	False	False
syslog_log2metric_humio_metrics	28.3KiB	0.22	95.54%	12.84MiB	341.27KiB	6.97KiB	0.0259599	12.86MiB	601.4KiB	12.24KiB	0.0456498	False	False
splunk_hec_to_splunk_hec_logs_noack	24.41KiB	0.1	94.94%	23.82MiB	510.66KiB	10.42KiB	0.0209348	23.84MiB	336.66KiB	6.87KiB	0.0137879	False	False
splunk_hec_indexer_ack_blackhole	18.36KiB	0.08	53.34%	23.75MiB	915.13KiB	18.61KiB	0.0376252	23.77MiB	836.27KiB	17.02KiB	0.034357	False	False
splunk_hec_to_splunk_hec_logs_acks	6.32KiB	0.03	21.27%	23.76MiB	811.85KiB	16.52KiB	0.0333628	23.77MiB	814.72KiB	16.58KiB	0.0334718	False	False
enterprise_http_to_http	-3.47KiB	-0.01	35.92%	23.85MiB	258.56KiB	5.28KiB	0.0105862	23.84MiB	256.15KiB	5.25KiB	0.0104889	False	False
splunk_hec_route_s3	-6.81KiB	-0.03	8.22%	19.02MiB	2.3MiB	47.97KiB	0.121093	19.02MiB	2.17MiB	45.3KiB	0.113909	False	False
file_to_blackhole	-48.13KiB	-0.05	38.49%	95.34MiB	3.06MiB	63.39KiB	0.0320666	95.29MiB	3.45MiB	71.72KiB	0.0361808	False	False
http_to_http_json	-30.55KiB	-0.13	98.80%	23.85MiB	332.11KiB	6.78KiB	0.0135978	23.82MiB	493.76KiB	10.09KiB	0.0202414	False	False
http_to_http_noack	-67.48KiB	-0.28	99.84%	23.84MiB	411.42KiB	8.42KiB	0.016852	23.77MiB	966.6KiB	19.69KiB	0.0397025	False	False
fluent_elasticsearch	-450.61KiB	-0.55	100.00%	79.47MiB	53.85KiB	1.09KiB	0.000661587	79.03MiB	4.45MiB	91.29KiB	0.0562315	False	False
http_pipelines_blackhole	-11.18KiB	-0.65	100.00%	1.68MiB	10.95KiB	229.27B	0.00636283	1.67MiB	117.17KiB	2.39KiB	0.0685443	False	False
syslog_regex_logs2metric_ddmetrics	-107.62KiB	-0.9	100.00%	11.63MiB	739.82KiB	15.06KiB	0.0621225	11.52MiB	713.17KiB	14.54KiB	0.0604312	False	False
http_pipelines_no_grok_blackhole	-109.9KiB	-0.97	100.00%	11.11MiB	102.69KiB	2.1KiB	0.00902142	11.01MiB	1.03MiB	21.54KiB	0.0939479	False	False
datadog_agent_remap_datadog_logs	-934.7KiB	-1.44	100.00%	63.53MiB	937.47KiB	19.2KiB	0.0144065	62.62MiB	4.42MiB	92.01KiB	0.0705596	False	False
syslog_log2metric_splunk_hec_metrics	-262.51KiB	-1.46	100.00%	17.57MiB	787.08KiB	16.04KiB	0.0437285	17.32MiB	987.55KiB	20.1KiB	0.0556781	False	False
syslog_humio_logs	-266.33KiB	-1.57	100.00%	16.58MiB	680.87KiB	13.9KiB	0.0401066	16.32MiB	910.89KiB	18.65KiB	0.0545111	False	False
http_to_http_acks	-337.91KiB	-1.89	87.05%	17.42MiB	7.32MiB	153.0KiB	0.419937	17.09MiB	7.76MiB	162.0KiB	0.453981	True	True
syslog_loki	-409.11KiB	-2.74	100.00%	14.56MiB	513.06KiB	10.51KiB	0.0343951	14.16MiB	845.19KiB	17.18KiB	0.0582589	False	False
http_text_to_http_json	-1.43MiB	-3.52	100.00%	40.53MiB	675.88KiB	13.8KiB	0.0162835	39.1MiB	791.98KiB	16.18KiB	0.0197778	False	False
socket_to_socket_blackhole	-945.64KiB	-3.91	100.00%	23.62MiB	254.86KiB	5.2KiB	0.0105335	22.7MiB	347.02KiB	7.09KiB	0.0149263	False	False

Aug 26 '22 16:08 github-actions[bot]

vector
vector copied to clipboard

feat(new source): A generic `http_scrape` source

Deploy Preview for vector-project ready!

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

vector vector copied to clipboard

feat(new source): A generic `http_scrape` source

✅ Deploy Preview for vector-project ready!

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

Soak Test Results

vector
vector copied to clipboard

Deploy Preview for vector-project ready!