feat(loki sink): Support snappy compressed proto

[xdatcloud]

trafficstars

Currently the default behavior of loki endpoint /loki/api/v1/push is for the POST body to be a snappy-compressed protobuf message, but sink loki doesn't support protobuf encoding and snappy compression, this PR introduces a new option snappy of existing field compression to enable that feature.

[bits-bot]

All committers have signed the CLA.

[netlify[bot]]

✅ Deploy Preview for vector-project ready!

Name	Link
🔨 Latest commit	d33fd3a8db595184e3e4e96f7733de3dd2a098d7
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/633b713124d9b4000831afa6
😎 Deploy Preview	https://deploy-preview-12927--vector-project.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[spencergilbert]

Hi @xdatcloud thanks for the contribution! I'm wondering if there's any advantage to support both JSON and Proto, and if we should just wholly switch to Proto with optional compression.

What are your thoughts?

[xdatcloud]

Hi @xdatcloud thanks for the contribution! I'm wondering if there's any advantage to support both JSON and Proto, and if we should just wholly switch to Proto with optional compression.

What are your thoughts?

Hi @spencergilbert , according to the codes of Push API, Loki ask for snappy-compressed data by default if content-type header is application/x-protobuf.

The Proto with snappy-compression performs much better than JSON with gzip-compression for speed, but it generate bigger payload in general.

I would like to use Proto with snappy-compression entirely in my production environment, but also need advice from others here.

[spencergilbert]

Hi @xdatcloud thanks for the contribution! I'm wondering if there's any advantage to support both JSON and Proto, and if we should just wholly switch to Proto with optional compression. What are your thoughts?

Hi @spencergilbert , according to the codes of Push API, Loki ask for snappy-compressed data by default if content-type header is application/x-protobuf.

The Proto with snappy-compression performs much better than JSON with gzip-compression for speed, but it generate bigger payload in general.

I would like to use Proto with snappy-compression entirely in my production environment, but also need advice from others here.

Ah, good background info - we didn't realize it had to be snappy compressed for protobuf.

@jszwedko what do you think here, we were thinking we'd be able to allow users to choose CPU vs size but that doesn't appear to be the case. Do we want to force compression always, or maintain both JSON and proto to allow picking and choosing?

[jszwedko]

Ah, good question. I'd say, let's try making snappy+proto the default, since it is Loki's preferred ingestion protocol, and see if the Loki soak tests show any difference (which will be reported on the PR).

[xdatcloud]

Hi @spencergilbert, sorry but I'm not familiar with the process to make a contribution since this is my first PR to Vector. Is there anything I can do to help?

[jszwedko]

Hi @spencergilbert, sorry but I'm not familiar with the process to make a contribution since this is my first PR to Vector. Is there anything I can do to help?

Hi @xdatcloud ! Thanks for the bump. We are curious to see what the performance tests show (which run automatically) if we switch to just using protobuf/snappy. I pushed a commit to make that the default. We can see what the soak tests say and decide whether we need / want to support both encodings.

[github-actions[bot]]

Soak Test Results

Baseline: f8bc670c1f9830f8f65a3c433539541453fa29a6 Comparison: ecab4fbbf063bd961c30d9ba8dc9e8e38e58a398 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
http_pipelines_no_grok_blackhole	208.83KiB	1.11	100.00%	18.36MiB	1.52MiB	31.72KiB	0	0.0825075	18.56MiB	1.64MiB	34.23KiB	0	0.0883533	False	False
syslog_log2metric_splunk_hec_metrics	74.15KiB	0.4	99.95%	18.11MiB	833.1KiB	16.96KiB	0	0.0449076	18.19MiB	641.52KiB	13.08KiB	0	0.0344432	False	False
http_to_http_noack	83.12KiB	0.34	99.99%	23.77MiB	993.51KiB	20.24KiB	0	0.0408166	23.85MiB	242.52KiB	4.95KiB	0	0.00992969	False	False
http_pipelines_blackhole_acks	11.73KiB	0.25	55.04%	4.57MiB	534.75KiB	10.93KiB	0	0.114274	4.58MiB	538.81KiB	11.01KiB	0	0.114855	False	False
syslog_regex_logs2metric_ddmetrics	28.58KiB	0.22	85.42%	12.68MiB	695.91KiB	14.16KiB	0	0.0535679	12.71MiB	668.85KiB	13.61KiB	0	0.051372	False	False
syslog_humio_logs	11.25KiB	0.06	98.24%	17.09MiB	162.92KiB	3.34KiB	0	0.00930509	17.11MiB	164.27KiB	3.36KiB	0	0.00937589	False	False
fluent_elasticsearch	36.64KiB	0.05	83.05%	79.44MiB	1.3MiB	26.64KiB	0	0.0163107	79.47MiB	54.28KiB	1.09KiB	0	0.00066688	False	False
syslog_splunk_hec_logs	6.42KiB	0.04	45.54%	16.99MiB	363.9KiB	7.43KiB	0	0.0209184	16.99MiB	370.13KiB	7.56KiB	0	0.0212686	False	False
socket_to_socket_blackhole	4.74KiB	0.03	24.64%	13.93MiB	585.74KiB	11.92KiB	0	0.0410438	13.94MiB	454.49KiB	9.27KiB	0	0.0318362	False	False
splunk_hec_to_splunk_hec_logs_acks	74.44B	0	0.26%	23.77MiB	783.68KiB	15.94KiB	0	0.0321878	23.77MiB	786.43KiB	16.0KiB	0	0.0323009	False	False
splunk_hec_to_splunk_hec_logs_noack	-367.92B	-0	2.97%	23.84MiB	333.28KiB	6.82KiB	0	0.0136519	23.84MiB	333.2KiB	6.82KiB	0	0.0136487	False	False
http_to_http_json	879.12B	0	6.64%	23.84MiB	359.62KiB	7.35KiB	0	0.0147268	23.84MiB	354.04KiB	7.23KiB	0	0.0144979	False	False
file_to_blackhole	-14.61KiB	-0.01	15.69%	95.24MiB	2.81MiB	58.51KiB	0	0.0295475	95.23MiB	2.18MiB	45.05KiB	0	0.022876	False	False
splunk_hec_indexer_ack_blackhole	-14.96KiB	-0.06	54.05%	23.79MiB	669.88KiB	13.64KiB	0	0.0274968	23.77MiB	733.12KiB	14.92KiB	0	0.030111	False	False
datadog_agent_remap_blackhole	-72.95KiB	-0.11	87.51%	62.28MiB	1.75MiB	36.67KiB	0	0.0280914	62.21MiB	1.44MiB	30.24KiB	0	0.0231929	False	False
http_pipelines_blackhole	-5.66KiB	-0.13	25.90%	4.4MiB	581.02KiB	11.91KiB	0	0.128838	4.4MiB	603.83KiB	12.32KiB	0	0.134065	False	False
datadog_agent_remap_datadog_logs	-123.65KiB	-0.17	75.50%	69.94MiB	3.54MiB	73.79KiB	0	0.0505336	69.82MiB	3.66MiB	76.58KiB	0	0.0523919	False	False
splunk_transforms_splunk3	-23.72KiB	-0.18	28.34%	12.54MiB	2.22MiB	46.36KiB	0	0.177016	12.52MiB	2.2MiB	46.04KiB	0	0.175664	False	False
datadog_agent_remap_datadog_logs_acks	-200.67KiB	-0.28	99.38%	71.01MiB	972.9KiB	19.88KiB	0	0.0133776	70.81MiB	3.38MiB	70.56KiB	0	0.0476716	False	False
splunk_hec_route_s3	-154.28KiB	-0.75	98.74%	20.21MiB	2.11MiB	44.06KiB	0	0.104279	20.06MiB	2.07MiB	43.32KiB	0	0.103284	False	False
http_to_http_acks	-169.22KiB	-0.92	53.70%	18.0MiB	7.71MiB	161.09KiB	0	0.428025	17.83MiB	7.89MiB	164.94KiB	0	0.442407	True	True
syslog_log2metric_humio_metrics	-203.1KiB	-1.37	100.00%	14.48MiB	438.62KiB	8.95KiB	0	0.0295836	14.28MiB	331.8KiB	6.79KiB	0	0.0226901	False	False
syslog_loki	-208.22KiB	-1.38	100.00%	14.77MiB	692.03KiB	14.1KiB	0	0.0457318	14.57MiB	524.65KiB	10.71KiB	0	0.0351542	False	False
datadog_agent_remap_blackhole_acks	-1019.41KiB	-1.47	100.00%	67.86MiB	3.84MiB	80.2KiB	0	0.056624	66.86MiB	3.23MiB	67.42KiB	0	0.0482285	False	False

[jszwedko]

Interestingly the performance was the same. I'd be content to just making this the one-and-only encoding and dropping the JSON encoder. What do you think @xdatcloud ?

[xdatcloud]

Interestingly the performance was the same. I'd be content to just making this the one-and-only encoding and dropping the JSON encoder. What do you think @xdatcloud ?

Hi @jszwedko, thanks for the job! It looks like users would reach higher throughput without extra resources. We can use proto/snappy as the one-and-only encoding, but it should be announced due to breaking changes to existing configs since users may have already used gzip compression.

[jszwedko]

Interestingly the performance was the same. I'd be content to just making this the one-and-only encoding and dropping the JSON encoder. What do you think @xdatcloud ?

Hi @jszwedko, thanks for the job! It looks like users would reach higher throughput without extra resources. We can use proto/snappy as the one-and-only encoding, but it should be announced due to breaking changes to existing configs since users may have already used gzip compression.

👍 makes sense. This would be a breaking change and so we would want to note it in the upgrade guide for 0.23:

https://github.com/vectordotdev/vector/blob/master/website/content/en/highlights/2022-05-23-0-23-0-upgrade-guide.md

[jszwedko]

Just noting that the 0.23.0 release is going out today so the upgrade note should be in 0.24.0 if pursued.

[jszwedko]

Hi @xdatcloud ! Just wanted to see if you were still interested in contributing this. As mentioned above, I think we can make it the default behavior since there doesn't seem to be a performance impact.

[xdatcloud]

Hi @xdatcloud ! Just wanted to see if you were still interested in contributing this. As mentioned above, I think we can make it the default behavior since there doesn't seem to be a performance impact.

Hi @jszwedko! I have made snappy the default behavior, needs reviews here, thanks!

Also thanks to @tobz , your suggestions is helpful! Since the snappy has been the default compression, the existing integration tests are making sense.

[github-actions[bot]]

Soak Test Results

Baseline: afcee9be8840174292d47a9e06d19d059f77e246 Comparison: 68291d36adc141f3f87c31e3fb60608e98121f83 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
syslog_loki	295.43KiB	1.96	100.00%	14.74MiB	266.94KiB	5.47KiB	0	0.0176862	15.02MiB	766.97KiB	15.59KiB	0	0.0498403	False	False
http_text_to_http_json	780.19KiB	1.94	100.00%	39.34MiB	767.16KiB	15.66KiB	0	0.0190401	40.1MiB	683.72KiB	13.96KiB	0	0.0166468	False	False
syslog_humio_logs	148.09KiB	0.88	100.00%	16.41MiB	169.23KiB	3.46KiB	0	0.0100693	16.55MiB	181.89KiB	3.72KiB	0	0.0107284	False	False
syslog_splunk_hec_logs	128.3KiB	0.77	100.00%	16.35MiB	745.53KiB	15.17KiB	0	0.0445104	16.48MiB	638.95KiB	13.03KiB	0	0.0378568	False	False
http_to_http_acks	131.37KiB	0.74	42.78%	17.26MiB	7.8MiB	163.11KiB	0	0.451976	17.39MiB	7.94MiB	165.78KiB	0	0.456749	True	True
datadog_agent_remap_datadog_logs	299.66KiB	0.47	99.44%	62.0MiB	2.65MiB	55.52KiB	0	0.0426973	62.3MiB	4.45MiB	92.77KiB	0	0.0714707	False	False
syslog_log2metric_splunk_hec_metrics	83.57KiB	0.46	99.99%	17.6MiB	705.71KiB	14.38KiB	0	0.039154	17.68MiB	776.04KiB	15.81KiB	0	0.0428571	False	False
datadog_agent_remap_blackhole_acks	273.48KiB	0.42	99.40%	63.5MiB	4.26MiB	88.64KiB	0	0.0670057	63.76MiB	2.16MiB	45.24KiB	0	0.0338619	False	False
splunk_hec_route_s3	74.69KiB	0.39	73.69%	18.54MiB	2.28MiB	47.47KiB	0	0.122967	18.61MiB	2.24MiB	46.9KiB	0	0.120527	False	False
http_pipelines_blackhole_acks	4.12KiB	0.34	87.78%	1.18MiB	103.68KiB	2.11KiB	0	0.0856887	1.19MiB	79.69KiB	1.62KiB	0	0.0656369	False	False
datadog_agent_remap_datadog_logs_acks	61.18KiB	0.1	37.61%	61.22MiB	3.67MiB	76.78KiB	0	0.0599683	61.28MiB	4.72MiB	98.32KiB	0	0.0770667	False	False
splunk_hec_to_splunk_hec_logs_noack	22.53KiB	0.09	92.91%	23.82MiB	513.7KiB	10.48KiB	0	0.0210602	23.84MiB	330.79KiB	6.75KiB	0	0.0135489	False	False
splunk_hec_to_splunk_hec_logs_acks	15.7KiB	0.06	48.37%	23.75MiB	882.1KiB	17.94KiB	0	0.0362568	23.77MiB	797.53KiB	16.23KiB	0	0.0327595	False	False
datadog_agent_remap_blackhole	8.15KiB	0.01	6.15%	63.33MiB	4.05MiB	84.43KiB	0	0.0639404	63.34MiB	3.05MiB	63.55KiB	0	0.0480803	False	False
enterprise_http_to_http	-1.95KiB	-0.01	20.92%	23.85MiB	252.12KiB	5.15KiB	0	0.0103225	23.85MiB	257.34KiB	5.27KiB	0	0.010537	False	False
splunk_hec_indexer_ack_blackhole	-1.68KiB	-0.01	5.60%	23.75MiB	817.11KiB	16.62KiB	0	0.033585	23.75MiB	844.11KiB	17.17KiB	0	0.0346972	False	False
file_to_blackhole	-40.77KiB	-0.04	28.70%	95.33MiB	3.62MiB	75.12KiB	0	0.0380034	95.29MiB	3.92MiB	81.52KiB	0	0.0411386	False	False
http_to_http_json	-32.42KiB	-0.13	98.69%	23.84MiB	358.28KiB	7.31KiB	0	0.0146716	23.81MiB	529.59KiB	10.81KiB	0	0.0217155	False	False
syslog_regex_logs2metric_ddmetrics	-17.87KiB	-0.14	72.73%	12.4MiB	563.5KiB	11.48KiB	0	0.0443817	12.38MiB	567.02KiB	11.55KiB	0	0.0447215	False	False
fluent_elasticsearch	-403.39KiB	-0.5	100.00%	79.47MiB	53.37KiB	1.08KiB	0	0.000655708	79.08MiB	4.26MiB	87.52KiB	0	0.0538638	False	False
http_to_http_noack	-124.37KiB	-0.51	100.00%	23.84MiB	258.36KiB	5.28KiB	0	0.0105791	23.72MiB	1.18MiB	24.66KiB	0	0.0498702	False	False
http_pipelines_blackhole	-12.68KiB	-0.79	100.00%	1.58MiB	32.41KiB	678.24B	0	0.0200798	1.56MiB	116.22KiB	2.37KiB	0	0.0725775	False	False
socket_to_socket_blackhole	-187.42KiB	-0.79	100.00%	23.02MiB	631.93KiB	12.9KiB	0	0.0267982	22.84MiB	652.18KiB	13.31KiB	0	0.0278785	False	False
http_pipelines_no_grok_blackhole	-116.13KiB	-1.03	100.00%	11.0MiB	259.72KiB	5.3KiB	0	0.0230443	10.89MiB	1.06MiB	22.04KiB	0	0.0971594	False	False
syslog_log2metric_humio_metrics	-159.98KiB	-1.25	100.00%	12.48MiB	414.4KiB	8.46KiB	0	0.0324221	12.32MiB	500.48KiB	10.19KiB	0	0.0396535	False	False

[github-actions[bot]]

Soak Test Results

Baseline: 488bccb1a4b7494af0f90072b07395f23c97f26b Comparison: b6a53f23facc27610eaae66917c9c319cbddf9a5 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
http_text_to_http_json	835.37KiB	2.19	100.00%	37.24MiB	932.97KiB	19.04KiB	0	0.0244618	38.05MiB	1.01MiB	21.2KiB	0	0.0266412	False	False
splunk_hec_route_s3	193.56KiB	1.04	99.46%	18.22MiB	2.39MiB	49.68KiB	0	0.130883	18.41MiB	2.33MiB	48.72KiB	0	0.126478	False	False
http_pipelines_blackhole_acks	10.99KiB	0.88	100.00%	1.21MiB	114.4KiB	2.33KiB	0	0.0920083	1.22MiB	52.78KiB	1.08KiB	0	0.0420777	False	False
syslog_regex_logs2metric_ddmetrics	102.21KiB	0.82	100.00%	12.22MiB	596.4KiB	12.15KiB	0	0.0476568	12.32MiB	537.7KiB	10.96KiB	0	0.0426179	False	False
syslog_loki	51.42KiB	0.35	99.60%	14.47MiB	448.29KiB	9.17KiB	0	0.0302416	14.52MiB	753.07KiB	15.31KiB	0	0.0506269	False	False
syslog_humio_logs	56.02KiB	0.33	100.00%	16.62MiB	102.13KiB	2.08KiB	0	0.0059996	16.67MiB	112.65KiB	2.31KiB	0	0.0065962	False	False
syslog_splunk_hec_logs	21.76KiB	0.13	65.45%	15.94MiB	874.79KiB	17.8KiB	0	0.0535868	15.96MiB	719.38KiB	14.68KiB	0	0.0440081	False	False
splunk_hec_to_splunk_hec_logs_noack	27.49KiB	0.11	96.44%	23.81MiB	547.26KiB	11.17KiB	0	0.0224414	23.84MiB	332.79KiB	6.79KiB	0	0.0136311	False	False
splunk_hec_to_splunk_hec_logs_acks	3.88KiB	0.02	12.34%	23.75MiB	877.29KiB	17.84KiB	0	0.0360726	23.75MiB	859.85KiB	17.49KiB	0	0.03535	False	False
enterprise_http_to_http	-2.35KiB	-0.01	24.86%	23.85MiB	252.71KiB	5.16KiB	0	0.0103467	23.84MiB	259.58KiB	5.31KiB	0	0.0106291	False	False
splunk_hec_indexer_ack_blackhole	-9.66KiB	-0.04	28.98%	23.75MiB	874.52KiB	17.79KiB	0	0.0359502	23.74MiB	932.21KiB	18.96KiB	0	0.0383368	False	False
http_to_http_json	-33.69KiB	-0.14	99.14%	23.84MiB	349.64KiB	7.14KiB	0	0.0143176	23.81MiB	521.84KiB	10.66KiB	0	0.0213988	False	False
file_to_blackhole	-186.47KiB	-0.19	86.54%	95.37MiB	3.5MiB	72.6KiB	0	0.0367294	95.19MiB	4.88MiB	101.29KiB	0	0.0512336	False	False
datadog_agent_remap_datadog_logs_acks	-146.55KiB	-0.24	85.41%	58.5MiB	2.61MiB	54.58KiB	0	0.0445873	58.36MiB	4.07MiB	84.69KiB	0	0.0697095	False	False
http_to_http_noack	-82.88KiB	-0.34	99.99%	23.84MiB	263.6KiB	5.39KiB	0	0.010794	23.76MiB	1009.51KiB	20.56KiB	0	0.0414786	False	False
fluent_elasticsearch	-308.22KiB	-0.38	100.00%	79.47MiB	54.08KiB	1.09KiB	0	0.000664363	79.17MiB	3.01MiB	61.88KiB	0	0.0380392	False	False
http_pipelines_blackhole	-10.62KiB	-0.62	100.00%	1.69MiB	12.7KiB	265.79B	0	0.00735951	1.67MiB	122.64KiB	2.5KiB	0	0.0714955	False	False
datadog_agent_remap_datadog_logs	-418.16KiB	-0.65	100.00%	62.49MiB	1.49MiB	31.2KiB	0	0.0238148	62.08MiB	4.54MiB	94.53KiB	0	0.0730871	False	False
http_pipelines_no_grok_blackhole	-110.12KiB	-0.98	100.00%	11.03MiB	31.25KiB	653.39B	0	0.00276675	10.92MiB	1.06MiB	21.99KiB	0	0.0966604	False	False
datadog_agent_remap_blackhole	-709.45KiB	-1.1	100.00%	63.03MiB	4.13MiB	86.13KiB	0	0.0655615	62.34MiB	3.18MiB	66.24KiB	0	0.0509318	False	False
http_to_http_acks	-209.66KiB	-1.18	64.11%	17.35MiB	7.7MiB	160.96KiB	0	0.443595	17.14MiB	7.77MiB	162.18KiB	0	0.452937	True	True
syslog_log2metric_splunk_hec_metrics	-325.75KiB	-1.81	100.00%	17.6MiB	587.12KiB	11.97KiB	0	0.0325707	17.28MiB	1.01MiB	20.94KiB	0	0.0581549	False	False
syslog_log2metric_humio_metrics	-252.89KiB	-1.88	100.00%	13.16MiB	316.45KiB	6.45KiB	0	0.0234862	12.91MiB	666.13KiB	13.57KiB	0	0.0503853	False	False
socket_to_socket_blackhole	-452.42KiB	-1.93	100.00%	22.89MiB	601.39KiB	12.28KiB	0	0.0256557	22.44MiB	598.25KiB	12.21KiB	0	0.0260242	False	False
datadog_agent_remap_blackhole_acks	-1.76MiB	-2.76	100.00%	63.73MiB	4.72MiB	98.31KiB	0	0.074026	61.98MiB	5.06MiB	105.8KiB	0	0.0816995	False	False

[tobz]

@xdatcloud We're super close!

As I mentioned in my initial review, we'll need an integration test for this to ensure things work whether using the default compression scheme, or the new Snappy/Protocol Buffers variant.

[xdatcloud]

Hi @tobz ! I pushed more integration tests to repo. According to the discussion above:

Interestingly the performance was the same. I'd be content to just making this the one-and-only encoding and dropping the JSON encoder. What do you think @xdatcloud ?

Hi @jszwedko, thanks for the job! It looks like users would reach higher throughput without extra resources. We can use proto/snappy as the one-and-only encoding, but it should be announced due to breaking changes to existing configs since users may have already used gzip compression.

The Snappy/Protocol Buffer has been the default mode of Loki sink while pushing:

impl Default for CompressionConfigAdapter {
    fn default() -> Self {
        CompressionConfigAdapter::Extended(ExtendedCompression::Snappy)
    }
}

The existing integration tests are now working with snappy compression. To ensure all cases are covered, two more integration tests are added:

• text_with_none_compression - for testing compression: none
• text_with_gzip_compression - for testing compression: gzip

[tobz]

The Snappy/Protocol Buffer has been the default mode of Loki sink while pushing:

Ah, yeah, fair play. 👍🏻

The two additional integration tests look good to me. Giving it one more pass and then I think we're good to go. 🚀

[tobz]

@xdatcloud Ah, woops: looks like there's still some lints being hit. Most of them are pretty straightforward, and you can work through them locally by running make check-fmt check-style check-clippy.

One of them is related to our recent upgrade to Rust 1.63.0, and specifically revolves around errors with not deriving Eq when PartialEq is present, specifically on the code generated for the Protocol Buffers definitions. You can solve this just simply adding #![allow(clippy::derive_partial_eq_without_eq)] to the top of lib/loki-logproto/src/lib.rs.

[github-actions[bot]]

Soak Test Results

Baseline: fee0950f2c0b8fd53c816954d3ca7bb97ddda357 Comparison: ed528117798fda03d52cec1ae5ef0ad4bd00fe11 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
syslog_log2metric_splunk_hec_metrics	283.24KiB	1.66	100.00%	16.69MiB	624.68KiB	12.73KiB	0	0.0365502	16.96MiB	851.85KiB	17.33KiB	0	0.0490292	False	False
datadog_agent_remap_blackhole_acks	943.92KiB	1.52	100.00%	60.84MiB	5.58MiB	116.18KiB	0	0.0917149	61.77MiB	3.91MiB	81.53KiB	0	0.0632268	False	False
http_pipelines_blackhole_acks	14.82KiB	1.2	100.00%	1.21MiB	111.1KiB	2.26KiB	0	0.0895874	1.23MiB	73.05KiB	1.49KiB	0	0.058211	False	False
syslog_humio_logs	142.3KiB	0.85	100.00%	16.31MiB	255.33KiB	5.22KiB	0	0.0152864	16.45MiB	278.66KiB	5.7KiB	0	0.0165423	False	False
splunk_hec_route_s3	148.23KiB	0.79	96.74%	18.3MiB	2.4MiB	49.98KiB	0	0.131143	18.44MiB	2.3MiB	48.05KiB	0	0.12455	False	False
syslog_regex_logs2metric_ddmetrics	94.29KiB	0.77	100.00%	11.88MiB	423.4KiB	8.63KiB	0	0.0347915	11.97MiB	426.16KiB	8.68KiB	0	0.0347489	False	False
syslog_splunk_hec_logs	127.8KiB	0.76	100.00%	16.47MiB	608.0KiB	12.39KiB	0	0.0360462	16.59MiB	505.21KiB	10.32KiB	0	0.029727	False	False
splunk_hec_to_splunk_hec_logs_noack	30.27KiB	0.12	97.85%	23.81MiB	549.15KiB	11.2KiB	0	0.0225209	23.84MiB	338.44KiB	6.91KiB	0	0.0138623	False	False
enterprise_http_to_http	-2.6KiB	-0.01	27.56%	23.85MiB	256.58KiB	5.24KiB	0	0.0105052	23.84MiB	252.85KiB	5.17KiB	0	0.0103535	False	False
splunk_hec_indexer_ack_blackhole	-7.58KiB	-0.03	22.13%	23.75MiB	910.28KiB	18.52KiB	0	0.0374268	23.74MiB	964.01KiB	19.6KiB	0	0.0396482	False	False
splunk_hec_to_splunk_hec_logs_acks	-7.14KiB	-0.03	23.79%	23.76MiB	801.86KiB	16.32KiB	0	0.0329454	23.76MiB	837.37KiB	17.04KiB	0	0.0344146	False	False
file_to_blackhole	-44.85KiB	-0.05	43.33%	95.37MiB	2.68MiB	55.58KiB	0	0.028114	95.33MiB	2.65MiB	55.11KiB	0	0.0277915	False	False
http_to_http_json	-29.2KiB	-0.12	97.66%	23.84MiB	364.14KiB	7.43KiB	0	0.0149139	23.81MiB	514.69KiB	10.51KiB	0	0.0211051	False	False
http_to_http_noack	-96.49KiB	-0.4	100.00%	23.85MiB	254.45KiB	5.2KiB	0	0.0104186	23.75MiB	1.07MiB	22.23KiB	0	0.0448675	False	False
fluent_elasticsearch	-503.04KiB	-0.62	100.00%	79.47MiB	52.55KiB	1.06KiB	0	0.000645621	78.98MiB	5.06MiB	103.89KiB	0	0.0640739	False	False
http_pipelines_blackhole	-18.51KiB	-1.06	100.00%	1.71MiB	8.27KiB	173.16B	0	0.00473058	1.69MiB	137.56KiB	2.8KiB	0	0.0795597	False	False
http_pipelines_no_grok_blackhole	-135.15KiB	-1.19	100.00%	11.13MiB	47.3KiB	989.16B	0	0.00415098	10.99MiB	1.0MiB	20.88KiB	0	0.091143	False	False
syslog_log2metric_humio_metrics	-175.27KiB	-1.37	100.00%	12.47MiB	382.99KiB	7.82KiB	0	0.0299949	12.3MiB	553.33KiB	11.26KiB	0	0.0439387	False	False
datadog_agent_remap_blackhole	-914.98KiB	-1.42	100.00%	62.91MiB	4.08MiB	85.11KiB	0	0.0649123	62.02MiB	3.37MiB	70.26KiB	0	0.0543114	False	False
http_to_http_acks	-272.29KiB	-1.54	74.76%	17.32MiB	8.12MiB	169.86KiB	0	0.468895	17.05MiB	7.98MiB	166.51KiB	0	0.46759	True	True
datadog_agent_remap_datadog_logs_acks	-1.39MiB	-2.2	100.00%	63.11MiB	2.68MiB	55.99KiB	0	0.0423863	61.72MiB	4.35MiB	90.6KiB	0	0.0705044	False	False
datadog_agent_remap_datadog_logs	-2.05MiB	-3.23	100.00%	63.39MiB	1.75MiB	36.77KiB	0	0.0276625	61.34MiB	4.52MiB	94.11KiB	0	0.0736383	False	False
http_text_to_http_json	-1.31MiB	-3.31	100.00%	39.54MiB	745.9KiB	15.23KiB	0	0.0184189	38.23MiB	856.74KiB	17.49KiB	0	0.0218795	False	False
syslog_loki	-543.84KiB	-3.56	100.00%	14.93MiB	267.9KiB	5.49KiB	0	0.0175142	14.4MiB	724.79KiB	14.73KiB	0	0.0491309	False	False
socket_to_socket_blackhole	-1.05MiB	-4.34	100.00%	24.31MiB	119.54KiB	2.44KiB	0	0.00480029	23.26MiB	242.93KiB	4.96KiB	0	0.0101972	False	False

[xdatcloud]

@xdatcloud Ah, woops: looks like there's still some lints being hit. Most of them are pretty straightforward, and you can work through them locally by running make check-fmt check-style check-clippy.

One of them is related to our recent upgrade to Rust 1.63.0, and specifically revolves around errors with not deriving Eq when PartialEq is present, specifically on the code generated for the Protocol Buffers definitions. You can solve this just simply adding #![allow(clippy::derive_partial_eq_without_eq)] to the top of lib/loki-logproto/src/lib.rs.

Yeah, I have solved this in the latest commit, let's keep things moving. 💪

[tobz]

@xdatcloud It looks like you fixed the Clippy issues, but not the other linter issues, specifically our style checks (trailing newlines, spaces, etc).

As mentioned above, you should be able to see these during local development by running make check-style.

[github-actions[bot]]

Soak Test Results

Baseline: 01c7a36a9d8fe796c4da5e0b92d7660cfe6d4433 Comparison: f4ba2472a890f16e39c862188e5f4207913bcd6a Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
datadog_agent_remap_blackhole_acks	2.12MiB	3.56	100.00%	59.45MiB	5.07MiB	105.62KiB	0	0.0852817	61.56MiB	4.3MiB	89.84KiB	0	0.069788	False	False
socket_to_socket_blackhole	776.11KiB	3.31	100.00%	22.88MiB	122.09KiB	2.49KiB	0	0.00521016	23.64MiB	141.31KiB	2.89KiB	0	0.00583661	False	False
http_text_to_http_json	666.27KiB	1.68	100.00%	38.83MiB	863.86KiB	17.63KiB	0	0.0217192	39.48MiB	812.84KiB	16.6KiB	0	0.0200996	False	False
syslog_regex_logs2metric_ddmetrics	158.2KiB	1.25	100.00%	12.37MiB	542.72KiB	11.06KiB	0	0.0428295	12.53MiB	585.5KiB	11.93KiB	0	0.0456361	False	False
datadog_agent_remap_blackhole	388.69KiB	0.61	99.99%	61.72MiB	3.72MiB	77.49KiB	0	0.0602134	62.1MiB	2.77MiB	57.76KiB	0	0.0445497	False	False
http_pipelines_no_grok_blackhole	45.55KiB	0.42	97.28%	10.61MiB	165.43KiB	3.38KiB	0	0.0152226	10.66MiB	999.45KiB	20.33KiB	0	0.0915825	False	False
syslog_loki	53.52KiB	0.38	99.97%	13.76MiB	228.76KiB	4.69KiB	0	0.0162297	13.81MiB	696.07KiB	14.15KiB	0	0.049198	False	False
splunk_hec_route_s3	64.92KiB	0.33	67.53%	18.94MiB	2.28MiB	47.39KiB	0	0.12015	19.0MiB	2.19MiB	45.81KiB	0	0.115375	False	False
http_pipelines_blackhole	703.7B	0.04	20.39%	1.66MiB	47.22KiB	988.31B	0	0.0277338	1.66MiB	121.58KiB	2.48KiB	0	0.0713715	False	False
splunk_hec_to_splunk_hec_logs_noack	3.33KiB	0.01	25.34%	23.83MiB	374.91KiB	7.65KiB	0	0.0153585	23.84MiB	337.08KiB	6.88KiB	0	0.0138071	False	False
splunk_hec_indexer_ack_blackhole	-3.11KiB	-0.01	9.99%	23.75MiB	848.33KiB	17.26KiB	0	0.034871	23.75MiB	875.0KiB	17.8KiB	0	0.0359719	False	False
enterprise_http_to_http	-2.21KiB	-0.01	22.19%	23.85MiB	274.06KiB	5.59KiB	0	0.011221	23.84MiB	268.81KiB	5.5KiB	0	0.011007	False	False
file_to_blackhole	-80.1KiB	-0.08	54.73%	95.34MiB	3.37MiB	69.89KiB	0	0.0353552	95.26MiB	3.88MiB	80.58KiB	0	0.0406933	False	False
splunk_hec_to_splunk_hec_logs_acks	-24.01KiB	-0.1	68.89%	23.77MiB	777.05KiB	15.82KiB	0	0.0319171	23.75MiB	868.25KiB	17.66KiB	0	0.035698	False	False
http_to_http_json	-37.57KiB	-0.15	99.65%	23.85MiB	334.78KiB	6.83KiB	0	0.013707	23.81MiB	532.75KiB	10.88KiB	0	0.0218465	False	False
fluent_elasticsearch	-133.04KiB	-0.16	100.00%	79.47MiB	53.36KiB	1.08KiB	0	0.000655596	79.34MiB	1.39MiB	28.62KiB	0	0.0175045	False	False
datadog_agent_remap_datadog_logs_acks	-121.61KiB	-0.19	70.68%	62.24MiB	3.35MiB	70.04KiB	0	0.0538009	62.13MiB	4.42MiB	92.06KiB	0	0.0711698	False	False
http_to_http_noack	-68.86KiB	-0.28	99.63%	23.83MiB	516.44KiB	10.55KiB	0	0.0211613	23.76MiB	1.02MiB	21.21KiB	0	0.0427781	False	False
syslog_humio_logs	-49.53KiB	-0.3	100.00%	16.28MiB	431.96KiB	8.82KiB	0	0.0259053	16.23MiB	380.33KiB	7.79KiB	0	0.0228767	False	False
syslog_splunk_hec_logs	-80.76KiB	-0.49	99.95%	15.95MiB	812.62KiB	16.55KiB	0	0.0497462	15.87MiB	802.94KiB	16.36KiB	0	0.0493973	False	False
http_to_http_acks	-111.55KiB	-0.63	36.44%	17.34MiB	8.05MiB	168.33KiB	0	0.46407	17.23MiB	7.89MiB	164.55KiB	0	0.457582	True	True
datadog_agent_remap_datadog_logs	-459.75KiB	-0.73	100.00%	61.72MiB	1.38MiB	28.95KiB	0	0.0223806	61.28MiB	3.88MiB	80.84KiB	0	0.0633032	False	False
syslog_log2metric_splunk_hec_metrics	-323.89KiB	-1.9	100.00%	16.63MiB	1.31MiB	27.29KiB	0	0.0786186	16.31MiB	1.57MiB	32.69KiB	0	0.0961098	False	False
syslog_log2metric_humio_metrics	-427.6KiB	-3.23	100.00%	12.92MiB	364.56KiB	7.44KiB	0	0.0275521	12.5MiB	506.09KiB	10.3KiB	0	0.0395258	False	False
http_pipelines_blackhole_acks	-37.73KiB	-3.53	100.00%	1.04MiB	102.02KiB	2.08KiB	0	0.0954045	1.01MiB	69.61KiB	1.42KiB	0	0.0674762	False	False

[xdatcloud]

@xdatcloud It looks like you fixed the Clippy issues, but not the other linter issues, specifically our style checks (trailing newlines, spaces, etc).

As mentioned above, you should be able to see these during local development by running make check-style.

Sorry for that. The latest commit I pushed just now has passed all checks from running make check-fmt check-style check-clippy on my local develop environment.

[github-actions[bot]]

Soak Test Results

Baseline: 7a887d4076d3588c92e861f54744319ac122c3d6 Comparison: 5ab864b7e3b30a043f2fef6d1a5f099bfcb64597 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
http_pipelines_blackhole_acks	53.95KiB	4.56	100.00%	1.16MiB	135.78KiB	2.76KiB	0	0.114772	1.21MiB	107.21KiB	2.18KiB	0	0.086668	False	False
syslog_regex_logs2metric_ddmetrics	205.3KiB	1.62	100.00%	12.34MiB	507.12KiB	10.34KiB	0	0.0401258	12.54MiB	532.22KiB	10.85KiB	0	0.041439	False	False
http_text_to_http_json	541.88KiB	1.35	100.00%	39.2MiB	801.82KiB	16.37KiB	0	0.0199689	39.73MiB	703.32KiB	14.36KiB	0	0.0172826	False	False
datadog_agent_remap_blackhole	648.63KiB	1.06	100.00%	60.01MiB	4.43MiB	92.32KiB	0	0.073835	60.64MiB	3.44MiB	71.86KiB	0	0.0567881	False	False
socket_to_socket_blackhole	240.48KiB	1.06	100.00%	22.25MiB	973.19KiB	19.87KiB	0	0.042701	22.49MiB	970.27KiB	19.81KiB	0	0.0421283	False	False
datadog_agent_remap_blackhole_acks	660.78KiB	1.05	100.00%	61.7MiB	4.69MiB	97.66KiB	0	0.0760108	62.34MiB	3.02MiB	63.28KiB	0	0.0484972	False	False
splunk_hec_route_s3	180.21KiB	0.99	98.44%	17.8MiB	2.58MiB	53.72KiB	0	0.144784	17.97MiB	2.46MiB	51.58KiB	0	0.137003	False	False
http_pipelines_blackhole	13.76KiB	0.85	100.00%	1.59MiB	83.66KiB	1.71KiB	0	0.0514674	1.6MiB	128.3KiB	2.62KiB	0	0.0782718	False	False
syslog_humio_logs	77.44KiB	0.47	99.95%	16.1MiB	779.17KiB	15.91KiB	0	0.0472478	16.18MiB	757.54KiB	15.51KiB	0	0.0457218	False	False
syslog_splunk_hec_logs	16.61KiB	0.1	60.28%	16.5MiB	695.29KiB	14.16KiB	0	0.0411553	16.51MiB	665.72KiB	13.57KiB	0	0.0393661	False	False
splunk_hec_indexer_ack_blackhole	3.65KiB	0.02	11.28%	23.75MiB	896.47KiB	18.24KiB	0	0.0368492	23.76MiB	894.02KiB	18.19KiB	0	0.0367429	False	False
splunk_hec_to_splunk_hec_logs_noack	4.59KiB	0.02	34.01%	23.83MiB	388.03KiB	7.92KiB	0	0.0158966	23.84MiB	332.41KiB	6.79KiB	0	0.0136155	False	False
enterprise_http_to_http	-908.86B	-0	9.74%	23.85MiB	249.9KiB	5.1KiB	0	0.010232	23.85MiB	252.01KiB	5.16KiB	0	0.0103187	False	False
syslog_log2metric_splunk_hec_metrics	841.71B	0	3.47%	17.86MiB	543.26KiB	11.07KiB	0	0.029702	17.86MiB	752.75KiB	15.32KiB	0	0.0411533	False	False
splunk_hec_to_splunk_hec_logs_acks	-3.15KiB	-0.01	9.69%	23.75MiB	890.71KiB	18.11KiB	0	0.0366182	23.75MiB	907.64KiB	18.45KiB	0	0.0373192	False	False
file_to_blackhole	-31.02KiB	-0.03	20.99%	95.33MiB	3.91MiB	81.11KiB	0	0.0410352	95.3MiB	4.02MiB	83.67KiB	0	0.0421752	False	False
http_to_http_json	-19.14KiB	-0.08	91.42%	23.85MiB	331.29KiB	6.76KiB	0	0.0135643	23.83MiB	432.35KiB	8.85KiB	0	0.0177157	False	False
syslog_loki	-22.71KiB	-0.15	83.78%	14.67MiB	292.8KiB	5.99KiB	0	0.0194838	14.65MiB	742.77KiB	15.1KiB	0	0.0495002	False	False
fluent_elasticsearch	-222.34KiB	-0.27	100.00%	79.47MiB	54.32KiB	1.1KiB	0	0.000667295	79.26MiB	1.85MiB	38.14KiB	0	0.0233908	False	False
http_to_http_noack	-96.54KiB	-0.4	99.99%	23.84MiB	372.52KiB	7.61KiB	0	0.0152578	23.74MiB	1.11MiB	23.06KiB	0	0.0465729	False	False
datadog_agent_remap_datadog_logs_acks	-383.81KiB	-0.61	99.92%	61.14MiB	3.33MiB	69.59KiB	0	0.0544416	60.76MiB	4.34MiB	90.29KiB	0	0.0713537	False	False
datadog_agent_remap_datadog_logs	-580.41KiB	-0.92	100.00%	61.43MiB	372.01KiB	7.61KiB	0	0.00591291	60.86MiB	4.07MiB	84.76KiB	0	0.0668623	False	False
http_to_http_acks	-166.7KiB	-0.94	52.42%	17.31MiB	8.04MiB	168.09KiB	0	0.464599	17.14MiB	7.79MiB	162.46KiB	0	0.454236	True	True
http_pipelines_no_grok_blackhole	-144.0KiB	-1.33	100.00%	10.6MiB	399.27KiB	8.15KiB	0	0.0367881	10.46MiB	1.11MiB	23.19KiB	0	0.106513	False	False
syslog_log2metric_humio_metrics	-306.87KiB	-2.37	100.00%	12.65MiB	414.52KiB	8.46KiB	0	0.0319879	12.35MiB	577.75KiB	11.77KiB	0	0.045666	False	False

[xdatcloud]

@xdatcloud It looks like you fixed the Clippy issues, but not the other linter issues, specifically our style checks (trailing newlines, spaces, etc). As mentioned above, you should be able to see these during local development by running make check-style.

Sorry for that. The latest commit I pushed just now has passed all checks from running make check-fmt check-style check-clippy on my local develop environment.

Hi @tobz , I have merged recent commits from origin/master, it looks like we are ready to merge this PR into master.

[github-actions[bot]]

Soak Test Results

Baseline: 8c669b37f4ee9e158a3a84e17ba8509b979bd810 Comparison: eff8206e8a88340483faac6862656e1a5fe62ade Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
syslog_humio_logs	400.03KiB	2.57	100.00%	15.21MiB	1.43MiB	29.81KiB	0	0.0937511	15.6MiB	1.26MiB	26.48KiB	0	0.0809524	False	False
socket_to_socket_blackhole	553.42KiB	2.43	100.00%	22.27MiB	161.3KiB	3.29KiB	0	0.00707024	22.81MiB	169.99KiB	3.47KiB	0	0.00727491	False	False
http_text_to_http_json	701.62KiB	1.82	100.00%	37.58MiB	910.74KiB	18.59KiB	0	0.0236632	38.26MiB	869.92KiB	17.76KiB	0	0.0221977	False	False
splunk_hec_route_s3	219.57KiB	1.14	99.93%	18.83MiB	2.21MiB	45.99KiB	0	0.117276	19.04MiB	2.17MiB	45.38KiB	0	0.113975	False	False
http_pipelines_blackhole_acks	9.51KiB	0.77	99.96%	1.21MiB	110.41KiB	2.25KiB	0	0.0889851	1.22MiB	73.55KiB	1.5KiB	0	0.0588282	False	False
syslog_splunk_hec_logs	129.43KiB	0.77	100.00%	16.43MiB	678.83KiB	13.82KiB	0	0.0403455	16.55MiB	554.25KiB	11.31KiB	0	0.0326894	False	False
datadog_agent_remap_blackhole_acks	385.51KiB	0.59	99.97%	64.03MiB	4.1MiB	85.49KiB	0	0.0640734	64.4MiB	3.01MiB	62.89KiB	0	0.0466963	False	False
datadog_agent_remap_datadog_logs_acks	325.7KiB	0.51	99.65%	62.2MiB	2.98MiB	62.34KiB	0	0.0479227	62.52MiB	4.43MiB	92.31KiB	0	0.0709183	False	False
syslog_regex_logs2metric_ddmetrics	59.52KiB	0.46	99.96%	12.61MiB	645.97KiB	13.15KiB	0	0.0500276	12.67MiB	501.77KiB	10.23KiB	0	0.0386816	False	False
datadog_agent_remap_blackhole	208.23KiB	0.35	97.64%	57.47MiB	3.75MiB	78.02KiB	0	0.0651707	57.67MiB	2.33MiB	48.62KiB	0	0.0403745	False	False
syslog_log2metric_humio_metrics	42.38KiB	0.32	99.96%	12.8MiB	306.23KiB	6.25KiB	0	0.0233657	12.84MiB	505.72KiB	10.3KiB	0	0.0384628	False	False
splunk_hec_to_splunk_hec_logs_noack	20.33KiB	0.08	89.87%	23.82MiB	509.61KiB	10.4KiB	0	0.0208903	23.84MiB	331.33KiB	6.76KiB	0	0.0135708	False	False
splunk_hec_indexer_ack_blackhole	14.7KiB	0.06	44.31%	23.74MiB	895.77KiB	18.22KiB	0	0.0368338	23.76MiB	843.19KiB	17.16KiB	0	0.0346507	False	False
splunk_hec_to_splunk_hec_logs_acks	-498.92B	-0	1.85%	23.78MiB	728.83KiB	14.84KiB	0	0.0299218	23.78MiB	726.57KiB	14.8KiB	0	0.0298298	False	False
enterprise_http_to_http	-1.52KiB	-0.01	16.52%	23.85MiB	252.39KiB	5.15KiB	0	0.0103336	23.85MiB	251.96KiB	5.16KiB	0	0.0103165	False	False
file_to_blackhole	-86.11KiB	-0.09	60.31%	95.34MiB	3.12MiB	64.61KiB	0	0.0326837	95.26MiB	3.77MiB	78.45KiB	0	0.0396046	False	False
datadog_agent_remap_datadog_logs	-56.99KiB	-0.09	46.40%	62.41MiB	1.02MiB	21.44KiB	0	0.0164047	62.35MiB	4.3MiB	89.55KiB	0	0.0689623	False	False
syslog_log2metric_splunk_hec_metrics	-21.36KiB	-0.12	72.73%	17.82MiB	620.2KiB	12.64KiB	0	0.0339885	17.8MiB	727.63KiB	14.81KiB	0	0.0399227	False	False
http_to_http_json	-42.21KiB	-0.17	99.88%	23.85MiB	327.75KiB	6.69KiB	0	0.013419	23.81MiB	549.05KiB	11.21KiB	0	0.0225187	False	False
http_pipelines_blackhole	-2.88KiB	-0.17	74.65%	1.64MiB	14.0KiB	293.13B	0	0.00835433	1.63MiB	122.88KiB	2.5KiB	0	0.0734519	False	False
fluent_elasticsearch	-274.68KiB	-0.34	99.96%	79.47MiB	52.79KiB	1.07KiB	0	0.000648573	79.2MiB	3.75MiB	77.2KiB	0	0.0473567	False	False
http_to_http_noack	-86.34KiB	-0.35	99.97%	23.84MiB	413.63KiB	8.46KiB	0	0.0169428	23.75MiB	1.06MiB	22.17KiB	0	0.0447535	False	False
http_pipelines_no_grok_blackhole	-58.34KiB	-0.52	98.79%	10.97MiB	193.13KiB	3.94KiB	0	0.017188	10.91MiB	1.1MiB	22.89KiB	0	0.100718	False	False
syslog_loki	-197.14KiB	-1.31	100.00%	14.74MiB	364.64KiB	7.46KiB	0	0.0241612	14.54MiB	714.62KiB	14.53KiB	0	0.0479777	False	False
http_to_http_acks	-371.51KiB	-2.06	89.61%	17.61MiB	7.88MiB	164.72KiB	0	0.447312	17.25MiB	7.58MiB	158.26KiB	0	0.439557	True	True

[spencergilbert]

Apologies for the long review cycle. @tobz you've been closer to the technical changes, so if you think those are good to merge thats a 👍 from me. However, it looks like we've missed both the updates to the cue docs and a note in the next upgrade guide.