vector icon indicating copy to clipboard operation
vector copied to clipboard

Published 20 hours ago • verified publisher icon vectordotdev

Feature/clickhouse native protocol from cabirdme

Open zamazan4ik opened this issue 2 years ago • 19 comments

This PR continues work on https://github.com/vectordotdev/vector/pull/14787 in a separate PR.

I have resolved all pointed issues from the previous PR (except integration tests).

zamazan4ik avatar Dec 20 '22 23:12 zamazan4ik

Deploy Preview for vrl-playground canceled.

Name Link
Latest commit c4ccd61f5a5c3d8607f4208b6173f73004eb04fc
Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/63c89917da9ab8000886be7b

netlify[bot] avatar Dec 20 '22 23:12 netlify[bot]

Deploy Preview for vector-project ready!

Name Link
Latest commit c4ccd61f5a5c3d8607f4208b6173f73004eb04fc
Latest deploy log https://app.netlify.com/sites/vector-project/deploys/63c899171b9acd000922ad12
Deploy Preview https://deploy-preview-15660--vector-project.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

netlify[bot] avatar Dec 20 '22 23:12 netlify[bot]

@jszwedko @bruceg @neuronull pinging you since you were involved in the previous PR.

zamazan4ik avatar Dec 20 '22 23:12 zamazan4ik

After the merging with the master branch I got the following error:

error[E0599]: the method `run` exists for struct `vector_core::stream::Driver<Batcher<futures_util::stream::Map<Pin<Box<dyn futures_util::Stream<Item = vector_core::event::Event> + std::marker::Send>>, [closure@src/sinks/clickhouse/native/sink.rs:78:18: 78:21]>, BatchConfigParts<SizeLimit<ByteSizeOfItemSize>, std::vec::Vec<vector_core::event::LogEvent>>>, ClickhouseService>`, but its trait bounds were not satisfied
   --> src/sinks/clickhouse/native/sink.rs:85:14
    |
85  |             .run()
    |              ^^^ method cannot be called on `vector_core::stream::Driver<Batcher<futures_util::stream::Map<Pin<Box<dyn futures_util::Stream<Item = vector_core::event::Event> + std::marker::Send>>, [closure@src/sinks/clickhouse/native/sink.rs:78:18: 78:21]>, BatchConfigParts<SizeLimit<ByteSizeOfItemSize>, std::vec::Vec<vector_core::event::LogEvent>>>, ClickhouseService>` due to unsatisfied trait bounds
    |
   ::: /Users/zamazan4ik/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/vec/mod.rs:400:1
    |
400 | pub struct Vec<T, #[unstable(feature = "allocator_api", issue = "32838")] A: Allocator = Global> {
    | ------------------------------------------------------------------------------------------------ doesn't satisfy `_: MetaDescriptive`
    |
    = note: the following trait bounds were not satisfied:
            `std::vec::Vec<vector_core::event::LogEvent>: MetaDescriptive`

Could anyone point me, to how should I resolve it since the Clickhouse request has no metadata?

@neuronull probably you could help here :)

zamazan4ik avatar Dec 20 '22 23:12 zamazan4ik

@neuronull regarding integration tests. We will need to change a CLICKHOUSE_ADDRESS variable in docker-compose.yml since ClickHouse uses a different port for the native protocol.

I am not familiar with your integration environment, so... what do I need to do except expose another address like CLICKHOUSE_ADDRESS_NATIVE from the docker-compose file?

zamazan4ik avatar Dec 20 '22 23:12 zamazan4ik

Thanks for pushing this forward @zamazan4ik ! Just a note that much of the team is on holiday so it may not be reviewed until the start of next year.

jszwedko avatar Dec 21 '22 17:12 jszwedko

Echoing Jesse's appreciation for helping get this change solidified!

I'll try to get to this today but if I don't, it's in my queue for the new year.

neuronull avatar Dec 21 '22 23:12 neuronull

Could anyone point me, to how should I resolve it since the Clickhouse request has no metadata?

Hi @zamazan4ik , it looks like we'll need to extend some of the LogEvent / EventArray logic in the core part of the code, to handle MetaDescriptive , similar to EventFinalizers.

I'll take a look at that, it might warrant a separate PR. But if it's small I'll tack it onto your branch here.

Thanks again for picking this up!

neuronull avatar Jan 06 '23 18:01 neuronull

I'll take a look at that, it might warrant a separate PR. But if it's small I'll tack it onto your branch here.

I suggest implementing it as a separate PR since it could be useful for other metadata-related PRs. No worries, after the merge of the metadata-related PR I'll rebase the Clickhouse PR. Just ping me here for that :)

Thank you a lot for the help!

zamazan4ik avatar Jan 06 '23 20:01 zamazan4ik

Looking at this more closely, I think the part in the sink run() function

.map(|e| e.into_log())

, could maybe be removed, and that leaves us with just an Event , which does have the provisions for being used with a RequestMetadataBuilder.

Essentially I think we need to change the type that the Service operates over from a Vec<LogEvent> , to a new struct, something like


struct NativeClickhouseRequest {
    event: Event,
    metadata: RequestMetadata,
 }

, and then create a RequestMetadataBuilder to build the RequestMetadata during the event processing, potentially similar to what is being done in the vector sink.

I can circle back to this next week but it might be a couple days. Feel free to have at it if you're feeling frosty 🙂

neuronull avatar Jan 07 '23 00:01 neuronull

Looking at this more closely, I think the part in the sink run() function

.map(|e| e.into_log())

, could maybe be removed, and that leaves us with just an Event , which does have the provisions for being used with a RequestMetadataBuilder.

Essentially I think we need to change the type that the Service operates over from a Vec , to a new struct, something like

struct NativeClickhouseRequest {
    event: Event,
    metadata: RequestMetadata,
 }

, and then create a RequestMetadataBuilder to build the RequestMetadata during the event processing, potentially similar to what is being done in the vector sink.

I can circle back to this next week but it might be a couple days. Feel free to have at it if you're feeling frosty 🙂

Hey @zamazan4ik , wanted to let you know that my bandwidth is pretty thin right now and I was wondering if you wouldn't mind taking on the addition of this MetaDescriptive changes.

Basically in addition to the above notes I made, generally speaking I think the issue is not that the "API" so to speak for MetaDescriptive is incomplete- but more that this sink needs to add metadata and that unfortunately there isn't a 1-1 direct example of another sink following this same model. But, I think the vector sink could be a good example.

Let me know if there is anything else I can clarify. Cheers~

neuronull avatar Jan 12 '23 23:01 neuronull

Thanks! I will look into the vector sink to find a similar example. I hope I can figure out how to add the required metadata to the Clickhouse sink. If I'll have a question - I'll ping you here, no worries :)

zamazan4ik avatar Jan 12 '23 23:01 zamazan4ik

I have tried to implement the request metadata enrichment based on vector sink implementation in the latest commit.

zamazan4ik avatar Jan 14 '23 18:01 zamazan4ik

Regression Test Results

Run ID: c41ab3b1-c468-4b5b-ba31-75e4c49d09a7
Baseline: 94fcd43bbe562e290ba277b1e78d324cd6000f31
Comparison: 0add90125bff37ffda5a61777679cb1f7d11bef1
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json 536.11KiB/CPU-s 2.19 100.00% 23.89MiB/CPU-s 867.33KiB/CPU-s 11.25KiB/CPU-s 0.0 0.035445 24.42MiB/CPU-s 915.71KiB/CPU-s 11.81KiB/CPU-s 0.0 0.03662 False False
syslog_loki 188.38KiB/CPU-s 2.12 100.00% 8.69MiB/CPU-s 409.5KiB/CPU-s 5.28KiB/CPU-s 0.0 0.046003 8.88MiB/CPU-s 216.6KiB/CPU-s 2.8KiB/CPU-s 0.0 0.023828 False False
datadog_agent_remap_datadog_logs_acks 238.94KiB/CPU-s 0.7 100.00% 33.27MiB/CPU-s 1.39MiB/CPU-s 18.32KiB/CPU-s 0.0 0.041664 33.51MiB/CPU-s 1.46MiB/CPU-s 19.32KiB/CPU-s 0.0 0.043632 False False
syslog_regex_logs2metric_ddmetrics 23.49KiB/CPU-s 0.64 99.92% 3.59MiB/CPU-s 389.27KiB/CPU-s 5.02KiB/CPU-s 0.0 0.105852 3.61MiB/CPU-s 374.09KiB/CPU-s 4.83KiB/CPU-s 0.0 0.101079 True False
syslog_splunk_hec_logs 31.49KiB/CPU-s 0.34 100.00% 9.02MiB/CPU-s 205.0KiB/CPU-s 2.65KiB/CPU-s 0.0 0.022193 9.05MiB/CPU-s 257.34KiB/CPU-s 3.32KiB/CPU-s 0.0 0.027764 False False
syslog_log2metric_splunk_hec_metrics 17.76KiB/CPU-s 0.19 99.39% 9.17MiB/CPU-s 420.38KiB/CPU-s 5.43KiB/CPU-s 0.0 0.044759 9.19MiB/CPU-s 273.32KiB/CPU-s 3.53KiB/CPU-s 0.0 0.029046 False False
datadog_agent_remap_datadog_logs 41.09KiB/CPU-s 0.12 79.17% 33.18MiB/CPU-s 1.72MiB/CPU-s 22.75KiB/CPU-s 0.0 0.05191 33.22MiB/CPU-s 1.77MiB/CPU-s 23.42KiB/CPU-s 0.0 0.053378 False False
enterprise_http_to_http 12.34KiB/CPU-s 0.09 99.21% 13.61MiB/CPU-s 327.66KiB/CPU-s 4.23KiB/CPU-s 0.0 0.023505 13.62MiB/CPU-s 148.14KiB/CPU-s 1.91KiB/CPU-s 0.0 0.010618 False False
http_to_http_json 8.82KiB/CPU-s 0.06 91.72% 13.58MiB/CPU-s 283.24KiB/CPU-s 3.65KiB/CPU-s 0.0 0.020366 13.59MiB/CPU-s 273.84KiB/CPU-s 3.53KiB/CPU-s 0.0 0.019678 False False
splunk_hec_to_splunk_hec_logs_noack 4.02KiB/CPU-s 0.03 70.52% 13.62MiB/CPU-s 227.33KiB/CPU-s 2.93KiB/CPU-s 0.0 0.016297 13.62MiB/CPU-s 191.69KiB/CPU-s 2.47KiB/CPU-s 0.0 0.013738 False False
splunk_hec_to_splunk_hec_logs_acks 112.52B/CPU-s 0.0 1.40% 13.62MiB/CPU-s 354.18KiB/CPU-s 4.57KiB/CPU-s 0.0 0.025396 13.62MiB/CPU-s 332.71KiB/CPU-s 4.29KiB/CPU-s 0.0 0.023857 False False
fluent_elasticsearch 146.47B/CPU-s 0.0 20.75% 45.41MiB/CPU-s 30.13KiB/CPU-s 393.74B/CPU-s 0.0 0.000648 45.41MiB/CPU-s 30.15KiB/CPU-s 393.64B/CPU-s 0.0 0.000648 False False
http_to_http_noack -307.7B/CPU-s -0.0 4.25% 13.61MiB/CPU-s 307.89KiB/CPU-s 3.97KiB/CPU-s 0.0 0.022084 13.61MiB/CPU-s 310.6KiB/CPU-s 4.01KiB/CPU-s 0.0 0.022279 False False
splunk_hec_indexer_ack_blackhole -613.92B/CPU-s -0.0 10.28% 13.62MiB/CPU-s 252.29KiB/CPU-s 3.25KiB/CPU-s 0.0 0.018092 13.62MiB/CPU-s 256.52KiB/CPU-s 3.31KiB/CPU-s 0.0 0.018396 False False
file_to_blackhole -23.65KiB/CPU-s -0.04 69.27% 54.49MiB/CPU-s 1.15MiB/CPU-s 15.18KiB/CPU-s 0.0 0.021085 54.47MiB/CPU-s 1.33MiB/CPU-s 17.5KiB/CPU-s 0.0 0.024359 False False
socket_to_socket_blackhole -52.63KiB/CPU-s -0.39 100.00% 13.07MiB/CPU-s 483.16KiB/CPU-s 6.24KiB/CPU-s 0.0 0.036091 13.02MiB/CPU-s 554.12KiB/CPU-s 7.16KiB/CPU-s 0.0 0.041555 False False
datadog_agent_remap_blackhole -136.51KiB/CPU-s -0.43 100.00% 31.29MiB/CPU-s 714.9KiB/CPU-s 9.23KiB/CPU-s 0.0 0.022312 31.15MiB/CPU-s 991.35KiB/CPU-s 12.79KiB/CPU-s 0.0 0.031072 False False
otlp_grpc_to_blackhole -9.71KiB/CPU-s -0.91 100.00% 1.05MiB/CPU-s 39.24KiB/CPU-s 518.63B/CPU-s 0.0 0.036576 1.04MiB/CPU-s 49.04KiB/CPU-s 648.17B/CPU-s 0.0 0.046134 False False
syslog_humio_logs -96.36KiB/CPU-s -1.04 100.00% 9.05MiB/CPU-s 230.28KiB/CPU-s 2.97KiB/CPU-s 0.0 0.024848 8.96MiB/CPU-s 407.24KiB/CPU-s 5.26KiB/CPU-s 0.0 0.044405 False False
http_to_http_acks -60.94KiB/CPU-s -1.12 76.06% 5.32MiB/CPU-s 2.84MiB/CPU-s 37.59KiB/CPU-s 0.0 0.534876 5.26MiB/CPU-s 2.7MiB/CPU-s 35.63KiB/CPU-s 0.0 0.512582 True False
otlp_http_to_blackhole -17.8KiB/CPU-s -1.13 100.00% 1.54MiB/CPU-s 117.02KiB/CPU-s 1.51KiB/CPU-s 0.0 0.074347 1.52MiB/CPU-s 130.21KiB/CPU-s 1.68KiB/CPU-s 0.0 0.083673 False False
syslog_log2metric_humio_metrics -74.89KiB/CPU-s -1.21 100.00% 6.04MiB/CPU-s 187.76KiB/CPU-s 2.42KiB/CPU-s 0.0 0.030379 5.96MiB/CPU-s 229.28KiB/CPU-s 2.96KiB/CPU-s 0.0 0.037552 False False
splunk_hec_route_s3 -254.04KiB/CPU-s -2.14 100.00% 11.61MiB/CPU-s 538.1KiB/CPU-s 6.94KiB/CPU-s 0.0 0.045274 11.36MiB/CPU-s 845.62KiB/CPU-s 10.91KiB/CPU-s 0.0 0.072701 False False
datadog_agent_remap_blackhole_acks -882.55KiB/CPU-s -2.73 100.00% 31.55MiB/CPU-s 804.41KiB/CPU-s 10.39KiB/CPU-s 0.0 0.024897 30.69MiB/CPU-s 1.56MiB/CPU-s 20.63KiB/CPU-s 0.0 0.05085 False False

github-actions[bot] avatar Jan 14 '23 19:01 github-actions[bot]

Regression Test Results

Run ID: 9332fac0-da53-4351-9918-9d368eaabca3
Baseline: 94fcd43bbe562e290ba277b1e78d324cd6000f31
Comparison: 7ec097307e90cc5adcbe07819548166715e407a6
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_loki 264.55KiB/CPU-s 3.0 100.00% 8.61MiB/CPU-s 343.33KiB/CPU-s 4.43KiB/CPU-s 0.0 0.038928 8.87MiB/CPU-s 182.15KiB/CPU-s 2.35KiB/CPU-s 0.0 0.020051 False False
syslog_humio_logs 175.87KiB/CPU-s 1.93 100.00% 8.88MiB/CPU-s 343.38KiB/CPU-s 4.43KiB/CPU-s 0.0 0.037771 9.05MiB/CPU-s 391.36KiB/CPU-s 5.05KiB/CPU-s 0.0 0.042231 False False
http_text_to_http_json 475.02KiB/CPU-s 1.93 100.00% 24.04MiB/CPU-s 661.13KiB/CPU-s 8.53KiB/CPU-s 0.0 0.026859 24.5MiB/CPU-s 731.78KiB/CPU-s 9.44KiB/CPU-s 0.0 0.029166 False False
syslog_log2metric_splunk_hec_metrics 150.17KiB/CPU-s 1.6 100.00% 9.16MiB/CPU-s 357.37KiB/CPU-s 4.61KiB/CPU-s 0.0 0.038114 9.3MiB/CPU-s 203.31KiB/CPU-s 2.62KiB/CPU-s 0.0 0.021342 False False
socket_to_socket_blackhole 148.85KiB/CPU-s 1.09 100.00% 13.3MiB/CPU-s 296.25KiB/CPU-s 3.82KiB/CPU-s 0.0 0.021743 13.45MiB/CPU-s 141.43KiB/CPU-s 1.83KiB/CPU-s 0.0 0.010268 False False
splunk_hec_route_s3 104.61KiB/CPU-s 0.88 100.00% 11.63MiB/CPU-s 584.22KiB/CPU-s 7.54KiB/CPU-s 0.0 0.049056 11.73MiB/CPU-s 562.46KiB/CPU-s 7.26KiB/CPU-s 0.0 0.046817 False False
otlp_http_to_blackhole 10.71KiB/CPU-s 0.69 100.00% 1.51MiB/CPU-s 120.22KiB/CPU-s 1.55KiB/CPU-s 0.0 0.077617 1.52MiB/CPU-s 120.71KiB/CPU-s 1.56KiB/CPU-s 0.0 0.077402 False False
datadog_agent_remap_blackhole_acks 169.6KiB/CPU-s 0.53 100.00% 31.1MiB/CPU-s 1.28MiB/CPU-s 16.98KiB/CPU-s 0.0 0.041299 31.27MiB/CPU-s 879.19KiB/CPU-s 11.35KiB/CPU-s 0.0 0.027457 False False
syslog_splunk_hec_logs 37.32KiB/CPU-s 0.41 100.00% 8.97MiB/CPU-s 235.43KiB/CPU-s 3.04KiB/CPU-s 0.0 0.025634 9.0MiB/CPU-s 315.35KiB/CPU-s 4.07KiB/CPU-s 0.0 0.034196 False False
enterprise_http_to_http 7.84KiB/CPU-s 0.06 86.95% 13.61MiB/CPU-s 329.94KiB/CPU-s 4.26KiB/CPU-s 0.0 0.023668 13.62MiB/CPU-s 228.86KiB/CPU-s 2.95KiB/CPU-s 0.0 0.016408 False False
splunk_hec_to_splunk_hec_logs_noack 5.43KiB/CPU-s 0.04 84.05% 13.62MiB/CPU-s 230.85KiB/CPU-s 2.98KiB/CPU-s 0.0 0.016551 13.62MiB/CPU-s 190.32KiB/CPU-s 2.46KiB/CPU-s 0.0 0.01364 False False
http_to_http_noack 22.15B/CPU-s 0.0 0.34% 13.62MiB/CPU-s 282.68KiB/CPU-s 3.65KiB/CPU-s 0.0 0.020272 13.62MiB/CPU-s 275.97KiB/CPU-s 3.56KiB/CPU-s 0.0 0.019791 False False
fluent_elasticsearch 226.22B/CPU-s 0.0 31.87% 45.41MiB/CPU-s 29.67KiB/CPU-s 387.78B/CPU-s 0.0 0.000638 45.41MiB/CPU-s 29.92KiB/CPU-s 391.09B/CPU-s 0.0 0.000643 False False
splunk_hec_indexer_ack_blackhole -568.14B/CPU-s -0.0 9.63% 13.62MiB/CPU-s 251.46KiB/CPU-s 3.24KiB/CPU-s 0.0 0.018032 13.62MiB/CPU-s 251.4KiB/CPU-s 3.24KiB/CPU-s 0.0 0.018029 False False
splunk_hec_to_splunk_hec_logs_acks 704.25B/CPU-s 0.0 8.20% 13.61MiB/CPU-s 363.61KiB/CPU-s 4.69KiB/CPU-s 0.0 0.026081 13.61MiB/CPU-s 369.12KiB/CPU-s 4.76KiB/CPU-s 0.0 0.026475 False False
http_to_http_json -746.37B/CPU-s -0.01 15.05% 13.62MiB/CPU-s 208.27KiB/CPU-s 2.69KiB/CPU-s 0.0 0.014929 13.62MiB/CPU-s 212.62KiB/CPU-s 2.74KiB/CPU-s 0.0 0.015242 False False
file_to_blackhole -6.66KiB/CPU-s -0.01 25.49% 54.49MiB/CPU-s 1.06MiB/CPU-s 13.99KiB/CPU-s 0.0 0.019431 54.49MiB/CPU-s 1.13MiB/CPU-s 14.95KiB/CPU-s 0.0 0.020794 False False
datadog_agent_remap_datadog_logs_acks -64.5KiB/CPU-s -0.19 98.90% 33.42MiB/CPU-s 1.31MiB/CPU-s 17.26KiB/CPU-s 0.0 0.03909 33.36MiB/CPU-s 1.41MiB/CPU-s 18.57KiB/CPU-s 0.0 0.042114 False False
syslog_regex_logs2metric_ddmetrics -7.28KiB/CPU-s -0.2 56.53% 3.54MiB/CPU-s 513.03KiB/CPU-s 6.62KiB/CPU-s 0.0 0.141355 3.54MiB/CPU-s 508.52KiB/CPU-s 6.56KiB/CPU-s 0.0 0.140393 True False
datadog_agent_remap_blackhole -104.34KiB/CPU-s -0.33 99.94% 30.59MiB/CPU-s 1.51MiB/CPU-s 19.97KiB/CPU-s 0.0 0.049382 30.49MiB/CPU-s 1.74MiB/CPU-s 23.04KiB/CPU-s 0.0 0.057183 False False
http_to_http_acks -38.18KiB/CPU-s -0.71 54.92% 5.23MiB/CPU-s 2.72MiB/CPU-s 35.91KiB/CPU-s 0.0 0.519564 5.19MiB/CPU-s 2.7MiB/CPU-s 35.7KiB/CPU-s 0.0 0.520083 True False
otlp_grpc_to_blackhole -8.73KiB/CPU-s -0.82 100.00% 1.05MiB/CPU-s 39.65KiB/CPU-s 524.05B/CPU-s 0.0 0.037004 1.04MiB/CPU-s 54.08KiB/CPU-s 714.45B/CPU-s 0.0 0.050888 False False
syslog_log2metric_humio_metrics -63.27KiB/CPU-s -1.03 100.00% 6.0MiB/CPU-s 273.48KiB/CPU-s 3.53KiB/CPU-s 0.0 0.044525 5.94MiB/CPU-s 214.27KiB/CPU-s 2.77KiB/CPU-s 0.0 0.035249 False False
datadog_agent_remap_datadog_logs -1.08MiB/CPU-s -3.21 100.00% 33.8MiB/CPU-s 1.21MiB/CPU-s 15.94KiB/CPU-s 0.0 0.035703 32.72MiB/CPU-s 1.82MiB/CPU-s 24.07KiB/CPU-s 0.0 0.055681 False False

github-actions[bot] avatar Jan 14 '23 21:01 github-actions[bot]

Regression Test Results

Run ID: 6fea43fe-0f1a-4f9e-9911-a316115e648e
Baseline: 94fcd43bbe562e290ba277b1e78d324cd6000f31
Comparison: 975c03016dd9de651c80fd6da63387dc5a96fd26
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json 734.39KiB/CPU-s 2.99 100.00% 23.96MiB/CPU-s 821.88KiB/CPU-s 10.61KiB/CPU-s 0.0 0.033492 24.68MiB/CPU-s 819.08KiB/CPU-s 10.57KiB/CPU-s 0.0 0.032408 False False
syslog_loki 142.26KiB/CPU-s 1.6 100.00% 8.69MiB/CPU-s 318.24KiB/CPU-s 4.11KiB/CPU-s 0.0 0.035747 8.83MiB/CPU-s 208.84KiB/CPU-s 2.7KiB/CPU-s 0.0 0.02309 False False
datadog_agent_remap_blackhole 328.1KiB/CPU-s 1.05 100.00% 30.63MiB/CPU-s 1.19MiB/CPU-s 15.76KiB/CPU-s 0.0 0.038922 30.95MiB/CPU-s 1.11MiB/CPU-s 14.69KiB/CPU-s 0.0 0.03593 False False
syslog_log2metric_splunk_hec_metrics 96.19KiB/CPU-s 1.01 100.00% 9.3MiB/CPU-s 218.37KiB/CPU-s 2.82KiB/CPU-s 0.0 0.022927 9.39MiB/CPU-s 351.78KiB/CPU-s 4.54KiB/CPU-s 0.0 0.036565 False False
syslog_humio_logs 84.35KiB/CPU-s 0.92 100.00% 9.0MiB/CPU-s 338.28KiB/CPU-s 4.37KiB/CPU-s 0.0 0.036706 9.08MiB/CPU-s 161.18KiB/CPU-s 2.08KiB/CPU-s 0.0 0.017331 False False
socket_to_socket_blackhole 124.53KiB/CPU-s 0.91 100.00% 13.4MiB/CPU-s 154.57KiB/CPU-s 2.0KiB/CPU-s 0.0 0.011262 13.52MiB/CPU-s 216.24KiB/CPU-s 2.79KiB/CPU-s 0.0 0.015614 False False
syslog_regex_logs2metric_ddmetrics 27.6KiB/CPU-s 0.76 99.99% 3.54MiB/CPU-s 392.04KiB/CPU-s 5.06KiB/CPU-s 0.0 0.108149 3.57MiB/CPU-s 404.57KiB/CPU-s 5.22KiB/CPU-s 0.0 0.110762 True False
datadog_agent_remap_blackhole_acks 178.18KiB/CPU-s 0.57 100.00% 30.32MiB/CPU-s 1.39MiB/CPU-s 18.41KiB/CPU-s 0.0 0.045923 30.49MiB/CPU-s 1.98MiB/CPU-s 26.24KiB/CPU-s 0.0 0.06509 False False
http_to_http_json 57.68KiB/CPU-s 0.42 100.00% 13.56MiB/CPU-s 300.33KiB/CPU-s 3.87KiB/CPU-s 0.0 0.021629 13.61MiB/CPU-s 227.26KiB/CPU-s 2.93KiB/CPU-s 0.0 0.016299 False False
syslog_splunk_hec_logs 9.5KiB/CPU-s 0.1 92.12% 8.94MiB/CPU-s 202.94KiB/CPU-s 2.62KiB/CPU-s 0.0 0.022166 8.95MiB/CPU-s 366.37KiB/CPU-s 4.73KiB/CPU-s 0.0 0.039976 False False
datadog_agent_remap_datadog_logs 25.99KiB/CPU-s 0.07 70.90% 33.88MiB/CPU-s 1.41MiB/CPU-s 18.56KiB/CPU-s 0.0 0.041471 33.9MiB/CPU-s 1.22MiB/CPU-s 16.17KiB/CPU-s 0.0 0.036113 False False
enterprise_http_to_http 6.12KiB/CPU-s 0.04 78.71% 13.61MiB/CPU-s 305.4KiB/CPU-s 3.94KiB/CPU-s 0.0 0.021905 13.62MiB/CPU-s 227.72KiB/CPU-s 2.94KiB/CPU-s 0.0 0.016327 False False
splunk_hec_to_splunk_hec_logs_noack 4.28KiB/CPU-s 0.03 73.48% 13.62MiB/CPU-s 229.67KiB/CPU-s 2.96KiB/CPU-s 0.0 0.016465 13.63MiB/CPU-s 189.43KiB/CPU-s 2.45KiB/CPU-s 0.0 0.013576 False False
splunk_hec_to_splunk_hec_logs_acks 2.57KiB/CPU-s 0.02 31.41% 13.62MiB/CPU-s 352.29KiB/CPU-s 4.55KiB/CPU-s 0.0 0.025263 13.62MiB/CPU-s 343.1KiB/CPU-s 4.43KiB/CPU-s 0.0 0.0246 False False
fluent_elasticsearch 251.69B/CPU-s 0.0 34.63% 45.41MiB/CPU-s 30.73KiB/CPU-s 401.64B/CPU-s 0.0 0.000661 45.41MiB/CPU-s 29.98KiB/CPU-s 391.77B/CPU-s 0.0 0.000645 False False
file_to_blackhole -5.0KiB/CPU-s -0.01 18.38% 54.49MiB/CPU-s 1.12MiB/CPU-s 14.73KiB/CPU-s 0.0 0.020462 54.49MiB/CPU-s 1.19MiB/CPU-s 15.69KiB/CPU-s 0.0 0.021827 False False
splunk_hec_indexer_ack_blackhole -2.28KiB/CPU-s -0.02 38.17% 13.62MiB/CPU-s 246.41KiB/CPU-s 3.18KiB/CPU-s 0.0 0.017668 13.62MiB/CPU-s 254.1KiB/CPU-s 3.28KiB/CPU-s 0.0 0.018223 False False
http_to_http_noack -4.1KiB/CPU-s -0.03 51.00% 13.61MiB/CPU-s 303.36KiB/CPU-s 3.91KiB/CPU-s 0.0 0.021759 13.61MiB/CPU-s 346.85KiB/CPU-s 4.48KiB/CPU-s 0.0 0.024885 False False
splunk_hec_route_s3 -47.39KiB/CPU-s -0.4 100.00% 11.69MiB/CPU-s 542.08KiB/CPU-s 6.99KiB/CPU-s 0.0 0.045291 11.64MiB/CPU-s 567.84KiB/CPU-s 7.33KiB/CPU-s 0.0 0.047632 False False
datadog_agent_remap_datadog_logs_acks -410.77KiB/CPU-s -1.21 100.00% 33.2MiB/CPU-s 1.28MiB/CPU-s 16.98KiB/CPU-s 0.0 0.038693 32.8MiB/CPU-s 1.82MiB/CPU-s 23.99KiB/CPU-s 0.0 0.055336 False False
otlp_grpc_to_blackhole -16.12KiB/CPU-s -1.52 100.00% 1.04MiB/CPU-s 48.28KiB/CPU-s 638.11B/CPU-s 0.0 0.045438 1.02MiB/CPU-s 54.91KiB/CPU-s 725.55B/CPU-s 0.0 0.052473 False False
http_to_http_acks -85.9KiB/CPU-s -1.58 90.63% 5.3MiB/CPU-s 2.75MiB/CPU-s 36.36KiB/CPU-s 0.0 0.519089 5.22MiB/CPU-s 2.73MiB/CPU-s 36.11KiB/CPU-s 0.0 0.523794 True False
otlp_http_to_blackhole -35.84KiB/CPU-s -2.26 100.00% 1.55MiB/CPU-s 105.1KiB/CPU-s 1.36KiB/CPU-s 0.0 0.066228 1.51MiB/CPU-s 129.13KiB/CPU-s 1.67KiB/CPU-s 0.0 0.083249 False False
syslog_log2metric_humio_metrics -255.42KiB/CPU-s -4.14 100.00% 6.03MiB/CPU-s 257.41KiB/CPU-s 3.32KiB/CPU-s 0.0 0.041675 5.78MiB/CPU-s 267.48KiB/CPU-s 3.45KiB/CPU-s 0.0 0.045174 False False

github-actions[bot] avatar Jan 14 '23 22:01 github-actions[bot]

Regression Test Results

Run ID: 5168c29a-a567-423e-90c9-de6681c071a1
Baseline: 94fcd43bbe562e290ba277b1e78d324cd6000f31
Comparison: 44eb057fc5319e8e84c4a6f364701ffc4630b255
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
syslog_regex_logs2metric_ddmetrics -300.05KiB/CPU-s -7.88 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_humio_logs 261.14KiB/CPU-s 2.88 100.00% 8.87MiB/CPU-s 333.02KiB/CPU-s 4.3KiB/CPU-s 0.0 0.036674 9.12MiB/CPU-s 325.98KiB/CPU-s 4.21KiB/CPU-s 0.0 0.034896 False False
datadog_agent_remap_blackhole_acks 730.47KiB/CPU-s 2.36 100.00% 30.21MiB/CPU-s 1.42MiB/CPU-s 18.75KiB/CPU-s 0.0 0.046924 30.93MiB/CPU-s 1.13MiB/CPU-s 14.99KiB/CPU-s 0.0 0.036658 False False
syslog_loki 115.02KiB/CPU-s 1.29 100.00% 8.7MiB/CPU-s 291.01KiB/CPU-s 3.76KiB/CPU-s 0.0 0.032667 8.81MiB/CPU-s 295.54KiB/CPU-s 3.82KiB/CPU-s 0.0 0.032753 False False
http_text_to_http_json 285.69KiB/CPU-s 1.15 100.00% 24.22MiB/CPU-s 607.34KiB/CPU-s 7.84KiB/CPU-s 0.0 0.02449 24.5MiB/CPU-s 1.09MiB/CPU-s 14.34KiB/CPU-s 0.0 0.044323 False False
datadog_agent_remap_blackhole 261.39KiB/CPU-s 0.84 100.00% 30.57MiB/CPU-s 1.24MiB/CPU-s 16.44KiB/CPU-s 0.0 0.040683 30.83MiB/CPU-s 1.17MiB/CPU-s 15.48KiB/CPU-s 0.0 0.037997 False False
datadog_agent_remap_datadog_logs 203.55KiB/CPU-s 0.6 100.00% 32.97MiB/CPU-s 1.49MiB/CPU-s 19.74KiB/CPU-s 0.0 0.04533 33.17MiB/CPU-s 1.23MiB/CPU-s 16.28KiB/CPU-s 0.0 0.037156 False False
http_to_http_json 78.46KiB/CPU-s 0.57 100.00% 13.53MiB/CPU-s 329.65KiB/CPU-s 4.25KiB/CPU-s 0.0 0.0238 13.6MiB/CPU-s 250.06KiB/CPU-s 3.23KiB/CPU-s 0.0 0.017952 False False
syslog_log2metric_splunk_hec_metrics 38.3KiB/CPU-s 0.4 100.00% 9.38MiB/CPU-s 293.9KiB/CPU-s 3.79KiB/CPU-s 0.0 0.030597 9.42MiB/CPU-s 323.53KiB/CPU-s 4.18KiB/CPU-s 0.0 0.033549 False False
datadog_agent_remap_datadog_logs_acks 116.11KiB/CPU-s 0.35 100.00% 32.85MiB/CPU-s 1.44MiB/CPU-s 19.09KiB/CPU-s 0.0 0.043975 32.97MiB/CPU-s 1.4MiB/CPU-s 18.49KiB/CPU-s 0.0 0.042441 False False
enterprise_http_to_http 8.23KiB/CPU-s 0.06 88.59% 13.61MiB/CPU-s 333.0KiB/CPU-s 4.3KiB/CPU-s 0.0 0.023889 13.62MiB/CPU-s 227.98KiB/CPU-s 2.94KiB/CPU-s 0.0 0.016345 False False
splunk_hec_to_splunk_hec_logs_noack 7.07KiB/CPU-s 0.05 92.15% 13.62MiB/CPU-s 245.5KiB/CPU-s 3.17KiB/CPU-s 0.0 0.017604 13.63MiB/CPU-s 191.76KiB/CPU-s 2.48KiB/CPU-s 0.0 0.013743 False False
splunk_hec_indexer_ack_blackhole 751.34B/CPU-s 0.01 12.81% 13.62MiB/CPU-s 254.22KiB/CPU-s 3.28KiB/CPU-s 0.0 0.01823 13.62MiB/CPU-s 244.62KiB/CPU-s 3.16KiB/CPU-s 0.0 0.017541 False False
file_to_blackhole 2.59KiB/CPU-s 0.0 9.57% 54.49MiB/CPU-s 1.15MiB/CPU-s 15.14KiB/CPU-s 0.0 0.021037 54.49MiB/CPU-s 1.16MiB/CPU-s 15.37KiB/CPU-s 0.0 0.021371 False False
fluent_elasticsearch -42.89B/CPU-s -0.0 6.19% 45.41MiB/CPU-s 30.04KiB/CPU-s 392.61B/CPU-s 0.0 0.000646 45.41MiB/CPU-s 29.76KiB/CPU-s 388.81B/CPU-s 0.0 0.00064 False False
splunk_hec_to_splunk_hec_logs_acks -1.15KiB/CPU-s -0.01 13.97% 13.62MiB/CPU-s 354.11KiB/CPU-s 4.57KiB/CPU-s 0.0 0.025393 13.62MiB/CPU-s 360.05KiB/CPU-s 4.65KiB/CPU-s 0.0 0.025821 False False
http_to_http_noack -2.33KiB/CPU-s -0.02 33.21% 13.62MiB/CPU-s 283.0KiB/CPU-s 3.65KiB/CPU-s 0.0 0.020295 13.61MiB/CPU-s 310.48KiB/CPU-s 4.01KiB/CPU-s 0.0 0.02227 False False
http_to_http_acks -7.03KiB/CPU-s -0.13 10.87% 5.28MiB/CPU-s 2.74MiB/CPU-s 36.19KiB/CPU-s 0.0 0.518315 5.28MiB/CPU-s 2.77MiB/CPU-s 36.58KiB/CPU-s 0.0 0.524843 True False
syslog_splunk_hec_logs -56.52KiB/CPU-s -0.62 100.00% 8.9MiB/CPU-s 257.02KiB/CPU-s 3.32KiB/CPU-s 0.0 0.028189 8.85MiB/CPU-s 325.06KiB/CPU-s 4.19KiB/CPU-s 0.0 0.035875 False False
splunk_hec_route_s3 -125.24KiB/CPU-s -1.04 100.00% 11.71MiB/CPU-s 546.13KiB/CPU-s 7.04KiB/CPU-s 0.0 0.045549 11.59MiB/CPU-s 726.76KiB/CPU-s 9.38KiB/CPU-s 0.0 0.061254 False False
otlp_grpc_to_blackhole -11.63KiB/CPU-s -1.09 100.00% 1.04MiB/CPU-s 41.96KiB/CPU-s 554.59B/CPU-s 0.0 0.039443 1.03MiB/CPU-s 53.5KiB/CPU-s 707.02B/CPU-s 0.0 0.050848 False False
otlp_http_to_blackhole -17.42KiB/CPU-s -1.1 100.00% 1.54MiB/CPU-s 109.72KiB/CPU-s 1.42KiB/CPU-s 0.0 0.06935 1.53MiB/CPU-s 123.28KiB/CPU-s 1.59KiB/CPU-s 0.0 0.078782 False False
socket_to_socket_blackhole -235.35KiB/CPU-s -1.72 100.00% 13.34MiB/CPU-s 263.12KiB/CPU-s 3.4KiB/CPU-s 0.0 0.019259 13.11MiB/CPU-s 293.23KiB/CPU-s 3.79KiB/CPU-s 0.0 0.021839 False False
syslog_log2metric_humio_metrics -165.51KiB/CPU-s -2.71 100.00% 5.97MiB/CPU-s 329.11KiB/CPU-s 4.25KiB/CPU-s 0.0 0.053852 5.81MiB/CPU-s 314.23KiB/CPU-s 4.06KiB/CPU-s 0.0 0.052848 False False
syslog_regex_logs2metric_ddmetrics -300.05KiB/CPU-s -7.88 100.00% 3.72MiB/CPU-s 418.65KiB/CPU-s 5.4KiB/CPU-s 0.0 0.109936 3.43MiB/CPU-s 357.02KiB/CPU-s 4.61KiB/CPU-s 0.0 0.101771 True False

github-actions[bot] avatar Jan 14 '23 23:01 github-actions[bot]

Uploaded requested changes.

zamazan4ik avatar Jan 17 '23 17:01 zamazan4ik

Regression Test Results

Run ID: 2038c497-dca4-4586-8d4b-f0f6b97b15e9
Baseline: 2b446f7a434a74bd7d65b930797fc2f518540320
Comparison: 804684ee584bb88c4187ffa5ce49f3f01486775b
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
http_text_to_http_json 2.04MiB/CPU-s 8.74 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json 2.04MiB/CPU-s 8.74 100.00% 23.37MiB/CPU-s 898.59KiB/CPU-s 11.6KiB/CPU-s 0.0 0.037539 25.42MiB/CPU-s 546.36KiB/CPU-s 7.05KiB/CPU-s 0.0 0.02099 False False
datadog_agent_remap_datadog_logs 813.63KiB/CPU-s 2.38 100.00% 33.45MiB/CPU-s 1.71MiB/CPU-s 22.61KiB/CPU-s 0.0 0.051175 34.24MiB/CPU-s 1.22MiB/CPU-s 16.08KiB/CPU-s 0.0 0.035546 False False
socket_to_socket_blackhole 252.03KiB/CPU-s 1.89 100.00% 13.04MiB/CPU-s 520.0KiB/CPU-s 6.71KiB/CPU-s 0.0 0.038937 13.29MiB/CPU-s 423.19KiB/CPU-s 5.47KiB/CPU-s 0.0 0.031101 False False
syslog_log2metric_humio_metrics 115.18KiB/CPU-s 1.84 100.00% 6.13MiB/CPU-s 131.19KiB/CPU-s 1.69KiB/CPU-s 0.0 0.020901 6.24MiB/CPU-s 91.07KiB/CPU-s 1.18KiB/CPU-s 0.0 0.014248 False False
syslog_loki 153.08KiB/CPU-s 1.7 100.00% 8.78MiB/CPU-s 338.34KiB/CPU-s 4.37KiB/CPU-s 0.0 0.037608 8.93MiB/CPU-s 384.15KiB/CPU-s 4.96KiB/CPU-s 0.0 0.041985 False False
http_to_http_acks 64.02KiB/CPU-s 1.19 78.16% 5.24MiB/CPU-s 2.77MiB/CPU-s 36.64KiB/CPU-s 0.0 0.528684 5.31MiB/CPU-s 2.79MiB/CPU-s 36.92KiB/CPU-s 0.0 0.526467 True False
syslog_regex_logs2metric_ddmetrics 35.01KiB/CPU-s 0.97 99.99% 3.51MiB/CPU-s 506.78KiB/CPU-s 6.54KiB/CPU-s 0.0 0.140913 3.55MiB/CPU-s 472.23KiB/CPU-s 6.1KiB/CPU-s 0.0 0.13004 True False
splunk_hec_route_s3 83.57KiB/CPU-s 0.69 100.00% 11.82MiB/CPU-s 520.61KiB/CPU-s 6.72KiB/CPU-s 0.0 0.043011 11.9MiB/CPU-s 521.15KiB/CPU-s 6.73KiB/CPU-s 0.0 0.042761 False False
otlp_grpc_to_blackhole 1.36KiB/CPU-s 0.13 87.38% 1.04MiB/CPU-s 44.64KiB/CPU-s 590.04B/CPU-s 0.0 0.041891 1.04MiB/CPU-s 52.24KiB/CPU-s 690.26B/CPU-s 0.0 0.048957 False False
datadog_agent_remap_blackhole 31.59KiB/CPU-s 0.1 79.16% 30.61MiB/CPU-s 1.56MiB/CPU-s 20.64KiB/CPU-s 0.0 0.050994 30.64MiB/CPU-s 1.08MiB/CPU-s 14.31KiB/CPU-s 0.0 0.035331 False False
http_to_http_noack 4.13KiB/CPU-s 0.03 62.21% 13.62MiB/CPU-s 286.16KiB/CPU-s 3.69KiB/CPU-s 0.0 0.020523 13.62MiB/CPU-s 222.54KiB/CPU-s 2.87KiB/CPU-s 0.0 0.015955 False False
enterprise_http_to_http 4.01KiB/CPU-s 0.03 55.87% 13.61MiB/CPU-s 309.43KiB/CPU-s 3.99KiB/CPU-s 0.0 0.022194 13.62MiB/CPU-s 258.98KiB/CPU-s 3.34KiB/CPU-s 0.0 0.01857 False False
splunk_hec_indexer_ack_blackhole 793.94B/CPU-s 0.01 13.42% 13.62MiB/CPU-s 253.15KiB/CPU-s 3.27KiB/CPU-s 0.0 0.018155 13.62MiB/CPU-s 249.95KiB/CPU-s 3.22KiB/CPU-s 0.0 0.017924 False False
splunk_hec_to_splunk_hec_logs_noack 1.23KiB/CPU-s 0.01 26.93% 13.62MiB/CPU-s 199.7KiB/CPU-s 2.58KiB/CPU-s 0.0 0.014313 13.63MiB/CPU-s 191.6KiB/CPU-s 2.47KiB/CPU-s 0.0 0.013731 False False
fluent_elasticsearch -176.15B/CPU-s -0.0 24.69% 45.41MiB/CPU-s 30.33KiB/CPU-s 396.64B/CPU-s 0.0 0.000652 45.41MiB/CPU-s 30.25KiB/CPU-s 395.44B/CPU-s 0.0 0.00065 False False
splunk_hec_to_splunk_hec_logs_acks -107.15B/CPU-s -0.0 1.31% 13.62MiB/CPU-s 353.45KiB/CPU-s 4.56KiB/CPU-s 0.0 0.025348 13.62MiB/CPU-s 343.91KiB/CPU-s 4.44KiB/CPU-s 0.0 0.024664 False False
http_to_http_json -198.82B/CPU-s -0.0 4.02% 13.62MiB/CPU-s 210.17KiB/CPU-s 2.71KiB/CPU-s 0.0 0.015065 13.62MiB/CPU-s 211.86KiB/CPU-s 2.73KiB/CPU-s 0.0 0.015186 False False
file_to_blackhole -10.24KiB/CPU-s -0.02 34.87% 54.49MiB/CPU-s 1.15MiB/CPU-s 15.17KiB/CPU-s 0.0 0.021075 54.48MiB/CPU-s 1.28MiB/CPU-s 16.84KiB/CPU-s 0.0 0.023425 False False
datadog_agent_remap_datadog_logs_acks -7.62KiB/CPU-s -0.02 30.19% 34.08MiB/CPU-s 974.43KiB/CPU-s 12.57KiB/CPU-s 0.0 0.027918 34.08MiB/CPU-s 1.14MiB/CPU-s 15.08KiB/CPU-s 0.0 0.03348 False False
syslog_splunk_hec_logs -92.97KiB/CPU-s -1.01 100.00% 9.01MiB/CPU-s 236.57KiB/CPU-s 3.05KiB/CPU-s 0.0 0.025627 8.92MiB/CPU-s 253.25KiB/CPU-s 3.27KiB/CPU-s 0.0 0.027713 False False
syslog_humio_logs -123.59KiB/CPU-s -1.31 100.00% 9.18MiB/CPU-s 195.38KiB/CPU-s 2.52KiB/CPU-s 0.0 0.02078 9.06MiB/CPU-s 357.54KiB/CPU-s 4.62KiB/CPU-s 0.0 0.038532 False False
datadog_agent_remap_blackhole_acks -421.43KiB/CPU-s -1.32 100.00% 31.25MiB/CPU-s 1.13MiB/CPU-s 14.93KiB/CPU-s 0.0 0.036124 30.84MiB/CPU-s 1.44MiB/CPU-s 18.99KiB/CPU-s 0.0 0.046579 False False
otlp_http_to_blackhole -26.17KiB/CPU-s -1.66 100.00% 1.54MiB/CPU-s 108.38KiB/CPU-s 1.4KiB/CPU-s 0.0 0.068832 1.51MiB/CPU-s 129.51KiB/CPU-s 1.67KiB/CPU-s 0.0 0.08364 False False
syslog_log2metric_splunk_hec_metrics -228.86KiB/CPU-s -2.41 100.00% 9.26MiB/CPU-s 412.15KiB/CPU-s 5.32KiB/CPU-s 0.0 0.043457 9.04MiB/CPU-s 425.34KiB/CPU-s 5.49KiB/CPU-s 0.0 0.045957 False False

github-actions[bot] avatar Jan 17 '23 17:01 github-actions[bot]

It looks like the changes made to website/cue/reference/components/sinks/base/clickhouse.cue are not in the latest diff. In this PR I think it might be better to explicitly edit the cue files to add documentation for the new method. As a side note- the doc comment for "Defaults to false using JSONEachRow over HTTP" will eventually need to be improved since it'll be user facing.

Hmm, I do not understand, what should I fix in the PR to resolve your comment. :(

zamazan4ik avatar Jan 17 '23 21:01 zamazan4ik

In this PR I think it might be better to explicitly edit the cue files to add documentation for the new method.

As far as I see, the documentation regarding native clickhouse protocol is already updated (added fields regarding use_native_proto and required corresponding fields). Did you mean any other piece of the documentation?

zamazan4ik avatar Jan 17 '23 22:01 zamazan4ik

Regression Test Results

Run ID: 9a9c0d57-8841-4704-b0cd-7975084c7344
Baseline: b1fc33c8bf11a9c3cd8a89db65c59e346b578cb0
Comparison: 6231a33c81d568e25a5e77998facfa8fb0a544a5
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
http_text_to_http_json 1.32MiB/CPU-s 5.64 100.00%
otlp_http_to_blackhole -109.02KiB/CPU-s -7.15 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json 1.32MiB/CPU-s 5.64 100.00% 23.49MiB/CPU-s 769.4KiB/CPU-s 9.93KiB/CPU-s 0.0 0.031991 24.81MiB/CPU-s 635.15KiB/CPU-s 8.19KiB/CPU-s 0.0 0.024999 False False
socket_to_socket_blackhole 611.7KiB/CPU-s 4.63 100.00% 12.9MiB/CPU-s 417.65KiB/CPU-s 5.39KiB/CPU-s 0.0 0.031616 13.5MiB/CPU-s 268.07KiB/CPU-s 3.46KiB/CPU-s 0.0 0.019395 False False
datadog_agent_remap_datadog_logs_acks 1.43MiB/CPU-s 4.43 100.00% 32.23MiB/CPU-s 2.11MiB/CPU-s 27.83KiB/CPU-s 0.0 0.065321 33.66MiB/CPU-s 1.05MiB/CPU-s 13.82KiB/CPU-s 0.0 0.031052 False False
syslog_loki 246.69KiB/CPU-s 2.78 100.00% 8.67MiB/CPU-s 263.93KiB/CPU-s 3.41KiB/CPU-s 0.0 0.029713 8.91MiB/CPU-s 181.72KiB/CPU-s 2.35KiB/CPU-s 0.0 0.019904 False False
datadog_agent_remap_datadog_logs 783.21KiB/CPU-s 2.34 100.00% 32.63MiB/CPU-s 1.73MiB/CPU-s 22.88KiB/CPU-s 0.0 0.053066 33.4MiB/CPU-s 1.61MiB/CPU-s 21.33KiB/CPU-s 0.0 0.04835 False False
syslog_log2metric_splunk_hec_metrics 216.2KiB/CPU-s 2.31 100.00% 9.16MiB/CPU-s 261.48KiB/CPU-s 3.38KiB/CPU-s 0.0 0.027885 9.37MiB/CPU-s 313.34KiB/CPU-s 4.05KiB/CPU-s 0.0 0.032663 False False
otlp_grpc_to_blackhole 18.5KiB/CPU-s 1.79 100.00% 1.01MiB/CPU-s 44.79KiB/CPU-s 592.05B/CPU-s 0.0 0.04327 1.03MiB/CPU-s 53.75KiB/CPU-s 710.17B/CPU-s 0.0 0.051013 False False
http_to_http_json 75.31KiB/CPU-s 0.54 100.00% 13.51MiB/CPU-s 356.44KiB/CPU-s 4.6KiB/CPU-s 0.0 0.025771 13.58MiB/CPU-s 281.82KiB/CPU-s 3.64KiB/CPU-s 0.0 0.020265 False False
syslog_humio_logs 46.8KiB/CPU-s 0.51 100.00% 9.0MiB/CPU-s 242.56KiB/CPU-s 3.13KiB/CPU-s 0.0 0.026321 9.04MiB/CPU-s 310.63KiB/CPU-s 4.01KiB/CPU-s 0.0 0.033537 False False
syslog_regex_logs2metric_ddmetrics 17.54KiB/CPU-s 0.48 97.40% 3.56MiB/CPU-s 421.16KiB/CPU-s 5.44KiB/CPU-s 0.0 0.115402 3.58MiB/CPU-s 441.57KiB/CPU-s 5.7KiB/CPU-s 0.0 0.120416 True False
enterprise_http_to_http 8.29KiB/CPU-s 0.06 92.51% 13.61MiB/CPU-s 304.07KiB/CPU-s 3.92KiB/CPU-s 0.0 0.02181 13.62MiB/CPU-s 193.35KiB/CPU-s 2.5KiB/CPU-s 0.0 0.01386 False False
splunk_hec_to_splunk_hec_logs_acks 3.17KiB/CPU-s 0.02 36.45% 13.61MiB/CPU-s 381.53KiB/CPU-s 4.92KiB/CPU-s 0.0 0.027366 13.62MiB/CPU-s 350.79KiB/CPU-s 4.53KiB/CPU-s 0.0 0.025155 False False
splunk_hec_to_splunk_hec_logs_noack 2.28KiB/CPU-s 0.02 46.59% 13.62MiB/CPU-s 210.5KiB/CPU-s 2.72KiB/CPU-s 0.0 0.015089 13.63MiB/CPU-s 191.28KiB/CPU-s 2.47KiB/CPU-s 0.0 0.013708 False False
fluent_elasticsearch 191.53B/CPU-s 0.0 26.66% 45.41MiB/CPU-s 30.43KiB/CPU-s 397.47B/CPU-s 0.0 0.000654 45.41MiB/CPU-s 30.45KiB/CPU-s 397.67B/CPU-s 0.0 0.000655 False False
splunk_hec_indexer_ack_blackhole 470.61B/CPU-s 0.0 8.14% 13.62MiB/CPU-s 247.58KiB/CPU-s 3.19KiB/CPU-s 0.0 0.017753 13.62MiB/CPU-s 245.29KiB/CPU-s 3.16KiB/CPU-s 0.0 0.017588 False False
file_to_blackhole -5.16KiB/CPU-s -0.01 19.33% 54.49MiB/CPU-s 1.05MiB/CPU-s 13.89KiB/CPU-s 0.0 0.0193 54.49MiB/CPU-s 1.2MiB/CPU-s 15.86KiB/CPU-s 0.0 0.022059 False False
http_to_http_noack -2.13KiB/CPU-s -0.02 30.53% 13.62MiB/CPU-s 284.12KiB/CPU-s 3.67KiB/CPU-s 0.0 0.020376 13.61MiB/CPU-s 310.78KiB/CPU-s 4.01KiB/CPU-s 0.0 0.022292 False False
datadog_agent_remap_blackhole_acks -32.83KiB/CPU-s -0.1 83.02% 31.0MiB/CPU-s 1.51MiB/CPU-s 19.99KiB/CPU-s 0.0 0.04878 30.97MiB/CPU-s 1016.5KiB/CPU-s 13.12KiB/CPU-s 0.0 0.032051 False False
http_to_http_acks -35.87KiB/CPU-s -0.67 52.02% 5.26MiB/CPU-s 2.7MiB/CPU-s 35.63KiB/CPU-s 0.0 0.512975 5.22MiB/CPU-s 2.74MiB/CPU-s 36.14KiB/CPU-s 0.0 0.523772 True False
syslog_splunk_hec_logs -159.3KiB/CPU-s -1.75 100.00% 8.88MiB/CPU-s 144.32KiB/CPU-s 1.86KiB/CPU-s 0.0 0.015867 8.73MiB/CPU-s 492.18KiB/CPU-s 6.35KiB/CPU-s 0.0 0.055074 False False
syslog_log2metric_humio_metrics -118.54KiB/CPU-s -1.96 100.00% 5.92MiB/CPU-s 377.51KiB/CPU-s 4.87KiB/CPU-s 0.0 0.062308 5.8MiB/CPU-s 309.91KiB/CPU-s 4.0KiB/CPU-s 0.0 0.052173 False False
datadog_agent_remap_blackhole -988.67KiB/CPU-s -3.11 100.00% 31.07MiB/CPU-s 1.35MiB/CPU-s 17.82KiB/CPU-s 0.0 0.043381 30.11MiB/CPU-s 1.74MiB/CPU-s 23.06KiB/CPU-s 0.0 0.057948 False False
splunk_hec_route_s3 -405.93KiB/CPU-s -3.41 100.00% 11.62MiB/CPU-s 640.37KiB/CPU-s 8.26KiB/CPU-s 0.0 0.053831 11.22MiB/CPU-s 727.98KiB/CPU-s 9.4KiB/CPU-s 0.0 0.063357 False False
otlp_http_to_blackhole -109.02KiB/CPU-s -7.15 100.00% 1.49MiB/CPU-s 125.48KiB/CPU-s 1.62KiB/CPU-s 0.0 0.082269 1.38MiB/CPU-s 413.57KiB/CPU-s 5.33KiB/CPU-s 0.0 0.292021 True False

github-actions[bot] avatar Jan 17 '23 22:01 github-actions[bot]

In this PR I think it might be better to explicitly edit the cue files to add documentation for the new method.

As far as I see, the documentation regarding native clickhouse protocol is already updated (added fields regarding use_native_proto and required corresponding fields). Did you mean any other piece of the documentation?

Don't mind me! I missed that this sink is already using the autogenerated docs. Safely ignore :)

neuronull avatar Jan 18 '23 20:01 neuronull

No worries :) Just let me know, what should I fix next in the PR :)

zamazan4ik avatar Jan 18 '23 20:01 zamazan4ik

It looks like there is a clippy error with a redundant closure. Otherwise CI is happy.

I've verified / marked the issues that I / @bruceg opened from @caibirdme 's original PR that have been fixed in this port of that. And you've now satisfied the MetaDescriptive constraint.

At this point the changes look good to me.

I think another set of eyes like @davidhuie-dd / @bruceg with another once-over would be a good idea but this is shaping up nicely. Thanks again @zamazan4ik

neuronull avatar Jan 18 '23 20:01 neuronull

It looks like there is a clippy error with a redundant closure.

I am not sure, how should I fix the error, since if I remove |t| closure, how could I get t variable to pass into the CHValue::Int64 function?

Thanks for the review!

zamazan4ik avatar Jan 18 '23 21:01 zamazan4ik

I am not sure, how should I fix the error, since if I remove |t| closure, how could I get t variable to pass into the CHValue::Int64 function?

The clippy hint covers it IMO. It's a neat feature where if the closure is just passing in the param to a function the compiler can ascertain that's what you want to do if the function args are aligned.

error: redundant closure
Error:    --> src/sinks/clickhouse/native/convert.rs:122:43
    |
122 |         SqlType::Int64 => into_integer(v, |t| CHValue::Int64(t)),
    |                                           ^^^^^^^^^^^^^^^^^^^^^ help: replace the closure with the function itself: `CHValue::Int64`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_closure
note: the lint level is defined here
   --> src/lib.rs:7:9
    |
7   | #![deny(warnings)]
    |         ^^^^^^^^
    = note: `#[deny(clippy::redundant_closure)]` implied by `#[deny(warnings)]`

Essentially,

         SqlType::Int64 => into_integer(v, CHValue::Int64),

neuronull avatar Jan 18 '23 21:01 neuronull

Didn't know about such a neat feature. Thanks! Fixed.

zamazan4ik avatar Jan 18 '23 22:01 zamazan4ik

Regression Test Results

Run ID: 57af05e0-15f1-42e5-86de-72e52f44f435
Baseline: 614961588662ac4349fae9c3df6405c219a07107
Comparison: 5b1d1bd5b83821b5d91dc2e07dfd92c6b9afdf45
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
http_text_to_http_json 1.41MiB/CPU-s 5.94 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json 1.41MiB/CPU-s 5.94 100.00% 23.68MiB/CPU-s 796.45KiB/CPU-s 10.28KiB/CPU-s 0.0 0.032845 25.09MiB/CPU-s 984.11KiB/CPU-s 12.7KiB/CPU-s 0.0 0.038307 False False
syslog_log2metric_humio_metrics 263.07KiB/CPU-s 4.49 100.00% 5.72MiB/CPU-s 463.47KiB/CPU-s 5.98KiB/CPU-s 0.0 0.079174 5.97MiB/CPU-s 250.83KiB/CPU-s 3.24KiB/CPU-s 0.0 0.041006 False False
syslog_regex_logs2metric_ddmetrics 72.45KiB/CPU-s 1.92 100.00% 3.68MiB/CPU-s 389.5KiB/CPU-s 5.03KiB/CPU-s 0.0 0.103472 3.75MiB/CPU-s 436.36KiB/CPU-s 5.63KiB/CPU-s 0.0 0.113733 True False
otlp_grpc_to_blackhole 16.01KiB/CPU-s 1.52 100.00% 1.03MiB/CPU-s 46.61KiB/CPU-s 616.06B/CPU-s 0.0 0.044282 1.04MiB/CPU-s 47.01KiB/CPU-s 621.28B/CPU-s 0.0 0.043995 False False
splunk_hec_route_s3 138.09KiB/CPU-s 1.16 100.00% 11.61MiB/CPU-s 587.52KiB/CPU-s 7.58KiB/CPU-s 0.0 0.049413 11.75MiB/CPU-s 593.98KiB/CPU-s 7.67KiB/CPU-s 0.0 0.049382 False False
datadog_agent_remap_datadog_logs_acks 324.92KiB/CPU-s 0.94 100.00% 33.82MiB/CPU-s 1.17MiB/CPU-s 15.43KiB/CPU-s 0.0 0.034538 34.13MiB/CPU-s 1.56MiB/CPU-s 20.57KiB/CPU-s 0.0 0.045603 False False
syslog_loki 58.81KiB/CPU-s 0.65 100.00% 8.81MiB/CPU-s 203.71KiB/CPU-s 2.63KiB/CPU-s 0.0 0.022581 8.87MiB/CPU-s 220.12KiB/CPU-s 2.84KiB/CPU-s 0.0 0.024241 False False
datadog_agent_remap_blackhole_acks 170.67KiB/CPU-s 0.53 100.00% 31.41MiB/CPU-s 743.29KiB/CPU-s 9.6KiB/CPU-s 0.0 0.023105 31.58MiB/CPU-s 728.73KiB/CPU-s 9.41KiB/CPU-s 0.0 0.022533 False False
socket_to_socket_blackhole 68.98KiB/CPU-s 0.51 100.00% 13.15MiB/CPU-s 213.46KiB/CPU-s 2.76KiB/CPU-s 0.0 0.015846 13.22MiB/CPU-s 615.62KiB/CPU-s 7.95KiB/CPU-s 0.0 0.045467 False False
datadog_agent_remap_blackhole 97.02KiB/CPU-s 0.3 100.00% 31.1MiB/CPU-s 1.1MiB/CPU-s 14.49KiB/CPU-s 0.0 0.035241 31.19MiB/CPU-s 1.01MiB/CPU-s 13.3KiB/CPU-s 0.0 0.032268 False False
syslog_log2metric_splunk_hec_metrics 27.37KiB/CPU-s 0.28 100.00% 9.43MiB/CPU-s 198.44KiB/CPU-s 2.56KiB/CPU-s 0.0 0.020541 9.46MiB/CPU-s 367.62KiB/CPU-s 4.75KiB/CPU-s 0.0 0.037945 False False
enterprise_http_to_http 12.46KiB/CPU-s 0.09 98.33% 13.61MiB/CPU-s 355.38KiB/CPU-s 4.59KiB/CPU-s 0.0 0.025498 13.62MiB/CPU-s 191.45KiB/CPU-s 2.47KiB/CPU-s 0.0 0.013724 False False
http_to_http_noack 11.72KiB/CPU-s 0.08 94.88% 13.61MiB/CPU-s 387.63KiB/CPU-s 5.0KiB/CPU-s 0.0 0.027818 13.62MiB/CPU-s 258.23KiB/CPU-s 3.33KiB/CPU-s 0.0 0.018517 False False
datadog_agent_remap_datadog_logs 25.01KiB/CPU-s 0.07 50.62% 33.81MiB/CPU-s 2.16MiB/CPU-s 28.59KiB/CPU-s 0.0 0.064005 33.84MiB/CPU-s 1.72MiB/CPU-s 22.78KiB/CPU-s 0.0 0.050968 False False
http_to_http_acks 1.74KiB/CPU-s 0.03 2.68% 5.29MiB/CPU-s 2.77MiB/CPU-s 36.55KiB/CPU-s 0.0 0.523073 5.29MiB/CPU-s 2.77MiB/CPU-s 36.65KiB/CPU-s 0.0 0.52416 True False
splunk_hec_to_splunk_hec_logs_acks 3.51KiB/CPU-s 0.03 40.83% 13.61MiB/CPU-s 370.84KiB/CPU-s 4.78KiB/CPU-s 0.0 0.0266 13.62MiB/CPU-s 345.64KiB/CPU-s 4.46KiB/CPU-s 0.0 0.024786 False False
splunk_hec_to_splunk_hec_logs_noack 891.83B/CPU-s 0.01 19.16% 13.62MiB/CPU-s 202.77KiB/CPU-s 2.62KiB/CPU-s 0.0 0.014533 13.63MiB/CPU-s 190.54KiB/CPU-s 2.46KiB/CPU-s 0.0 0.013655 False False
fluent_elasticsearch -316.68B/CPU-s -0.0 37.28% 45.41MiB/CPU-s 39.85KiB/CPU-s 520.73B/CPU-s 0.0 0.000857 45.41MiB/CPU-s 30.04KiB/CPU-s 392.56B/CPU-s 0.0 0.000646 False False
file_to_blackhole -2.65KiB/CPU-s -0.0 9.76% 54.49MiB/CPU-s 1.13MiB/CPU-s 14.91KiB/CPU-s 0.0 0.020715 54.49MiB/CPU-s 1.18MiB/CPU-s 15.61KiB/CPU-s 0.0 0.021707 False False
splunk_hec_indexer_ack_blackhole 432.88B/CPU-s 0.0 7.34% 13.62MiB/CPU-s 253.21KiB/CPU-s 3.27KiB/CPU-s 0.0 0.018158 13.62MiB/CPU-s 250.13KiB/CPU-s 3.23KiB/CPU-s 0.0 0.017937 False False
syslog_humio_logs -5.4KiB/CPU-s -0.06 61.07% 9.04MiB/CPU-s 355.84KiB/CPU-s 4.59KiB/CPU-s 0.0 0.03842 9.04MiB/CPU-s 330.74KiB/CPU-s 4.27KiB/CPU-s 0.0 0.035731 False False
http_to_http_json -43.07KiB/CPU-s -0.31 100.00% 13.58MiB/CPU-s 288.62KiB/CPU-s 3.72KiB/CPU-s 0.0 0.020752 13.54MiB/CPU-s 344.29KiB/CPU-s 4.44KiB/CPU-s 0.0 0.024832 False False
otlp_http_to_blackhole -20.83KiB/CPU-s -1.32 100.00% 1.54MiB/CPU-s 111.43KiB/CPU-s 1.44KiB/CPU-s 0.0 0.070776 1.52MiB/CPU-s 123.06KiB/CPU-s 1.59KiB/CPU-s 0.0 0.079211 False False
syslog_splunk_hec_logs -171.55KiB/CPU-s -1.86 100.00% 9.01MiB/CPU-s 217.5KiB/CPU-s 2.81KiB/CPU-s 0.0 0.023562 8.85MiB/CPU-s 313.71KiB/CPU-s 4.05KiB/CPU-s 0.0 0.034627 False False

github-actions[bot] avatar Jan 18 '23 22:01 github-actions[bot]

Happy to see this work continuing since it seems like the interface offers improved performance.

Not sure about that, honestly :) As far as I know, the native tcp-based protocol is much more tweakable. Regarding performance, at least this article (warning: 3 years old) says that HTTP-based protocol could be even better from the QPS point of view: https://altinity.com/blog/clickhouse-in-the-storm-part-1

zamazan4ik avatar Jan 19 '23 01:01 zamazan4ik