vector icon indicating copy to clipboard operation
vector copied to clipboard

Published 20 hours ago • verified publisher icon vectordotdev

feat(new source): new `file_descriptor` source

Open mcasper opened this issue 2 years ago • 14 comments

This is a proposed solution to https://github.com/vectordotdev/vector/issues/13379

This PR introduces a file_descriptor source which is a generalized version of the stdin source that can read from any file descriptor. To prevent conflicts with stdin, both sources share a new Fd resource with the file descriptor number. The pipe source only works on unix platforms. This wouldn't make sense on Windows, since handles typically can't be transferred.

We considered naming this source unix_pipe to indicate its platform availability, but wanted to open it up to feedback from the maintainers.

mcasper avatar Jun 29 '22 21:06 mcasper

CLA assistant check
All committers have signed the CLA.

bits-bot avatar Jun 29 '22 21:06 bits-bot

Deploy Preview for vector-project ready!

Name Link
Latest commit 8d9556fc7706bf95936fc14a55b539c60da39a46
Latest deploy log https://app.netlify.com/sites/vector-project/deploys/62f199fc4535f40008a9797a
Deploy Preview https://deploy-preview-13389--vector-project.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

netlify[bot] avatar Jun 29 '22 21:06 netlify[bot]

Soak Test Results

Baseline: 78b0a76c8826d62ad71a2b323e27dbb3b322ed09 Comparison: 88c7115d99fff214f2f7ff53c066f2c2fe851ade Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_pipelines_blackhole_acks 16.78KiB 1.38 100.00% 1.19MiB 120.46KiB 2.45KiB 0 0.099052 1.2MiB 76.55KiB 1.56KiB 0 0.0620895 False False
splunk_hec_to_splunk_hec_logs_noack 18.29KiB 0.07 87.99% 23.82MiB 471.54KiB 9.63KiB 0 0.019326 23.84MiB 330.88KiB 6.75KiB 0 0.0135508 False False
splunk_hec_to_splunk_hec_logs_acks 11.46KiB 0.05 35.18% 23.74MiB 898.49KiB 18.27KiB 0 0.0369537 23.75MiB 846.41KiB 17.22KiB 0 0.0347952 False False
http_text_to_http_json 16.19KiB 0.04 58.73% 40.17MiB 691.84KiB 14.12KiB 0 0.0168155 40.19MiB 677.5KiB 13.83KiB 0 0.0164606 False False
splunk_hec_indexer_ack_blackhole 5.75KiB 0.02 18.10% 23.75MiB 885.69KiB 18.02KiB 0 0.0364055 23.76MiB 860.78KiB 17.52KiB 0 0.035373 False False
http_to_http_acks -32.43KiB -0.18 11.20% 17.82MiB 7.78MiB 162.75KiB 0 0.436687 17.79MiB 7.79MiB 162.82KiB 0 0.438092 True True
http_to_http_json -45.36KiB -0.19 99.94% 23.85MiB 323.45KiB 6.6KiB 0 0.013243 23.8MiB 561.3KiB 11.45KiB 0 0.0230243 False False
fluent_elasticsearch -181.28KiB -0.22 100.00% 79.47MiB 52.75KiB 1.07KiB 0 0.000648068 79.3MiB 1.65MiB 33.96KiB 0 0.020803 False False
file_to_blackhole -248.44KiB -0.25 88.93% 95.29MiB 4.03MiB 83.35KiB 0 0.0422692 95.05MiB 6.35MiB 131.57KiB 0 0.0668123 False False
http_to_http_noack -87.22KiB -0.36 99.94% 23.83MiB 513.77KiB 10.51KiB 0 0.0210515 23.74MiB 1.11MiB 23.08KiB 0 0.046616 False False
syslog_splunk_hec_logs -129.74KiB -0.77 100.00% 16.48MiB 805.65KiB 16.4KiB 0 0.0477231 16.36MiB 735.67KiB 15.0KiB 0 0.0439154 False False
http_pipelines_blackhole -15.63KiB -0.96 100.00% 1.59MiB 42.58KiB 890.57B 0 0.0261069 1.58MiB 130.21KiB 2.65KiB 0 0.080607 False False
syslog_log2metric_splunk_hec_metrics -212.86KiB -1.16 100.00% 17.9MiB 536.47KiB 10.93KiB 0 0.0292672 17.69MiB 690.33KiB 14.06KiB 0 0.0381041 False False
socket_to_socket_blackhole -217.33KiB -1.54 100.00% 13.8MiB 116.89KiB 2.39KiB 0 0.00827029 13.59MiB 452.99KiB 9.25KiB 0 0.0325499 False False
datadog_agent_remap_blackhole_acks -1.06MiB -1.63 100.00% 64.82MiB 4.4MiB 91.61KiB 0 0.0678546 63.76MiB 3.34MiB 69.68KiB 0 0.0523239 False False
syslog_humio_logs -275.02KiB -1.64 100.00% 16.37MiB 510.98KiB 10.43KiB 0 0.0304747 16.1MiB 567.4KiB 11.62KiB 0 0.0344038 False False
datadog_agent_remap_datadog_logs_acks -1.07MiB -1.68 100.00% 63.39MiB 4.08MiB 85.22KiB 0 0.0642968 62.33MiB 4.87MiB 101.37KiB 0 0.0781192 False False
datadog_agent_remap_datadog_logs -1.22MiB -1.86 100.00% 65.42MiB 1.1MiB 22.97KiB 0 0.0167509 64.21MiB 4.44MiB 92.37KiB 0 0.0690848 False False
http_pipelines_no_grok_blackhole -220.74KiB -1.86 100.00% 11.61MiB 49.42KiB 1.01KiB 0 0.00415639 11.39MiB 1.13MiB 23.56KiB 0 0.0992829 False False
splunk_hec_route_s3 -383.84KiB -1.95 100.00% 19.23MiB 2.25MiB 46.92KiB 0 0.117218 18.85MiB 2.27MiB 47.42KiB 0 0.120236 False False
syslog_regex_logs2metric_ddmetrics -282.95KiB -2.21 100.00% 12.52MiB 625.26KiB 12.74KiB 0 0.0487713 12.24MiB 774.08KiB 15.77KiB 0 0.0617424 False False
syslog_loki -361.82KiB -2.38 100.00% 14.84MiB 767.29KiB 15.7KiB 0 0.0504911 14.48MiB 896.73KiB 18.23KiB 0 0.0604489 False False
datadog_agent_remap_blackhole -1.56MiB -2.48 100.00% 62.69MiB 4.66MiB 97.14KiB 0 0.0743622 61.14MiB 4.03MiB 84.09KiB 0 0.0659016 False False
syslog_log2metric_humio_metrics -349.78KiB -2.54 100.00% 13.44MiB 442.72KiB 9.03KiB 0 0.032151 13.1MiB 605.5KiB 12.32KiB 0 0.045119 False False

github-actions[bot] avatar Jun 30 '22 00:06 github-actions[bot]

For extra context, we've successfully integrated this new "pipe" source into our application. Here's an example of how we setup the pipes and launch vector now:


[sources.first]
  type = "pipe"
  fd = 0

[sources.second]
  type = "pipe"
  fd = 3

[sources.third]
  type = "pipe"
  fd = 4

# ... [snip the rest of the file] ...


union UnixPipe {
    struct {
        int read;
        int write;
    } direction;
    int array[2];
};

struct VectorWritePipes {
    int this_process;
    int next_process;
    int third_process;
};

static void checked_dup2(int oldfd, int newfd) {
    if (newfd != dup2(oldfd, newfd)) {
        fprintf(stderr, "error: dup2 from %d to %d failed, errno=%d", oldfd, newfd, errno);
        exit(0xff);
    }
}

VectorWritePipes spawn_vector() {

    struct {
        UnixPipe this_process;
        UnixPipe next_process;
        UnixPipe third_process;
    } pipes;

    if (0 != pipe(pipes.this_process.array)) {
        fprintf(stderr, "error: pipe function 1 failed, errno=%d\n", errno);
        exit(0xff);
    }
    if (0 != pipe(pipes.next_process.array)) {
        fprintf(stderr, "error: pipe function 2 failed, errno=%d\n", errno);
        exit(0xff);
    }
    if (0 != pipe(pipes.third_process.array)) {
        fprintf(stderr, "error: pipe function 3 failed, errno=%d\n", errno);
        exit(0xff);
    }

    // Make sure no child processes keeps our pipe alive
    if (0 != fcntl(pipes.this_process.direction.write, F_SETFD, FD_CLOEXEC)) {
        fprintf(stderr, "error: fcntl CLOEXEC on pipe failed, errno=%d\n", errno);
        exit(0xff);
    }

    pid_t pid = fork();
    if (pid == -1) {
        fprintf(stderr, "error: fork (meant for vector process) failed, errno=%d\n", errno);
        exit(0xff);
    }
    if (pid != 0) {
        close(pipes.this_process.direction.read);
        close(pipes.next_process.direction.read);
        close(pipes.third_process.direction.read);
        return {
            .this_process = pipes.this_process.direction.write,
            .next_process = pipes.next_process.direction.write,
            .third_process = pipes.third_process.direction.write,
        };
    }

    // close(pipes.this_process.direction.write); this will be closed automatically because of CLOEXEC
    close(pipes.next_process.direction.write);
    close(pipes.third_process.direction.write);

    checked_dup2(pipes.this_process.direction.read, 0);

    // We're about to set the next_process read fd to 3, make sure we don't
    // override the "third_process" read fd
    if (pipes.third_process.direction.read == 3) {
        checked_dup2(pipes.third_process.direction.read, 5);
        pipes.third_process.direction.read = 5;
    }
    checked_dup2(pipes.next_process.direction.read, 3);

    checked_dup2(pipes.third_process.direction.read, 4);

    const char *vector_argv[] = {
        "/path/to/vector",
        "--config",
        "/path/to/vector-config.toml",
        nullptr,
    };

    execve(vector_argv[0], (char *const*)vector_argv, environ);
    fprintf(stderr, "error: execve '%s' failed, errno=%d", vector_exe, errno);
    exit(0xff);
}

marler8997 avatar Jun 30 '22 21:06 marler8997

Thanks for this contribution @mcasper ! We'll get it reviewed shortly.

jszwedko avatar Jul 05 '22 16:07 jszwedko

I'm going to run CI even though there might be more changes coming in , because CI may catch some things.

neuronull avatar Jul 19 '22 21:07 neuronull

Soak Test Results

Baseline: 3b71e0085dc40306163825e4384e22c367f3eae6 Comparison: 5b9c87060ca87f73e7e3223593c2afead03beead Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_humio_metrics 351.05KiB 2.71 100.00% 12.64MiB 206.0KiB 4.21KiB 0 0.0159134 12.98MiB 555.9KiB 11.31KiB 0 0.0418096 False False
syslog_loki 293.09KiB 2.06 100.00% 13.91MiB 520.9KiB 10.67KiB 0 0.0365662 14.19MiB 828.22KiB 16.84KiB 0 0.0569677 False False
syslog_regex_logs2metric_ddmetrics 264.73KiB 2.01 100.00% 12.84MiB 667.08KiB 13.59KiB 0 0.0507223 13.1MiB 641.46KiB 13.07KiB 0 0.0478118 False False
splunk_hec_route_s3 382.36KiB 2.01 100.00% 18.55MiB 2.31MiB 48.08KiB 0 0.124244 18.93MiB 2.28MiB 47.54KiB 0 0.120203 False False
datadog_agent_remap_blackhole_acks 1008.12KiB 1.59 100.00% 61.84MiB 6.26MiB 130.52KiB 0 0.101291 62.82MiB 5.77MiB 120.56KiB 0 0.0917896 False False
socket_to_socket_blackhole 183.75KiB 1.34 100.00% 13.43MiB 147.62KiB 3.01KiB 0 0.0107316 13.61MiB 186.2KiB 3.8KiB 0 0.0133581 False False
http_to_http_acks 242.89KiB 1.29 70.33% 18.35MiB 8.06MiB 168.56KiB 0 0.43921 18.59MiB 7.68MiB 160.48KiB 0 0.413152 True True
datadog_agent_remap_blackhole 653.37KiB 1.01 100.00% 63.36MiB 4.61MiB 96.06KiB 0 0.0727357 63.99MiB 3.73MiB 77.88KiB 0 0.0583089 False False
syslog_splunk_hec_logs 83.92KiB 0.48 100.00% 17.24MiB 741.08KiB 15.08KiB 0 0.0419756 17.32MiB 649.68KiB 13.24KiB 0 0.036624 False False
http_pipelines_blackhole_acks 3.88KiB 0.32 87.72% 1.18MiB 104.46KiB 2.13KiB 0 0.0865799 1.18MiB 65.48KiB 1.34KiB 0 0.0540974 False False
http_text_to_http_json 110.11KiB 0.28 100.00% 38.26MiB 835.74KiB 17.06KiB 0 0.0213295 38.36MiB 830.28KiB 16.95KiB 0 0.0211307 False False
syslog_humio_logs 45.5KiB 0.26 99.93% 16.97MiB 460.11KiB 9.4KiB 0 0.0264793 17.01MiB 465.17KiB 9.52KiB 0 0.0267007 False False
http_pipelines_blackhole 3.18KiB 0.21 66.77% 1.51MiB 84.8KiB 1.73KiB 0 0.0547572 1.52MiB 136.83KiB 2.79KiB 0 0.0881679 False False
datadog_agent_remap_datadog_logs_acks 115.54KiB 0.18 68.42% 63.57MiB 3.21MiB 67.14KiB 0 0.0505221 63.68MiB 4.49MiB 93.57KiB 0 0.0705726 False False
splunk_hec_to_splunk_hec_logs_noack 6.76KiB 0.03 48.51% 23.83MiB 385.34KiB 7.87KiB 0 0.0157866 23.84MiB 332.18KiB 6.78KiB 0 0.0136051 False False
syslog_log2metric_splunk_hec_metrics 4.7KiB 0.02 20.40% 18.87MiB 523.99KiB 10.68KiB 0 0.0271055 18.88MiB 723.59KiB 14.74KiB 0 0.0374215 False False
enterprise_http_to_http -2.86KiB -0.01 30.46% 23.85MiB 250.8KiB 5.12KiB 0 0.0102681 23.84MiB 255.19KiB 5.22KiB 0 0.0104493 False False
splunk_hec_to_splunk_hec_logs_acks -1.32KiB -0.01 4.24% 23.75MiB 863.89KiB 17.57KiB 0 0.0355158 23.75MiB 861.77KiB 17.53KiB 0 0.0354305 False False
splunk_hec_indexer_ack_blackhole -10.18KiB -0.04 30.60% 23.76MiB 875.05KiB 17.81KiB 0 0.0359624 23.75MiB 922.56KiB 18.77KiB 0 0.0379308 False False
file_to_blackhole -101.89KiB -0.1 68.89% 95.35MiB 3.1MiB 64.24KiB 0 0.0324895 95.25MiB 3.72MiB 77.4KiB 0 0.0390923 False False
fluent_elasticsearch -96.44KiB -0.12 100.00% 79.47MiB 53.79KiB 1.09KiB 0 0.000660842 79.38MiB 1.01MiB 20.9KiB 0 0.012779 False False
http_to_http_json -30.12KiB -0.12 98.32% 23.84MiB 347.76KiB 7.1KiB 0 0.0142404 23.81MiB 509.01KiB 10.4KiB 0 0.0208694 False False
http_to_http_noack -105.81KiB -0.43 100.00% 23.84MiB 406.32KiB 8.31KiB 0 0.0166428 23.73MiB 1.15MiB 23.96KiB 0 0.0484141 False False
datadog_agent_remap_datadog_logs -474.61KiB -0.76 99.98% 61.07MiB 3.08MiB 64.62KiB 0 0.0504796 60.6MiB 5.25MiB 109.29KiB 0 0.0866026 False False
http_pipelines_no_grok_blackhole -96.95KiB -0.83 100.00% 11.35MiB 267.97KiB 5.47KiB 0 0.0230606 11.25MiB 1.03MiB 21.36KiB 0 0.0911174 False False

github-actions[bot] avatar Jul 19 '22 23:07 github-actions[bot]

Soak Test Results

Baseline: 3b71e0085dc40306163825e4384e22c367f3eae6 Comparison: 60ba3b9506224f26a8d6771fd5bf131548f2552c Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
splunk_hec_route_s3 446.53KiB 2.28 100.00% 19.12MiB 2.25MiB 46.8KiB 0 0.117443 19.56MiB 2.11MiB 44.08KiB 0 0.107823 False False
datadog_agent_remap_datadog_logs_acks 1.38MiB 2.25 100.00% 61.21MiB 4.52MiB 94.48KiB 0 0.0738672 62.59MiB 5.03MiB 104.8KiB 0 0.0804299 False False
syslog_log2metric_humio_metrics 285.74KiB 2.13 100.00% 13.09MiB 389.59KiB 7.94KiB 0 0.0290566 13.37MiB 569.41KiB 11.59KiB 0 0.041582 False False
syslog_regex_logs2metric_ddmetrics 269.17KiB 2.08 100.00% 12.63MiB 650.38KiB 13.25KiB 0 0.0502937 12.89MiB 727.73KiB 14.81KiB 0 0.0551273 False False
http_pipelines_blackhole_acks 23.38KiB 2.08 100.00% 1.1MiB 128.42KiB 2.61KiB 0 0.114136 1.12MiB 96.13KiB 1.96KiB 0 0.0837037 False False
socket_to_socket_blackhole 261.28KiB 1.98 100.00% 12.88MiB 176.06KiB 3.59KiB 0 0.0133445 13.14MiB 152.48KiB 3.11KiB 0 0.0113328 False False
syslog_loki 270.01KiB 1.86 100.00% 14.2MiB 381.74KiB 7.82KiB 0 0.0262435 14.47MiB 768.13KiB 15.61KiB 0 0.0518447 False False
datadog_agent_remap_blackhole 1.1MiB 1.74 100.00% 63.03MiB 4.35MiB 90.67KiB 0 0.069033 64.13MiB 2.7MiB 56.43KiB 0 0.0421285 False False
datadog_agent_remap_blackhole_acks 854.51KiB 1.28 100.00% 65.34MiB 4.33MiB 90.09KiB 0 0.066181 66.18MiB 2.89MiB 60.52KiB 0 0.0437245 False False
syslog_humio_logs 85.23KiB 0.48 100.00% 17.52MiB 129.9KiB 2.65KiB 0 0.00724123 17.6MiB 115.65KiB 2.37KiB 0 0.00641604 False False
http_text_to_http_json 124.09KiB 0.32 100.00% 38.06MiB 815.02KiB 16.64KiB 0 0.0209065 38.18MiB 830.53KiB 16.96KiB 0 0.0212368 False False
splunk_hec_to_splunk_hec_logs_acks 11.17KiB 0.05 34.32% 23.74MiB 894.64KiB 18.19KiB 0 0.0367942 23.75MiB 852.07KiB 17.33KiB 0 0.0350275 False False
splunk_hec_to_splunk_hec_logs_noack 11.15KiB 0.05 69.24% 23.83MiB 422.51KiB 8.63KiB 0 0.0173138 23.84MiB 328.39KiB 6.7KiB 0 0.0134509 False False
enterprise_http_to_http 596.71B 0 6.32% 23.85MiB 254.22KiB 5.19KiB 0 0.0104093 23.85MiB 253.79KiB 5.2KiB 0 0.0103915 False False
file_to_blackhole -68.24KiB -0.07 54.53% 95.36MiB 2.86MiB 59.39KiB 0 0.0300353 95.29MiB 3.33MiB 69.31KiB 0 0.0349485 False False
syslog_splunk_hec_logs -11.7KiB -0.07 33.87% 17.02MiB 975.32KiB 19.83KiB 0 0.0559462 17.01MiB 876.42KiB 17.87KiB 0 0.050307 False False
http_to_http_json -23.59KiB -0.1 95.29% 23.84MiB 345.4KiB 7.05KiB 0 0.014143 23.82MiB 467.43KiB 9.55KiB 0 0.0191582 False False
splunk_hec_indexer_ack_blackhole -30.42KiB -0.13 74.85% 23.76MiB 857.29KiB 17.45KiB 0 0.0352235 23.73MiB 982.82KiB 19.98KiB 0 0.0404315 False False
syslog_log2metric_splunk_hec_metrics -39.25KiB -0.2 94.73% 19.09MiB 536.31KiB 10.93KiB 0 0.0274283 19.05MiB 837.49KiB 17.04KiB 0 0.0429176 False False
fluent_elasticsearch -269.87KiB -0.33 99.98% 79.47MiB 52.68KiB 1.06KiB 0 0.000647219 79.21MiB 3.56MiB 73.21KiB 0 0.0449186 False False
http_pipelines_blackhole -6.8KiB -0.41 98.69% 1.62MiB 21.54KiB 451.06B 0 0.0129641 1.62MiB 132.8KiB 2.7KiB 0 0.0802453 False False
http_to_http_noack -102.78KiB -0.42 100.00% 23.84MiB 254.34KiB 5.2KiB 0 0.0104145 23.74MiB 1.09MiB 22.72KiB 0 0.0458757 False False
http_to_http_acks -97.68KiB -0.52 31.09% 18.19MiB 8.2MiB 171.47KiB 0 0.450905 18.09MiB 8.32MiB 173.79KiB 0 0.459795 True True
datadog_agent_remap_datadog_logs -442.42KiB -0.71 100.00% 60.46MiB 2.15MiB 45.11KiB 0 0.0355997 60.03MiB 4.64MiB 96.55KiB 0 0.077235 False False
http_pipelines_no_grok_blackhole -215.22KiB -1.91 100.00% 11.0MiB 290.42KiB 5.93KiB 0 0.0257884 10.79MiB 1.09MiB 22.61KiB 0 0.100683 False False

github-actions[bot] avatar Jul 20 '22 00:07 github-actions[bot]

Soak Test Results

Baseline: ce65a860f8bec07aaae44cc9700626b0ea0d332c Comparison: 209c9cec7b0e06d19da5e17ae31e33e04c0e3ebc Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json 1.27MiB 3.44 100.00% 36.89MiB 1.09MiB 22.76KiB 0 0.0295194 38.15MiB 1.02MiB 21.31KiB 0 0.026708 False False
socket_to_socket_blackhole 267.06KiB 2.05 100.00% 12.75MiB 170.9KiB 3.49KiB 0 0.0130858 13.01MiB 139.38KiB 2.85KiB 0 0.0104589 False False
splunk_hec_route_s3 288.06KiB 1.48 100.00% 18.97MiB 2.32MiB 48.22KiB 0 0.122066 19.25MiB 2.15MiB 45.05KiB 0 0.111884 False False
http_pipelines_blackhole_acks 11.01KiB 0.96 99.99% 1.11MiB 103.4KiB 2.11KiB 0 0.0905639 1.13MiB 85.39KiB 1.74KiB 0 0.0740795 False False
syslog_log2metric_humio_metrics 118.42KiB 0.91 100.00% 12.75MiB 505.96KiB 10.33KiB 0 0.0387489 12.86MiB 599.63KiB 12.22KiB 0 0.0455097 False False
syslog_regex_logs2metric_ddmetrics 93.91KiB 0.71 100.00% 12.85MiB 715.46KiB 14.56KiB 0 0.0543557 12.94MiB 714.95KiB 14.56KiB 0 0.0539323 False False
datadog_agent_remap_datadog_logs 328.81KiB 0.53 99.86% 60.77MiB 2.15MiB 45.1KiB 0 0.0353917 61.09MiB 4.45MiB 92.76KiB 0 0.0729073 False False
syslog_loki 74.13KiB 0.5 100.00% 14.45MiB 149.14KiB 3.06KiB 0 0.0100758 14.52MiB 725.94KiB 14.76KiB 0 0.0488011 False False
syslog_humio_logs 82.24KiB 0.46 100.00% 17.29MiB 342.12KiB 6.98KiB 0 0.0193174 17.37MiB 361.95KiB 7.41KiB 0 0.0203429 False False
http_pipelines_blackhole 3.49KiB 0.21 84.89% 1.64MiB 10.23KiB 214.36B 0 0.00608491 1.65MiB 119.0KiB 2.42KiB 0 0.0706111 False False
syslog_log2metric_splunk_hec_metrics 38.24KiB 0.2 93.41% 18.75MiB 594.23KiB 12.11KiB 0 0.030945 18.79MiB 829.93KiB 16.89KiB 0 0.0431335 False False
syslog_splunk_hec_logs 29.06KiB 0.16 83.79% 17.21MiB 739.61KiB 15.06KiB 0 0.04195 17.24MiB 702.87KiB 14.33KiB 0 0.0398004 False False
http_to_http_acks 19.42KiB 0.11 6.04% 17.87MiB 8.79MiB 183.82KiB 0 0.491954 17.89MiB 8.54MiB 178.36KiB 0 0.477333 True True
splunk_hec_to_splunk_hec_logs_noack 5.65KiB 0.02 41.27% 23.83MiB 379.35KiB 7.74KiB 0 0.0155412 23.84MiB 340.17KiB 6.94KiB 0 0.013933 False False
datadog_agent_remap_blackhole_acks 8.68KiB 0.01 5.00% 63.3MiB 5.08MiB 105.79KiB 0 0.0802626 63.31MiB 4.27MiB 89.12KiB 0 0.0673571 False False
enterprise_http_to_http 90.11B 0 0.98% 23.85MiB 246.77KiB 5.04KiB 0 0.010104 23.85MiB 249.29KiB 5.1KiB 0 0.0102073 False False
splunk_hec_to_splunk_hec_logs_acks -943.46B -0 3.07% 23.76MiB 832.81KiB 16.94KiB 0 0.0342255 23.76MiB 831.62KiB 16.92KiB 0 0.0341778 False False
splunk_hec_indexer_ack_blackhole -10.21KiB -0.04 31.30% 23.75MiB 856.79KiB 17.43KiB 0 0.0352175 23.74MiB 903.67KiB 18.38KiB 0 0.0371599 False False
file_to_blackhole -90.92KiB -0.09 56.96% 95.34MiB 3.58MiB 74.26KiB 0 0.0375652 95.25MiB 4.24MiB 88.19KiB 0 0.0445539 False False
datadog_agent_remap_datadog_logs_acks -65.11KiB -0.1 46.10% 62.56MiB 2.47MiB 51.8KiB 0 0.0395388 62.49MiB 4.44MiB 92.45KiB 0 0.0710515 False False
http_to_http_json -39.86KiB -0.16 99.71% 23.84MiB 346.27KiB 7.07KiB 0 0.0141789 23.81MiB 555.98KiB 11.35KiB 0 0.0228031 False False
fluent_elasticsearch -185.54KiB -0.23 100.00% 79.47MiB 51.53KiB 1.04KiB 0 0.000633116 79.29MiB 1.59MiB 32.61KiB 0 0.0199882 False False
http_to_http_noack -86.94KiB -0.36 99.98% 23.84MiB 401.89KiB 8.22KiB 0 0.0164612 23.75MiB 1.05MiB 21.91KiB 0 0.0442206 False False
http_pipelines_no_grok_blackhole -86.56KiB -0.75 99.98% 11.32MiB 83.79KiB 1.71KiB 0 0.00722553 11.24MiB 1.12MiB 23.28KiB 0 0.0994908 False False
datadog_agent_remap_blackhole -1.42MiB -2.26 100.00% 62.8MiB 4.3MiB 89.64KiB 0 0.0685066 61.38MiB 3.25MiB 67.79KiB 0 0.0529496 False False

github-actions[bot] avatar Jul 20 '22 00:07 github-actions[bot]

@bruceg @neuronull I've taken a stab at addressing all of your feedback thus far, let me know what you think!

mcasper avatar Aug 02 '22 22:08 mcasper

@bruceg @neuronull I've taken a stab at addressing all of your feedback thus far, let me know what you think!

Thanks for tackling all those changes @mcasper ! It is looking great. I think there is one recommendation from Bruce remaining, about increasing the readability of the source() function (which I think is a good idea) , and would involve turning the inline closures into functions of that Trait.

I'll give Bruce a chance to review the new changes as well, and then we can kick off another CI round ~

neuronull avatar Aug 03 '22 14:08 neuronull

Soak Test Results

Baseline: ce92df232466e6b992d21cff303b61efaae488c4 Comparison: 5e6e3a7afa6e3d6a06f414ab4d3aaa287d22a879 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
socket_to_socket_blackhole 994.55KiB 4.29 100.00% 22.62MiB 254.6KiB 5.2KiB 0 0.0109896 23.59MiB 374.98KiB 7.66KiB 0 0.0155193 False False
http_text_to_http_json 1.53MiB 4.01 100.00% 38.23MiB 872.89KiB 17.82KiB 0 0.0222926 39.77MiB 793.03KiB 16.19KiB 0 0.0194714 False False
splunk_hec_route_s3 293.13KiB 1.56 100.00% 18.4MiB 2.42MiB 50.31KiB 0 0.131247 18.68MiB 2.28MiB 47.67KiB 0 0.122081 False False
datadog_agent_remap_datadog_logs_acks 818.02KiB 1.27 100.00% 63.01MiB 3.32MiB 69.29KiB 0 0.0526146 63.81MiB 4.51MiB 93.86KiB 0 0.0706473 False False
http_pipelines_blackhole_acks 12.76KiB 1.07 100.00% 1.17MiB 119.61KiB 2.43KiB 0 0.0999943 1.18MiB 55.33KiB 1.13KiB 0 0.0457666 False False
datadog_agent_remap_datadog_logs 536.47KiB 0.85 100.00% 61.94MiB 1.93MiB 40.48KiB 0 0.0311711 62.46MiB 4.32MiB 90.01KiB 0 0.0691571 False False
syslog_regex_logs2metric_ddmetrics 45.46KiB 0.35 98.52% 12.51MiB 651.07KiB 13.27KiB 0 0.0508016 12.56MiB 643.33KiB 13.11KiB 0 0.0500205 False False
splunk_hec_to_splunk_hec_logs_noack 17.47KiB 0.07 85.93% 23.82MiB 468.97KiB 9.57KiB 0 0.0192231 23.84MiB 342.58KiB 6.99KiB 0 0.0140326 False False
http_pipelines_no_grok_blackhole 3.04KiB 0.03 10.73% 11.19MiB 180.5KiB 3.69KiB 0 0.0157503 11.19MiB 1.07MiB 22.2KiB 0 0.0952267 False False
enterprise_http_to_http 1.65KiB 0.01 17.89% 23.85MiB 252.21KiB 5.15KiB 0 0.0103269 23.85MiB 251.41KiB 5.15KiB 0 0.0102935 False False
file_to_blackhole -53.67KiB -0.05 43.79% 95.35MiB 2.83MiB 58.62KiB 0 0.0296492 95.3MiB 3.44MiB 71.65KiB 0 0.0361413 False False
splunk_hec_indexer_ack_blackhole -12.4KiB -0.05 37.00% 23.76MiB 870.48KiB 17.71KiB 0 0.0357729 23.75MiB 917.85KiB 18.67KiB 0 0.0377388 False False
syslog_splunk_hec_logs -21.29KiB -0.12 69.35% 16.68MiB 792.15KiB 16.12KiB 0 0.0463616 16.66MiB 646.08KiB 13.18KiB 0 0.0378599 False False
splunk_hec_to_splunk_hec_logs_acks -28.7KiB -0.12 74.77% 23.76MiB 817.78KiB 16.64KiB 0 0.033603 23.73MiB 922.05KiB 18.74KiB 0 0.0379323 False False
http_to_http_json -33.64KiB -0.14 99.07% 23.84MiB 356.13KiB 7.27KiB 0 0.0145833 23.81MiB 523.18KiB 10.68KiB 0 0.0214536 False False
fluent_elasticsearch -195.23KiB -0.24 100.00% 79.47MiB 53.73KiB 1.09KiB 0 0.000660058 79.28MiB 1.65MiB 33.91KiB 0 0.0207886 False False
datadog_agent_remap_blackhole -239.13KiB -0.36 96.53% 64.07MiB 4.33MiB 90.14KiB 0 0.0675402 63.84MiB 3.28MiB 68.45KiB 0 0.051391 False False
http_to_http_noack -102.95KiB -0.42 99.99% 23.83MiB 510.8KiB 10.44KiB 0 0.0209292 23.73MiB 1.17MiB 24.34KiB 0 0.0492035 False False
http_pipelines_blackhole -9.91KiB -0.6 100.00% 1.62MiB 9.91KiB 207.28B 0 0.00595734 1.61MiB 116.93KiB 2.38KiB 0 0.0707401 False False
syslog_humio_logs -110.68KiB -0.63 100.00% 17.28MiB 151.11KiB 3.09KiB 0 0.00854038 17.17MiB 169.27KiB 3.46KiB 0 0.00962682 False False
datadog_agent_remap_blackhole_acks -687.93KiB -1.07 100.00% 62.88MiB 5.21MiB 108.49KiB 0 0.0828448 62.21MiB 4.33MiB 90.54KiB 0 0.0696516 False False
syslog_log2metric_splunk_hec_metrics -198.02KiB -1.09 100.00% 17.81MiB 829.62KiB 16.91KiB 0 0.0454899 17.61MiB 1000.26KiB 20.35KiB 0 0.0554484 False False
http_to_http_acks -222.01KiB -1.2 62.26% 18.12MiB 8.53MiB 178.31KiB 0 0.470509 17.91MiB 8.5MiB 177.35KiB 0 0.474438 True True
syslog_log2metric_humio_metrics -208.17KiB -1.56 100.00% 13.05MiB 328.27KiB 6.7KiB 0 0.0245688 12.84MiB 531.47KiB 10.82KiB 0 0.0404068 False False
syslog_loki -263.66KiB -1.82 100.00% 14.17MiB 440.64KiB 9.01KiB 0 0.0303675 13.91MiB 706.78KiB 14.37KiB 0 0.0496104 False False

github-actions[bot] avatar Aug 03 '22 15:08 github-actions[bot]

Soak Test Results

Baseline: 65fe41b1f8ef41fc9cddec7cd10561751685922a Comparison: 1fced172b6a2b0855e472736b83a5bdce1832509 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
splunk_hec_route_s3 774.81KiB 4.27 100.00% 17.7MiB 2.57MiB 53.52KiB 0 0.145173 18.46MiB 2.49MiB 52.05KiB 0 0.134631 False False
http_to_http_acks 258.28KiB 1.41 68.64% 17.89MiB 8.73MiB 182.56KiB 0 0.48791 18.15MiB 8.62MiB 179.87KiB 0 0.474729 True True
syslog_log2metric_humio_metrics 168.37KiB 1.28 100.00% 12.81MiB 529.52KiB 10.8KiB 0 0.0403667 12.97MiB 487.57KiB 9.93KiB 0 0.0366978 False False
datadog_agent_remap_blackhole 375.65KiB 0.6 99.67% 61.53MiB 4.87MiB 101.35KiB 0 0.079064 61.9MiB 3.74MiB 78.01KiB 0 0.0603833 False False
http_pipelines_blackhole_acks 4.03KiB 0.35 81.68% 1.11MiB 115.16KiB 2.34KiB 0 0.100933 1.12MiB 93.95KiB 1.91KiB 0 0.0820498 False False
splunk_hec_to_splunk_hec_logs_noack 9.75KiB 0.04 62.61% 23.83MiB 422.56KiB 8.62KiB 0 0.0173133 23.84MiB 332.1KiB 6.78KiB 0 0.0136014 False False
splunk_hec_indexer_ack_blackhole 10.93KiB 0.04 32.80% 23.74MiB 923.29KiB 18.78KiB 0 0.0379677 23.75MiB 870.08KiB 17.7KiB 0 0.0357631 False False
enterprise_http_to_http -639.74B -0 6.92% 23.85MiB 249.89KiB 5.1KiB 0 0.0102319 23.84MiB 248.15KiB 5.08KiB 0 0.0101606 False False
splunk_hec_to_splunk_hec_logs_acks -10.16KiB -0.04 33.78% 23.77MiB 788.83KiB 16.06KiB 0 0.0324071 23.76MiB 827.38KiB 16.83KiB 0 0.0340048 False False
file_to_blackhole -58.85KiB -0.06 51.88% 95.36MiB 2.66MiB 55.2KiB 0 0.0279201 95.3MiB 3.02MiB 62.69KiB 0 0.0316368 False False
http_to_http_json -46.42KiB -0.19 99.93% 23.84MiB 348.82KiB 7.12KiB 0 0.0142836 23.8MiB 572.46KiB 11.67KiB 0 0.0234855 False False
datadog_agent_remap_blackhole_acks -178.84KiB -0.28 82.92% 63.21MiB 5.0MiB 104.17KiB 0 0.0791355 63.03MiB 3.77MiB 78.72KiB 0 0.0597214 False False
http_to_http_noack -84.08KiB -0.34 99.97% 23.84MiB 404.6KiB 8.27KiB 0 0.0165727 23.75MiB 1.03MiB 21.58KiB 0 0.0435545 False False
syslog_splunk_hec_logs -66.77KiB -0.39 99.57% 16.56MiB 873.21KiB 17.76KiB 0 0.0514969 16.49MiB 744.12KiB 15.16KiB 0 0.0440576 False False
http_pipelines_blackhole -6.63KiB -0.42 95.72% 1.54MiB 80.13KiB 1.64KiB 0 0.0507967 1.53MiB 138.97KiB 2.83KiB 0 0.0884704 False False
fluent_elasticsearch -561.24KiB -0.69 100.00% 79.47MiB 61.74KiB 1.24KiB 0 0.00075845 78.93MiB 5.58MiB 114.61KiB 0 0.0706941 False False
datadog_agent_remap_datadog_logs_acks -585.44KiB -0.89 100.00% 63.93MiB 3.59MiB 74.88KiB 0 0.0560666 63.36MiB 4.53MiB 94.38KiB 0 0.0715492 False False
syslog_regex_logs2metric_ddmetrics -123.11KiB -0.94 100.00% 12.77MiB 576.14KiB 11.74KiB 0 0.0440564 12.65MiB 553.35KiB 11.28KiB 0 0.0427166 False False
syslog_humio_logs -166.34KiB -0.96 100.00% 17.01MiB 197.71KiB 4.04KiB 0 0.0113509 16.84MiB 177.78KiB 3.64KiB 0 0.0103054 False False
socket_to_socket_blackhole -237.64KiB -0.99 100.00% 23.52MiB 856.61KiB 17.49KiB 0 0.0355532 23.29MiB 1009.24KiB 20.6KiB 0 0.0423053 False False
datadog_agent_remap_datadog_logs -750.32KiB -1.13 100.00% 64.78MiB 509.08KiB 10.42KiB 0 0.00767243 64.05MiB 4.31MiB 89.72KiB 0 0.0672549 False False
syslog_log2metric_splunk_hec_metrics -221.89KiB -1.2 100.00% 18.11MiB 1.07MiB 22.3KiB 0 0.0589686 17.9MiB 1.19MiB 24.88KiB 0 0.0667249 False False
http_text_to_http_json -790.72KiB -1.94 100.00% 39.89MiB 716.98KiB 14.64KiB 0 0.0175471 39.12MiB 772.91KiB 15.78KiB 0 0.0192893 False False
http_pipelines_no_grok_blackhole -296.39KiB -2.63 100.00% 11.03MiB 281.24KiB 5.74KiB 0 0.024904 10.74MiB 1.1MiB 22.92KiB 0 0.102509 False False
syslog_loki -420.87KiB -2.78 100.00% 14.8MiB 531.87KiB 10.88KiB 0 0.0350938 14.39MiB 809.02KiB 16.45KiB 0 0.0549057 False False

github-actions[bot] avatar Aug 03 '22 20:08 github-actions[bot]

Soak Test Results

Baseline: 8f2ae47fea9f20d4bf170b643cf0ea5a1f980c57 Comparison: 8d9556fc7706bf95936fc14a55b539c60da39a46 Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_splunk_hec_logs 208.11KiB 1.25 100.00% 16.29MiB 758.51KiB 15.43KiB 0 0.045463 16.49MiB 547.44KiB 11.18KiB 0 0.0324076 False False
http_to_http_acks 198.29KiB 1.12 59.01% 17.26MiB 8.05MiB 168.27KiB 0 0.466174 17.45MiB 8.25MiB 172.0KiB 0 0.472502 True True
syslog_regex_logs2metric_ddmetrics 100.2KiB 0.9 98.21% 10.88MiB 1.43MiB 29.87KiB 0 0.131616 10.97MiB 1.43MiB 29.94KiB 0 0.130603 False False
splunk_hec_route_s3 74.1KiB 0.4 72.38% 17.9MiB 2.31MiB 48.1KiB 0 0.12902 17.97MiB 2.3MiB 48.12KiB 0 0.128115 False False
http_text_to_http_json 105.03KiB 0.26 100.00% 39.39MiB 735.33KiB 15.01KiB 0 0.0182264 39.49MiB 743.96KiB 15.2KiB 0 0.0183925 False False
http_pipelines_blackhole_acks 2.58KiB 0.22 62.80% 1.13MiB 107.45KiB 2.19KiB 0 0.0925458 1.14MiB 92.94KiB 1.89KiB 0 0.0798695 False False
splunk_hec_to_splunk_hec_logs_noack 21.33KiB 0.09 91.75% 23.82MiB 501.93KiB 10.25KiB 0 0.0205766 23.84MiB 331.46KiB 6.77KiB 0 0.0135762 False False
splunk_hec_to_splunk_hec_logs_acks 6.83KiB 0.03 20.87% 23.75MiB 875.92KiB 17.81KiB 0 0.0360148 23.75MiB 917.84KiB 18.67KiB 0 0.0377278 False False
enterprise_http_to_http 744.6B 0 7.92% 23.85MiB 254.95KiB 5.2KiB 0 0.0104387 23.85MiB 251.42KiB 5.14KiB 0 0.010294 False False
splunk_hec_indexer_ack_blackhole -5.13KiB -0.02 18.20% 23.77MiB 761.96KiB 15.51KiB 0 0.0312997 23.76MiB 787.79KiB 16.03KiB 0 0.0323675 False False
file_to_blackhole -40.45KiB -0.04 40.74% 95.36MiB 2.51MiB 52.03KiB 0 0.0263151 95.32MiB 2.64MiB 54.84KiB 0 0.0276415 False False
http_to_http_json -26.83KiB -0.11 96.83% 23.84MiB 346.77KiB 7.08KiB 0 0.0141992 23.82MiB 503.26KiB 10.29KiB 0 0.0206298 False False
syslog_log2metric_splunk_hec_metrics -27.75KiB -0.15 73.30% 17.69MiB 698.47KiB 14.24KiB 0 0.0385514 17.66MiB 1009.31KiB 20.55KiB 0 0.055794 False False
syslog_log2metric_humio_metrics -41.52KiB -0.33 98.50% 12.47MiB 502.65KiB 10.26KiB 0 0.0393662 12.43MiB 669.46KiB 13.63KiB 0 0.0526009 False False
http_pipelines_blackhole -6.21KiB -0.38 98.78% 1.6MiB 46.36KiB 970.06B 0 0.0282048 1.6MiB 112.31KiB 2.29KiB 0 0.0685834 False False
http_to_http_noack -104.43KiB -0.43 100.00% 23.85MiB 248.78KiB 5.09KiB 0 0.010186 23.74MiB 1.1MiB 23.0KiB 0 0.0464493 False False
fluent_elasticsearch -358.03KiB -0.44 99.99% 79.47MiB 52.09KiB 1.05KiB 0 0.000639968 79.12MiB 4.43MiB 91.08KiB 0 0.0559578 False False
datadog_agent_remap_blackhole_acks -295.67KiB -0.48 89.90% 60.47MiB 6.65MiB 138.51KiB 0 0.110024 60.18MiB 5.51MiB 115.32KiB 0 0.0915838 False False
datadog_agent_remap_blackhole -401.78KiB -0.62 99.95% 63.13MiB 4.33MiB 90.32KiB 0 0.0686052 62.73MiB 3.39MiB 70.8KiB 0 0.054046 False False
http_pipelines_no_grok_blackhole -73.69KiB -0.64 99.89% 11.22MiB 133.0KiB 2.71KiB 0 0.0115701 11.15MiB 1.08MiB 22.45KiB 0 0.0966428 False False
datadog_agent_remap_datadog_logs_acks -846.25KiB -1.29 100.00% 63.89MiB 2.7MiB 56.61KiB 0 0.0423277 63.06MiB 4.48MiB 93.2KiB 0 0.0709852 False False
syslog_loki -207.73KiB -1.43 100.00% 14.15MiB 612.95KiB 12.55KiB 0 0.0422844 13.95MiB 835.53KiB 16.98KiB 0 0.0584774 False False
datadog_agent_remap_datadog_logs -1.05MiB -1.7 100.00% 62.17MiB 2.38MiB 49.93KiB 0 0.038329 61.11MiB 4.49MiB 93.54KiB 0 0.0734726 False False
syslog_humio_logs -301.93KiB -1.83 100.00% 16.11MiB 792.22KiB 16.17KiB 0 0.0480026 15.82MiB 853.69KiB 17.48KiB 0 0.0526915 False False
socket_to_socket_blackhole -562.83KiB -2.45 100.00% 22.4MiB 225.96KiB 4.61KiB 0 0.00984968 21.85MiB 216.26KiB 4.42KiB 0 0.00966366 False False

github-actions[bot] avatar Aug 09 '22 00:08 github-actions[bot]

Thanks for this contribution @mcasper !

jszwedko avatar Aug 12 '22 17:08 jszwedko