Support for alias of a plugin

[koenekle]

Is your feature request related to a problem? Please describe.

Currently we are using the alias feature of fluent-bit, to reference fluent-bit plugin metrics within custom dashboards. Unfortunately this is not possible with fluent-operator.

To us, this is especially important, as we cannot be sure which index a plugin will get, as the order of plugins - and therefore their index - is managed by fluent-operator.

Describe the solution you'd like

I would like to be able to set the alias like this:

apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterFilter
metadata:
  name: throttle-application-xy
spec:
  match: application.xy
  filters:
    - throttle:
        alias: throttle.application-xy
        rate: 200
        window: 300
        interval: 1s

which would result in such a configuration block for fluent-bit:

[Filter]
   Name    throttle
   Match   application.xy
   Alias   throttle.application-xy
   Rate    200
   Window  300
   Interval 1s

Additional context

I would be happy to help out implementing this. Please let me know, if I should create a PR.

[mangoGoForward]

Welcome to submit PR～ cc @benjaminhuo @wenchajun

[benjaminhuo]

@wanjunlei @wenchajun the alias field seems to be a common field for all input and output plugins, where should we add it to?

[wanjunlei]

Like the tls, we can add a struct named Common, and add all common field to it

[benjaminhuo]

@Gentleelephant will work on this issue

[koenekle]

During the weekend I tried to find a quick solution. I have recognized, that there is an alias for input and output on ClusterInputSpec level. But this would not enable me, to put different alias on single filters of a filter chain, which I would require.

Therefore I hacked a quick solution as @wanjunlei suggested: https://github.com/koenekle/fluent-operator/commit/93557c84facfef462e19f5c91690fcb04841d729

[koenekle]

Actually I have realized, that there might be a way more simple solution, that does not require a common struct and therefore changes a lot less code: https://github.com/koenekle/fluent-operator/commit/60ed941e443a10dbe5b29ac93f259e78d8fde413

Please let me know which direction your prefer.

[benjaminhuo]

@koenekle Thanks for your contribution. If I understand the [alias](alias feature) correctly, it is for input and output plugins only and it's not for filter plugins:

[koenekle]

@benjaminhuo No, it will also work for filter plugins. I have tested so with the throttle plugin. But you are right, the documentation is a bit misleading there.

[benjaminhuo]

After you add an alias for the throttle filter, what's the effect of this alias? The docs says the input/output plugin's alias is for metrics purpose.

[koenekle]

The effect is, that you won't have your metrics with numbered instance names, but with your configured alias. There is an metric for all dropped records of the throttle plugin. I just recognized, this one is also not documented yet within fluent-bit.

But also for other plugins, like rewrite tag. They also expose input metrics, because they create new records. Therefore they also have metrics, that are exposed as fluentbit_input_records_total. This is why we should enable alias for filter plugins aswell, as this improves the metrics usability by a great deal, as we can observe the pipeline at specific steps and not just the whole input and output.