bug: fluentd splunk output is missing

[maxkokocom]

Describe the issue

When applying Fluentd ClusterOutput with splunk output the resource fields validation fails.

I am deploying operator via kubectl apply -f https://raw.githubusercontent.com/fluent/fluentbit-operator/master/manifests/setup/setup.yaml which should deploy operator as it is on master. I can see splunk MR was recently merged.

error validating data: ValidationError(ClusterOutput.spec.outputs[0]): unknown field "splunk" in io.fluent.fluentd.v1alpha1.ClusterOutput.spec.outputs; if you choose to ignore these errors, turn validation off with --validate=false

To Reproduce

spec:
  outputs:
  - splunk:
      host: https://splunk.xxx
      port: 8443
      eventIndex:  "k8s_fluent_operator_testing"
      splunkToken:
        valueFrom:
            secretKeyRef:
              name: fluentd-splunk
              key: splunk-http-token

Expected behavior

It should apply successfully.

Your Environment

- Fluent Operator version: master
- Container Runtime: containerd 
- Operating system: amazon linux
- Kernel version:

How did you install fluent operator?

kubectl apply -f https://raw.githubusercontent.com/fluent/fluentbit-operator/master/manifests/setup/setup.yaml

Additional context

No response

[benjaminhuo]

@wenchajun

[benjaminhuo]

@Gentleelephant would you take a look at this? @wenchajun can help you if necessary.

[Gentleelephant]

If you want to use the splunk plugin, you should configure fluentbit instead of fluentd. like this apiVersion: fluentbit.fluent.io/v1alpha2

[maxkokocom]

Ohh sorry I did not realised it works only for fluent-bit. What about a HA setup when I deploy both of them in the forwarder/aggregator pattern? I want to deploy fluent-bit as a forwarder to fluentd, and then fluentd would output to splunk. I need some fluentd plugins, while making it a bit better performance and scalability wise.

[Gentleelephant]

Do you want to use fluentbit+fluentd mode? https://github.com/kubesphere-sigs/fluent-operator-walkthrough#fluent-bit--fluentd-mode

[Gentleelephant]

This is the list of fluentd plugins supported by fluent-operator. https://github.com/fluent/fluent-operator/blob/master/docs/plugins/fluentd/index.md

[maxkokocom]

Hi, yes I am trying to use fluentbit+fluentd mode. Splunk output plugin is missing for fluentd. This is the point of this ticket yes.

[benjaminhuo]

Hi, yes I am trying to use fluentbit+fluentd mode. Splunk output plugin is missing for fluentd. This is the point of this ticket yes.

OK, then this is not a bug but instead a feature request. We've community contributors who helped to add the fluentbit Splunk output plugin.

[gopikishanm]

Hi everyone, is there a plan to implement the splunk plugin for fluentd

[benjaminhuo]

Consider using custom plugin for unsupported plugins https://github.com/fluent/fluent-operator/blob/master/docs/best-practice/custom-plugin.md