fluent
Fluentd fails to load separate gems with same @type name
Describe the bug
I have installed two splunk gems, namely, fluent-plugin-splunk-enterprise and fluent-plugin-splunk-hec. Both the gems (even though 2 separate gems) have types with same name splunk_hec. Even though the type names are same, they have different parameters, that makes them distinct.
But when I use @type splunk_hec in my fluentd configuration with the parameters of fluent-plugin-splunk-enterprise, it fails to recognize them and throws below error:
2025-11-06 17:05:28 +0000 [error]: config error file="/etc/fluent/fluentd.conf" error_class=Fluent::ConfigError error="'hec_token' parameter is required"
It asks for hec_token, which is basically a parameter of fluent-plugin-splunk-hec and not fluent-plugin-splunk-enterprise.
To Reproduce
This bug can be reproduced by simply installing both these gems:
- fluent-gem install fluent-plugin-splunk-hec
- fluent-gem install fluent-plugin-splunk-enterprise
Fluentd will automatically register the @type name from first gem and ignore the second one. (which is happening in my case)
Expected behavior
Now, I do understand that this is a known behavior of fluentd and it has been like that for some time. So, this understanding brings me to a question that is there a workaround for this without having to remove any one of the 2 gems?
Is there a way that we can install both gems and use them in our configuration without conflicting with each other??
Your Environment
- Fluentd version: 1.19.0
- Package version: 6.0.0
- Operating system: Rocky 8
- Kernel version: 5
Your Configuration
<match splunk.**>
@type splunk_hec
protocol http
host "example.com"
port 8088
token 00000000-0000-0000-0000-000000000000
sourcetype "_json"
<buffer>
@type memory
flush_interval 10s
retry_max_times 2
retry_wait 1s
</buffer>
</match>
Your Error Log
2025-11-06 17:05:28 +0000 [error]: config error in:
<match splunk.**>
@type splunk_hec
protocol http
host "example.com"
port 8088
token 00000000-0000-0000-0000-000000000000
sourcetype "_json"
<buffer>
@type memory
flush_interval 10s
retry_max_times 2
retry_wait 1s
</buffer>
</match>
2025-11-06 17:05:28 +0000 [error]: config error file="/etc/fluent/fluentd.conf" error_class=Fluent::ConfigError error="'hec_token' parameter is required"
Additional context
I am well aware of the workaround by forking the fluent-plugin-splunk-enterprise and renaming the type, and then building the gem. But this workaround is not feasible and convenient for my use case. I am much interested in a solution where fluentd handles this internally or we handle it internally without having to make modifications in the gem.
As you mentioned, duplicated plugin type name is not allowed at the time. If repack gem is not feasible workaround for you, launch multiple Fluentd instance for conflicted gem might be another workaround. (but not work if you want to use them in same pipeline)
