fluent-plugin-splunk
fluent-plugin-splunk copied to clipboard
Configurable metadata fields for TCP output plugin
This is a feature request to allow configurable default metadata fields (host
, source
, sourcetype
, index
) for the TCP output, as is currently supported by the HEC output.
I believe this should be possible by sending a header over the TCP socket as the first line of the payload, which should look like:
***SPLUNK*** <metadata field>=<string> <metadata field>=<string> ...
More info at: https://docs.splunk.com/Documentation/Splunk/8.0.4/Data/Assignmetadatatoeventsdynamically
I'm thinking something along the lines of this oughta do it. Seems to be working as expected from some basic testing.
--- a/lib/fluent/plugin/out_splunk_tcp.rb
+++ b/lib/fluent/plugin/out_splunk_tcp.rb
@@ -14,6 +14,11 @@ module Fluent
config_param :host, :string
config_param :port, :integer
+ config_param :default_host, :string, default: nil
+ config_param :default_source, :string, default: nil
+ config_param :default_sourcetype, :string, default: nil
+ config_param :default_index, :string, default: nil
+
config_param :format, :string, default: 'raw'
# for raw format
@@ -74,6 +79,20 @@ module Fluent
else
raise ConfigError, "invalid 'format' option: #{@format}"
end
+
+ @headers = []
+ if @default_host
+ @headers << "host=#{@default_host}"
+ end
+ if @default_source
+ @headers << "source=#{@default_source}"
+ end
+ if @default_sourcetype
+ @headers << "sourcetype=#{@default_sourcetype}"
+ end
+ if @default_index
+ @headers << "index=#{@default_index}"
+ end
end
def multi_workers_ready?
@@ -92,6 +111,11 @@ module Fluent
return if chunk.empty?
payload = ''
+ unless @headers.empty?
+ payload << "***SPLUNK*** #{@headers.join(' ')}"
+ payload << @line_breaker
+ end
+
chunk.msgpack_each do |time, record|
event = @formatter.call(time, record)
unless event.empty?