Splunk output plugin doesn't send metadata like sourcetype and index in raw mode

[ahahu]

Bug Report

Describe the bug When using splunk_send_raw=On the Splunk output plugin doesn't send the values set in event_host, event_source, event_sourcetype, event_sourcetype_key, event_index, event_index_key to Splunk. The section "Sending Raw Events" states that "if the option splunk_send_raw has been enabled, the user must take care to put all log details in the event field, and only specify fields known to Splunk in the top level event. This might have been correct in the past, but the current official Splunk documentation describes that these fields can be passed as request parameters: https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTinput#services.2Fcollector.2Fraw And here you can find examples: https://docs.splunk.com/Documentation/Splunk/latest/Data/HECExamples

To Reproduce Setup Splunk output with event_* Metadata:

[INPUT]
    Name  dummy
    Tag   dummy

[OUTPUT]
    Name        splunk
    Match       *
    Host        127.0.0.1
    Port        8081
    TLS         On
    TLS.Verify  Off
    Splunk_Send_Raw On
    Splunk_Token aaaaaaa-bbbb-cccc-dddd-eeeeeeeeee
    event_sourcetype cpudata

Observe the submitted URL request (see screenshot below) by using a proxy. It doesn't contain sourcetype as request parameter.

Expected behavior The request to Splunk should contain a parameter "?sourcetype=cpudata"

Screenshots

Your Environment

• Version used: 1.9.3

Additional context In my experience passing the metadata as field value doesn't work reliably. For example it is not possible to overwrite the index our sourcetype that is set in Splunk for the HEC collector token that is used.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[ahahu]

Maybe someone could do the triage?

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.

[lecaros]

The metadata is set only if the destination is not the raw endpoint https://github.com/fluent/fluent-bit/blob/9444fdc5ee858a42b58b9a97010d997096e63634/plugins/out_splunk/splunk.c#L281-L293

As per the documentation, the host, index, source, and sourcetype can be sent as query parameters. https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTinput#services.2Fcollector.2Fraw This was probably not possible before.

Hi @ahahu, as per the review, this would be an evolutive change in the plugin. If you want to contribute with a PR you're most welcome.