Issue with Connecting Fluent-Bit in EKS to Splunk Endpoint in AWS

[illoRocks]

Bug Report

A connection from fluent-bit in EKS to a Splunk Endpoint (Software as a Service in AWS) is not working.

We received the following URL from the team that manages the Splunk account: example.com/services/collector/event. Our Security Group is enabled in the Splunk account.

We have tried the following combinations in our values.yaml:

1. Variant

          outputs: |                
            [OUTPUT]    
                Name  splunk    
                Match kube.*    
                Host example.com/services/collector/event    
                Port 8088    
                event_index my_index  
                splunk_token ****    
                TLS On    
                TLS.Verify Off    
                TLS.Debug On    

This leads to the following error message:

[2024/02/02 10:37:48] [ warn] [net] getaddrinfo(host='https://example.com/services/collector/event', err=8): Misformatted domain name  
[2024/02/02 10:37:48] [ warn] [engine] failed to flush chunk '1-1706870267.507666006.flb', retry in 6 seconds: task_id=0, input=tail.0 > output=splunk.0 (out_id=0)  

2. Variant

          outputs: |                
            [OUTPUT]    
                Name  splunk    
                Match kube.*    
                Host example.com  
                Port 8088    
                event_index my_index    
                splunk_token ****    
                TLS On    
                TLS.Verify Off    
                TLS.Debug On    

This leads to the following error message:

[2024/02/02 10:23:44] [error] [net] TCP connection failed: example.com:8088 (Connection refused)  

It also did not work with port 443 or Port "".

Environment

• Version used: 2.2.1

How do we need to adjust our values.yaml, or configure the Splunk AWS account to write from fluent-bit to Splunk?

[patrick-stephens]

What port should you be using? TLS defaults to port 443 but you seem to be providing 8088 above? The host I think is just the main hostname, I think what you're after is specifying custom endpoints underneath it right?

[shree007]

Please use following config, it is working for me

[OUTPUT]
       Name splunk
       Match kube.*
       Host example.com
       Port 443
       Splunk_Token  hec-token
       Retry_Limit False
       TLS            On
       TLS.Verify     Off
       event_index    my-index

In order to troubleshoot you may use telnet for splunk endpoint and port from machine.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.