Unable to add splunk universal forwarder as a sidecar

[tkorrapati1]

trafficstars

Getting below error when I'm trying to set up a splunk side car container.

ERROR OBSERVED

TASK [splunk_universal_forwarder : Setup global HEC] ***************************
task path: /opt/ansible/roles/splunk_common/tasks/set_as_hec_receiver.yml:4
fatal: [localhost]: FAILED! => {
    "cache_control": "private",
    "changed": false,
    "connection": "Close",
    "content_length": "130",
    "content_type": "text/xml; charset=UTF-8",
    "date": "Tue, 07 Dec 2021 09:34:20 GMT",
    "elapsed": 0,
    "redirected": false,
    "server": "Splunkd",
    "status": 401,
    "url": "https://127.0.0.1:8089/services/data/inputs/http/http",
    "vary": "Cookie, Authorization",
    "www_authenticate": "Basic realm=\"/splunk\"",
    "x_content_type_options": "nosniff",
    "x_frame_options": "SAMEORIGIN"
}

MSG:

Status code was 401 and not [200]: HTTP Error 401: Unauthorized

How I'm adding universal forwarder to my deployment in K8s

  - name: splunk-forwarder
    image: splunk/universalforwarder:8.2
    env:
      - name: SPLUNK_START_ARGS
        value: "--accept-license"
      - name: ANSIBLE_EXTRA_FLAGS
        value:  "-vv"
      - name: SPLUNK_CMD
        value: 'install app /tmp/splunk-creds/splunkclouduf.spl, add monitor /app/logs'
      - name: SPLUNK_PASSWORD
        valueFrom:
          secretKeyRef:
            name: mia-env-secret
            key: SPLUNK_UF_PASSWORD
    resources: {}
    volumeMounts:
      - name: splunk-uf-creds-spl
        mountPath: tmp/splunk-creds
      - name: logs
        mountPath: /app/logs

There aren't many examples of how to use docker universalforwarder out there, any help or reference to how to containerized version of UF is appreciated.