docker-splunk icon indicating copy to clipboard operation
docker-splunk copied to clipboard

Splunk fails to start without content message

Open guentoan opened this issue 2 years ago • 3 comments

Hi team, I'm using Splunk Enterprise with 1year license(my license will expire on Jul 14th). This morning, I have restarted server, and then start docker by command docker-compose up -d, Unfortunately, bellow error is coming without content message.

splunk                | included: /opt/ansible/roles/splunk_standalone/tasks/../../splunk_common/tasks/set_as_hec_receiver.yml for localhost
splunk                | Monday 04 July 2022  14:59:54 +0900 (0:00:00.052)       0:01:35.307 ***********
splunk                |
splunk                | TASK [splunk_standalone : Setup global HEC] ************************************
splunk                | task path: /opt/ansible/roles/splunk_common/tasks/set_as_hec_receiver.yml:4
splunk                | fatal: [localhost]: FAILED! => {
splunk                |     "cache_control": "private",
splunk                |     "changed": false,
splunk                |     "connection": "Close",
splunk                |     "content_length": "130",
splunk                |     "content_type": "text/xml; charset=UTF-8",
splunk                |     "date": "Mon, 04 Jul 2022 05:59:55 GMT",
splunk                |     "elapsed": 0,
splunk                |     "redirected": false,
splunk                |     "server": "Splunkd",
splunk                |     "status": 401,
splunk                |     "url": "https://127.0.0.1:8089/services/data/inputs/http/http",
splunk                |     "vary": "Cookie, Authorization",
splunk                |     "www_authenticate": "Basic realm=\"/splunk\"",
splunk                |     "x_content_type_options": "nosniff",
splunk                |     "x_frame_options": "SAMEORIGIN"
splunk                | }
splunk                |
splunk                | MSG:
splunk                |
splunk                | Status code was 401 and not [200]: HTTP Error 401: Unauthorized
splunk                |
splunk                | PLAY RECAP *********************************************************************
splunk                | localhost                  : ok=58   changed=2    unreachable=0    failed=1    skipped=60   rescued=0    ignored=0
splunk                |

docker-compose.yml

version: '3.1'

services:
  splunk:
    image: splunk/splunk:8.2.2
    container_name: splunk
    environment:
      SPLUNK_START_ARGS: --accept-license
      SPLUNK_ADD: tcp 1514
      SPLUNK_ENABLE_LISTEN: 9997
      SPLUNK_PASSWORD: *********
      ANSIBLE_EXTRA_FLAGS: -vv
      TZ: Asia/Tokyo
      DEBUG: "true"
      SPLUNK_DEFAULTS_URL: https://***.com/splunk.yml
    ports:
      - "8000:8000"
      - "9997:9997"
      - "8088:8088"
      - "1514:1514"
    networks:
      - ***
    dns:
      - 8.8.8.8
      - 8.8.4.4
    volumes:
      - opt-splunk-etc-1:/opt/splunk/etc
      - opt-splunk-var:/opt/splunk/var

guentoan avatar Jul 04 '22 06:07 guentoan

I'm having this same issue with a new installation of Splunk Docker. My docker-compose.yml is even more minimal, but the error is the same. The DEBUG=true flag doesn't provide more information.

erichiggins avatar Aug 08 '22 20:08 erichiggins

Just passing by when trying to solve my own problem....

@guentoan - had you by chance deleted the admin account? I had and was greeted with this same ansible playbook error on pod restart. I scaled the replicas back to zero, went into the host path (a kubernetes nfs PV, in my case) that I mount at /opt/splunk/etc in the container, and changed the username of another admin account back admin. I changed SPLUNK_PASSWORD to that user's password and the container/pod started again.

@erichiggins - not sure in your case as you mentioned it's a new installation and the users should only contain admin, whose password you're passing as an env var.

zchef2k avatar Sep 08 '22 00:09 zchef2k

had the same issue after changing the admin password from inside the splunk UI. make sure to update the splunk password on the variable SPLUNK_PASSWORD with the new password changed in the UI.

here is my docker-compose.yaml :

version: '3'
services:
  splunk:
    image: splunk/splunk:latest
    container_name: spl
    ports:
      - "127.0.0.1:8000:8000"
    environment:
      SPLUNK_PASSWORD: MyNewPasswordChangedInTheUI
      SPLUNK_START_ARGS: --accept-license
    volumes:
      - splunk_etc:/opt/splunk/etc
      - splunk_var:/opt/splunk/var
volumes:
  splunk_etc:
  splunk_var:

AnsenIO avatar Apr 27 '24 20:04 AnsenIO