Several issues with Phantom on local

[cnnrshd]

I had three issues with Phantom when installing locally. This ticket tracks the issues and some solutions I had, and is more for reference purposes than a request for help:

1. Missing glibc (failed out in soar-prepare-system)
2. Missing mirrors (due to Redhat archiving them)
3. Out of disk space

Missing glibc

The related error message is:

TASK [phantom : prepare phantom install script without apps] *******************
fatal: [ar-phantom-attack-range-key-pair-ar]: FAILED! => {"changed": true, "cmd": "sudo /home/vagrant/splunk-soar/soar-prepare-system --splunk-soar-home /opt/soar --no-prompt", "delta": "0:00:00.011626", "end": "2023-10-17 12:13:22.217484", "msg": "non-zero return code", "rc": 1, "start": "2023-10-17 12:13:22.205858", "stderr": "/home/vagrant/splunk-soar/usr/python39/bin/python3.9: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /home/vagrant/splunk-soar/usr/python39/bin/../lib/libpython3.9.so.1.0)\n/home/vagrant/splunk-soar/usr/python39/bin/python3.9: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by /home/vagrant/splunk-soar/usr/python39/bin/../lib/libpython3.9.so.1.0)\n/home/vagrant/splunk-soar/usr/python39/bin/python3.9: /lib64/libc.so.6: version `GLIBC_2.26' not found (required by /home/vagrant/splunk-soar/usr/python39/bin/../lib/libpython3.9.so.1.0)\n/home/vagrant/splunk-soar/usr/python39/bin/python3.9: /lib64/libc.so.6: version `GLIBC_2.27' not found (required by /home/vagrant/splunk-soar/usr/python39/bin/../lib/libpython3.9.so.1.0)", "stderr_lines": ["/home/vagrant/splunk-soar/usr/python39/bin/python3.9: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /home/vagrant/splunk-soar/usr/python39/bin/../lib/libpython3.9.so.1.0)", "/home/vagrant/splunk-soar/usr/python39/bin/python3.9: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by /home/vagrant/splunk-soar/usr/python39/bin/../lib/libpython3.9.so.1.0)", "/home/vagrant/splunk-soar/usr/python39/bin/python3.9: /lib64/libc.so.6: version `GLIBC_2.26' not found (required by /home/vagrant/splunk-soar/usr/python39/bin/../lib/libpython3.9.so.1.0)", "/home/vagrant/splunk-soar/usr/python39/bin/python3.9: /lib64/libc.so.6: version `GLIBC_2.27' not found (required by /home/vagrant/splunk-soar/usr/python39/bin/../lib/libpython3.9.so.1.0)"], "stdout": "", "stdout_lines": []}

PLAY RECAP *********************************************************************
ar-phantom-attack-range-key-pair-ar : ok=2    changed=2    unreachable=0    failed=1    skipped=6    rescued=0    ignored=0   

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

Attempting to update glibc through yum does not work - the glibc version appears to be locked at 2.17. Switching to centos/8 in the phantom Vagrantfile fixed this.

Missing mirrors

Due to Redhat archiving the centos mirrors, the yum update -y called during soar installation (I believe) failed. I found a fix in this guide - some sed commands to find/replace mirror references to point to the vault:

sudo sed -i -e "s|mirrorlist=|#mirrorlist=|g" /etc/yum.repos.d/CentOS-*
sudo sed -i -e "s|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g" /etc/yum.repos.d/CentOS-*

Out of disk space

By default, the centos/8 image only has a 10GB disk. I initially replaced the cp /home/vagrant/splunk-soar /home/phantom/splunk-soar to a mv, but that wasn't enough. The SOAR installation wants at least 20GiB of free space (see the error message below):

fatal: [ar-phantom-attack-range-key-pair-ar]: FAILED! => {"changed": true, "cmd": "./soar-install --splunk-soar-home /opt/soar --no-prompt --ignore-warnings", "delta": "0:00:58.956249", "end": "2023-10-17 17:51:32.626061", "msg": "non-zero return code", "rc": 1, "start": "2023-10-17 17:50:33.669812", "stderr": "\u001b[33mWarning: Partition '/opt/soar' should have at least 500 GiB of available space, only found 1.93/9.99 GiB free\u001b[0m\n\u001b[33mWarning: Partition '/opt/soar/data' should have at least 20 GiB of available space, only found 1.93/9.99 GiB free\u001b[0m\n\u001b[33mWarning: Partition '/opt/soar/vault' should have at least 20 GiB of available space, only found 1.93/9.99 GiB free\u001b[0m\n\u001b[33mWarning: Partition '/opt/soar/data/db' should have at least 500 GiB of available space, only found 1.93/9.99 GiB free\u001b[0m\n\u001b[33mWarning: Partition '/opt/soar/data/splunk' should have at least 500 GiB of available space, only found 1.93/9.99 GiB free\u001b[0m\n\u001b[33mWarning: Partition '/opt/soar/var/log/phantom' should have at least 10 GiB of available space, only found 1.93/9.99 GiB free\u001b[0m\n--- Logging error ---\nTraceback (most recent call last):\n  File \"/home/phantom/splunk-soar/install/console.py\", line 208, in run\n    proc = subprocess.run(normalized_cmd, **cmd_args)  # noqa: PH112\n  File \"/home/phantom/splunk-soar/usr/python39/lib/python3.9/subprocess.py\", line 528, in run\n    raise CalledProcessError(retcode, process.args,\nsubprocess.CalledProcessError: Command '['tar', '--exclude', './manifest.csv', '-xf', '/home/phantom/splunk-soar/soar_component_dependencies.tar', '-C', '/opt/soar']' returned non-zero exit status 2.\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/home/phantom/splunk-soar/install/console.py\", line 225, in run\n    raise InstallError(\ninstall.install_common.InstallError: An unexpected error occurred while running a subprocess\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/home/phantom/splunk-soar/usr/python39/lib/python3.9/logging/__init__.py\", line 1086, in emit\n    stream.write(msg + self.terminator)\nOSError: [Errno 28] No space left on device\nCall stack:\n  File \"/home/phantom/splunk-soar/./soar-install\", line 127, in <module>\n    main()\n  File \"/home/phantom/splunk-soar/./soar-install\", line 85, in main\n    deployment.run()\n  File \"/home/phantom/splunk-soar/install/deployments/deployment.py\", line 134, in run\n    self.run_deploy()\n  File \"/home/phantom/splunk-soar/usr/python39/lib/python3.9/contextlib.py\", line 79, in inner\n    return func(*args, **kwds)\n  File \"/home/phantom/splunk-soar/install/deployments/deployment.py\", line 202, in run_deploy\n    operation.run()\n  File \"/home/phantom/splunk-soar/install/operations/deployment_operation.py\", line 135, in run\n    self.install()\n  File \"/home/phantom/splunk-soar/install/operations/tasks/install_files.py\", line 322, in install\n    super().install()\n  File \"/home/phantom/splunk-soar/install/operations/tasks/install_files.py\", line 72, in install\n    self._install_files()\n  File \"/home/phantom/splunk-soar/install/operations/tasks/install_files.py\", line 171, in _install_files\n    self._untar_file(tar, self.options.phantom_home)\n  File \"/home/phantom/splunk-soar/install/retry.py\", line 39, in wrapper\n    return func(*args, **kwargs)\n  File \"/home/phantom/splunk-soar/install/operations/tasks/install_files.py\", line 135, in _untar_file\n    self.shell.run(\n  File \"/home/phantom/splunk-soar/install/console.py\", line 251, in run\n    logger.debug(\"Subprocess completed.\")\nMessage: 'Subprocess completed.'\nArguments: ()\n\u001b[1;31mException raised while untarring (/home/phantom/splunk-soar/soar_component_dependencies.tar) to (/opt/soar): \n\n An unexpected error occurred while running a subprocess\u001b[0m\nTraceback (most recent call last):\n  File \"/home/phantom/splunk-soar/install/console.py\", line 208, in run\n    proc = subprocess.run(normalized_cmd, **cmd_args)  # noqa: PH112\n  File \"/home/phantom/splunk-soar/usr/python39/lib/python3.9/subprocess.py\", line 528, in run\n    raise CalledProcessError(retcode, process.args,\nsubprocess.CalledProcessError: Command '['tar', '--exclude', './manifest.csv', '-xf', '/home/phantom/splunk-soar/soar_component_dependencies.tar', '-C', '/opt/soar']' returned non-zero exit status 2.

Vagrant has recently added support for disk resizing - I used this by modifying the vagrantfile to include this line:

config.vm.disk :disk, size: "50GB", primary: true

Some commands need to be run on the host to resize the disk:

sudo parted /dev/sda resizepart 1 100%
sudo xfs_growfs /dev/sda1

These changes resolved all issues, and I was able to python3 ./attack_range.py build with the following config:

general:
  cloud_provider: local
  attack_range_password: redacted
local: {}
windows_servers:
- hostname: ar-win-dc
  windows_image: windows-2016-v3-0-0
  create_domain: '1'
  install_red_team_tools: '1'
  bad_blood: '1'
- hostname: ar-win-2
  windows_image: windows-2019-v3-0-0
  join_domain: '1'
  install_red_team_tools: '1'
linux_servers:
- hostname: ar-linux
phantom_server:
  phantom_server: '1'
  phantom_app: splunk_soar-unpriv.tgz

Since these issues are resolved, I'll be sending a PR soon.

[cnnrshd]

When i tried to get this into Ansible, I realized parted does not like to be automated. Went with growpart instead:

sudo growpart /dev/sda 1

[gowthamarajr]

Thank you for the PR. We are currently reviewing it. @cnnrshd

[P4T12ICK]

@cnnrshd we changed our install process of Splunk SOAR last year. You will need to download it from the Splunk website, save it into the apps folder and give the name in the attack_range.yml configuration. Therefore, we can't accept your PR. Sorry.

[cnnrshd]

I don't see how that relates at all.

The Phantom/SOAR installation did not work when running locally due to several issues (Documented above), none of which are related to where the {{ phantom_app }} variable expands to.

If you compare the commits, the current develop branch for copying SOAR to the server is identical to my PR, and both are copying from ../../apps/{{ phantom_app }}