ibm-spectrum-scale-install-infra icon indicating copy to clipboard operation
ibm-spectrum-scale-install-infra copied to clipboard

How to reprovision spectrum scale node?

Open sjpb opened this issue 2 years ago • 0 comments

Describe the bug Having created a cluster, I deleted/recreated a vm/node/host, and ran the playbooks again. This fails, saying the node is already in the cluster. It might be from Spectrum Scale's point of view but it didn't even have the binaries. Not sure how to recover from this else I'd suggest a PR!

To Reproduce Steps to reproduce the behavior:

  1. Install ibm-spectrum-scale-install-infra.git at v2.5.0
  2. Run playbook like this on RH8 hosts:
- name: Setup SpectrumScale
  hosts: spectrum_scale
  become: yes
  gather_facts: yes
  tags:
    - spectrum_scale
  collections:
    - ibm.spectrum_scale
  tasks:
    - include_role:
        name: ibm.spectrum_scale.core_prepare
    - include_role:
        name: ibm.spectrum_scale.core_install
    - include_role:
        name: ibm.spectrum_scale.core_configure
    - include_role:
        name: ibm.spectrum_scale.core_verify

Expected behavior Host gets added back to the cluster.

Actual behavior

TASK [ibm.spectrum_scale.core_configure : cluster | Add new nodes to cluster] ***********************************************************************************************************************************
fatal: [compute-0]: FAILED! => {
    "changed": true,
    "cmd": [
        "/usr/lpp/mmfs/bin/mmaddnode",
        "-N",
        "/var/mmfs/tmp/NodeFile",
        "--accept"
    ],
    "delta": "0:00:08.582840",
    "end": "2023-04-19 11:32:12.819687",
    "rc": 1,
    "start": "2023-04-19 11:32:04.236847"
}

STDOUT:

mmaddnode: Node compute-5.<redacted> will be designated as possessing client license.


STDERR:

mmaddnode: Node compute-5 already belongs to the GPFS cluster.
mmaddnode: mmaddnode quitting.  None of the specified nodes are valid.
mmaddnode: Command failed. Examine previous error messages to determine cause.

Environment Please run the following an paste your output here:

$ ansible --version
ansible [core 2.13.7]
  config file = /home/rocky/<redacted>/environments/prd/ansible.cfg
  configured module search path = ['/home/rocky/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/rocky/<redacted>/venv/lib64/python3.8/site-packages/ansible
  ansible collection location = /home/rocky/<redacted>/ansible/collections
  executable location = /home/rocky/<redacted>/venv/bin/ansible
  python version = 3.8.13 (default, Nov  8 2022, 17:19:05) [GCC 8.5.0 20210514 (Red Hat 8.5.0-15)]
  jinja version = 3.1.2
  libyaml = True
$ python --version
Python 3.8.13

# Deployment
Spectrum_Scale_Erasure_Code-5.1.6.1-x86_64-Linux-install

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.

sjpb avatar Apr 19 '23 12:04 sjpb