ibm-spectrum-scale-install-infra icon indicating copy to clipboard operation
ibm-spectrum-scale-install-infra copied to clipboard

Provide troubleshooting guidance, when NSD names include invalid characters

Open troppens opened this issue 3 years ago • 2 comments

Describe the bug

Having invalid characters in NSD names, the resulting error message is not readable for humans:

TASK [core/cluster : storage | Create new NSDs] ****************************************************************************************** failed: [sc1-n1 -> sc1-n1] (item={'diff': [], 'dest': '/var/mmfs/tmp/StanzaFile.new.fs1', 'src': '/root/.ansible/tmp/ansible-tmp-1637690812.2709622-156623-28533950443182/source', 'md5sum': '61903b5a4b37ec17d669f6abfd6cc228', 'checksum': 'c33c980ddba2ac2a328d35cf3c68682a57625a61', 'changed': True, 'uid': 0, 'gid': 0, 'owner': 'root', 'group': 'root', 'mode': '0644', 'state': 'file', 'size': 108, 'invocation': {'module_args': {'src': '/root/.ansible/tmp/ansible-tmp-1637690812.2709622-156623-28533950443182/source', 'dest': '/var/mmfs/tmp/StanzaFile.new.fs1', 'mode': None, 'follow': False, '_original_basename': 'StanzaFile.j2', 'checksum': 'c33c980ddba2ac2a328d35cf3c68682a57625a61', 'backup': False, 'force': True, 'unsafe_writes': False, 'content': None, 'validate': None, 'directory_mode': None, 'remote_src': None, 'local_follow': None, 'owner': None, 'group': None, 'seuser': None, 'serole': None, 'selevel': None, 'setype': None, 'attributes': None, 'regexp': None, 'delimiter': None}}, 'failed': False, 'item': 'fs1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": true, "cmd": ["/usr/lpp/mmfs/bin/mmcrnsd", "-F", "/var/mmfs/tmp/StanzaFile.new.fs1", "-v", "yes"], "delta": "0:00:02.567071", "end": "2021-11-23 10:07:00.920722", "item": {"ansible_loop_var": "item", "changed": true, "checksum": "c33c980ddba2ac2a328d35cf3c68682a57625a61", "dest": "/var/mmfs/tmp/StanzaFile.new.fs1", "diff": [], "failed": false, "gid": 0, "group": "root", "invocation": {"module_args": {"_original_basename": "StanzaFile.j2", "attributes": null, "backup": false, "checksum": "c33c980ddba2ac2a328d35cf3c68682a57625a61", "content": null, "delimiter": null, "dest": "/var/mmfs/tmp/StanzaFile.new.fs1", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/root/.ansible/tmp/ansible-tmp-1637690812.2709622-156623-28533950443182/source", "unsafe_writes": false, "validate": null}}, "item": "fs1", "md5sum": "61903b5a4b37ec17d669f6abfd6cc228", "mode": "0644", "owner": "root", "size": 108, "src": "/root/.ansible/tmp/ansible-tmp-1637690812.2709622-156623-28533950443182/source", "state": "file", "uid": 0}, "msg": "non-zero return code", "rc": 1, "start": "2021-11-23 10:06:58.353651", "stderr": "mmcrnsd: Name \"n1-1\" is not allowed.\nIt contains the following invalid special character: -\nmmcrnsd: Error found while processing stanza\n %nsd:\n device=/dev/vdb\n nsd=n1-1\n servers=sc1-n1\n usage=dataAndMetadata\n failureGroup=1\n pool=system\nmmcrnsd: File /var/mmfs/tmp/StanzaFile.new.fs1 does not contain any NSD descriptors or stanzas.\nmmcrnsd: Command failed. Examine previous error messages to determine cause.", "stderr_lines": ["mmcrnsd: Name \"n1-1\" is not allowed.", "It contains the following invalid special character: -", "mmcrnsd: Error found while processing stanza", " %nsd:", " device=/dev/vdb", " nsd=n1-1", " servers=sc1-n1", " usage=dataAndMetadata", " failureGroup=1", " pool=system", "mmcrnsd: File /var/mmfs/tmp/StanzaFile.new.fs1 does not contain any NSD descriptors or stanzas.", "mmcrnsd: Command failed. Examine previous error messages to determine cause."], "stdout": "", "stdout_lines": []} `` NO MORE HOSTS LEFT ***********************************************************************************************************************

Having the following lines in ansible.cfg does the trick:

[root@control1 ansible]# cat ansible.cfg
[defaults]
# Use the YAML callback plugin for human-readable formatting or error messages
stdout_callback = yaml

[root@control1 ansible]#

Now the same error message is nicely formatted:

TASK [core/cluster : storage | Create new NSDs] ******************************************************************************************
failed: [sc1-n1 -> sc1-n1] (item={'diff': {'before': {'path': '/var/mmfs/tmp/StanzaFile.new.fs1'}, 'after': {'path': '/var/mmfs/tmp/StanzaFile.new.fs1'}}, 'path': '/var/mmfs/tmp/StanzaFile.new.fs1', 'changed': False, 'uid': 0, 'gid': 0, 'owner': 'root', 'group': 'root', 'mode': '0644', 'state': 'file', 'size': 108, 'invocation': {'module_args': {'mode': None, 'follow': False, 'dest': '/var/mmfs/tmp/StanzaFile.new.fs1', '_original_basename': 'StanzaFile.j2', 'recurse': False, 'state': 'file', 'path': '/var/mmfs/tmp/StanzaFile.new.fs1', 'force': False, 'modification_time_format': '%Y%m%d%H%M.%S', 'access_time_format': '%Y%m%d%H%M.%S', 'unsafe_writes': False, '_diff_peek': None, 'src': None, 'modification_time': None, 'access_time': None, 'owner': None, 'group': None, 'seuser': None, 'serole': None, 'selevel': None, 'setype': None, 'attributes': None, 'content': None, 'backup': None, 'remote_src': None, 'regexp': None, 'delimiter': None, 'directory_mode': None}}, 'checksum': 'c33c980ddba2ac2a328d35cf3c68682a57625a61', 'dest': '/var/mmfs/tmp/StanzaFile.new.fs1', 'failed': False, 'item': 'fs1', 'ansible_loop_var': 'item'}) => changed=true
  ansible_loop_var: item
  cmd:
  - /usr/lpp/mmfs/bin/mmcrnsd
  - -F
  - /var/mmfs/tmp/StanzaFile.new.fs1
  - -v
  - 'yes'
  delta: '0:00:02.927420'
  end: '2021-11-23 10:11:26.775862'
  item:
    ansible_loop_var: item
    changed: false
    checksum: c33c980ddba2ac2a328d35cf3c68682a57625a61
    dest: /var/mmfs/tmp/StanzaFile.new.fs1
    diff:
      after:
        path: /var/mmfs/tmp/StanzaFile.new.fs1
      before:
        path: /var/mmfs/tmp/StanzaFile.new.fs1
    failed: false
    gid: 0
    group: root
    invocation:
      module_args:
        _diff_peek: null
        _original_basename: StanzaFile.j2
        access_time: null
        access_time_format: '%Y%m%d%H%M.%S'
        attributes: null
        backup: null
        content: null
        delimiter: null
        dest: /var/mmfs/tmp/StanzaFile.new.fs1
        directory_mode: null
        follow: false
        force: false
        group: null
        mode: null
        modification_time: null
        modification_time_format: '%Y%m%d%H%M.%S'
        owner: null
        path: /var/mmfs/tmp/StanzaFile.new.fs1
        recurse: false
        regexp: null
        remote_src: null
        selevel: null
        serole: null
        setype: null
        seuser: null
        src: null
        state: file
        unsafe_writes: false
    item: fs1
    mode: '0644'
    owner: root
    path: /var/mmfs/tmp/StanzaFile.new.fs1
    size: 108
    state: file
    uid: 0
  msg: non-zero return code
  rc: 1
  start: '2021-11-23 10:11:23.848442'
  stderr: |-
    mmcrnsd: Name "n1-1" is not allowed.
    It contains the following invalid special character:  -
    mmcrnsd: Error found while processing stanza
        %nsd:
          device=/dev/vdb
          nsd=n1-1
          servers=sc1-n1
          usage=dataAndMetadata
          failureGroup=1
          pool=system
    mmcrnsd: File /var/mmfs/tmp/StanzaFile.new.fs1 does not contain any NSD descriptors or stanzas.
    mmcrnsd: Command failed. Examine previous error messages to determine cause.
  stderr_lines: <omitted>
  stdout: ''
  stdout_lines: <omitted>

NO MORE HOSTS LEFT ***********************************************************************************************************************

Assuming that some users of the Spectrum Scale roles are new to Ansible and Spectrum Scale, a respective tip in the Troubleshooting section would be useful for beginners.

Additional suggestions:

  • Add a task which checks valid names for NSDs, filesystems, etc. I could envision that many new users choose to have a - in entity names.
  • Add respective comments in example files and playbooks.

troppens avatar Nov 23 '21 20:11 troppens

That's a good one - let me add a statement to the documentation...

acch avatar Nov 24 '21 16:11 acch