ceph-salt barfs out a Traceback when Salt gets "Valid metadata not found at specified URL" error from zypper

[smithfarm]

When repos are not fresh (and sometimes even when they are), zypper occasionally throws a "valid metadata not found at specified URL" error. Salt and ceph-salt could handle this more gracefully than they do now:

    master: Starting the execution of: salt -G 'ceph-salt:member' state.apply ceph-salt
    master: [2020-04-30 18:28:47.269958] [node2.octopus-4n] [STAGE] [BEGIN] Distribute SSH keys
    master: [2020-04-30 18:28:47.305720] [node2.octopus-4n] [STAGE] [END  ] Distribute SSH keys
    master: [2020-04-30 18:28:47.310786] [node2.octopus-4n] [STAGE] [BEGIN] Install and update required packages
    master: [2020-04-30 18:28:47.367788] [node1.octopus-4n] [STAGE] [BEGIN] Distribute SSH keys
    master: [2020-04-30 18:28:47.372835] [node3.octopus-4n] [STAGE] [BEGIN] Distribute SSH keys
    master: [2020-04-30 18:28:47.375437] [node1.octopus-4n] [STAGE] [END  ] Distribute SSH keys
    master: [2020-04-30 18:28:47.378390] [node1.octopus-4n] [STAGE] [BEGIN] Install and update required packages
    master: [2020-04-30 18:28:47.384565] [node3.octopus-4n] [STAGE] [END  ] Distribute SSH keys
    master: [2020-04-30 18:28:47.388160] [node3.octopus-4n] [STAGE] [BEGIN] Install and update required packages
    master: [2020-04-30 18:28:47.700442] [master.octopus-4] [STAGE] [BEGIN] Distribute SSH keys
    master: [2020-04-30 18:28:47.711976] [master.octopus-4] [STAGE] [END  ] Distribute SSH keys
    master: [2020-04-30 18:28:47.716377] [master.octopus-4] [STAGE] [BEGIN] Install and update required packages
    master: [2020-04-30 18:29:08.774148] [node2.octopus-4n] Finished with failures
    master:
    master: Failure in minion: node2.octopus-4node.com
    master: __id__: install required packages
    master: __run_num__: 9
    master: __sls__: ceph-salt.software
    master: changes: {}
    master: comment: "An exception occurred in this state: Traceback (most recent call last):\n\
    master:   \  File \"/usr/lib/python3.6/site-packages/salt/state.py\", line 1987, in call\n\
    master:   \    ret = self.states[cdata['full']](*cdata['args'], **cdata['kwargs'])\n  File\
    master:   \ \"/usr/lib/python3.6/site-packages/salt/loader.py\", line 2030, in wrapper\n \
    master:   \   return f(*args, **kwargs)\n  File \"/usr/lib/python3.6/site-packages/salt/states/pkg.py\"\
    master:   , line 1606, in installed\n    pkgs, refresh = _resolve_capabilities(pkgs, refresh=refresh,\
    master:   \ **kwargs)\n  File \"/usr/lib/python3.6/site-packages/salt/states/pkg.py\", line\
    master:   \ 944, in _resolve_capabilities\n    ret = __salt__['pkg.resolve_capabilities'](pkgs,\
    master:   \ refresh=refresh, **kwargs)\n  File \"/usr/lib/python3.6/site-packages/salt/modules/zypperpkg.py\"\
    master:   , line 2797, in resolve_capabilities\n    refresh_db(root)\n  File \"/usr/lib/python3.6/site-packages/salt/modules/zypperpkg.py\"\
    master:   , line 1300, in refresh_db\n    out = __zypper__(root=root).refreshable.call('refresh',\
    master:   \ '--force')\n  File \"/usr/lib/python3.6/site-packages/salt/modules/zypperpkg.py\"\
    master:   , line 351, in __call\n    raise CommandExecutionError('Zypper command failure:\
    master:   \ {0}'.format(self.error_msg))\nsalt.exceptions.CommandExecutionError: Zypper command\
    master:   \ failure: Signature verification failed for file 'repomd.xml' from repository 'octopus-repo1'.\n\
    master:   Repository 'octopus-repo1' is invalid.\n[octopus-repo1|https://download.opensuse.org/repositories/filesystems:/ceph:/octopus:/upstream/openSUSE_Leap_15.2]\
    master:   \ Valid metadata not found at specified URL\nHistory:\n - Signature verification\
    master:   \ failed for repomd.xml\n - Can't provide /repodata/repomd.xml\n\nPlease check if\
    master:   \ the URIs defined for this repository are pointing to a valid repository.\nSkipping\
    master:   \ repository 'octopus-repo1' because of the above error.\nSome of the repositories\
    master:   \ have not been refreshed because of an error.\n"
    master: duration: 14717.992
    master: name: install required packages
    master: result: false
    master: start_time: '20:28:53.730077'
    master: state: pkg_|-install required packages_|-install required packages_|-installed
    master: [2020-04-30 18:29:08.913287] [node1.octopus-4n] Finished with failures

[sebastian-philipp]

properly formatted:

An exception occurred in this state: Traceback (most recent call last):

  File "/usr/lib/python3.6/site-packages/salt/state.py", line 1987, in call
    ret = self.states[cdata['full']](*cdata['args'], **cdata['kwargs'])
  File "/usr/lib/python3.6/site-packages/salt/loader.py", line 2030, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/states/pkg.py", line 1606, in installed
    pkgs, refresh = _resolve_capabilities(pkgs, refresh=refresh, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/states/pkg.py", line 944, in _resolve_capabilities
    ret = __salt__['pkg.resolve_capabilities'](pkgs, refresh=refresh, **kwargs)
  File "/usr/lib/python3.6/site-packages/salt/modules/zypperpkg.py", line 2797, in resolve_capabilities
    refresh_db(root)
  File "/usr/lib/python3.6/site-packages/salt/modules/zypperpkg.py", line 1300, in refresh_db
    out = __zypper__(root=root).refreshable.call('refresh', '--force')
  File "/usr/lib/python3.6/site-packages/salt/modules/zypperpkg.py", line 351, in __call
    raise CommandExecutionError('Zypper command failure:

salt.exceptions.CommandExecutionError: Zypper command
 failure: Signature verification failed for file 'repomd.xml' from repository 'octopus-repo1'.

 Repository 'octopus-repo1' is invalid.

 [octopus-repo1|https://download.opensuse.org/repositories/filesystems:/ceph:/octopus:/upstream/openSUSE_Leap_15.2]
 Valid metadata not found at specified URL
History:
 - Signature verification
 failed for repomd.xml
 - Can't provide /repodata/repomd.xml

Please check if the URIs defined for this repository are pointing to a valid repository.
Skipping repository 'octopus-repo1' because of the above error.
Some of the repositories have not been refreshed because of an error.

[smithfarm]

More fun with the Salt Zypper module!