kickstart-tests icon indicating copy to clipboard operation
kickstart-tests copied to clipboard

A flake on rhel-10: "01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:gi.overrides.BlockDev.MpathError: Process reported exit code 1: Job for multipathd.service failed."

Open rvykydal opened this issue 1 year ago • 2 comments

First hit: 10-07-2024

01:51:51,732 NOTICE audit:AVC avc:  denied  { name_connect } for  pid=3055 comm="chronyd" dest=4460 scontext=system_u:system_r:kernel_t:s0 tcontext=system_u:object_r:ntske_port_t:s0 tclass=tcp_socket permissive=1
01:51:51,732 WARNING brltty:unsupported generic resource identifier: bluetooth:
01:51:52,022 WARNING org.fedoraproject.Anaconda.Modules.Storage:INFO:program:stdout[2]:
01:51:52,023 WARNING org.fedoraproject.Anaconda.Modules.Storage:INFO:program:stderr[2]: Job for multipathd.service failed.
01:51:52,023 WARNING org.fedoraproject.Anaconda.Modules.Storage:See "systemctl status multipathd.service" and "journalctl -xeu multipathd.service" for details.
01:51:52,023 WARNING org.fedoraproject.Anaconda.Modules.Storage:INFO:program:...done [2] (exit code: 1)
01:51:52,027 WARNING org.fedoraproject.Anaconda.Modules.Storage:INFO:anaconda.core.threads:Thread Failed: AnaTaskThread-ScanDevicesTask-1 (140259463530176)
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:ERROR:anaconda.modules.common.task.task:Thread AnaTaskThread-ScanDevicesTask-1 has failed: Traceback (most recent call last):
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib64/python3.13/site-packages/gi/overrides/BlockDev.py", line 1250, in wrapped
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    ret = orig_obj(*args, **kwargs)
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:gi.repository.GLib.GError: g-bd-utils-exec-error-quark: Process reported exit code 1: Job for multipathd.service failed.
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:See "systemctl status multipathd.service" and "journalctl -xeu multipathd.service" for details.
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage: (0)
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:During handling of the above exception, another exception occurred:
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:Traceback (most recent call last):
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib64/python3.13/site-packages/pyanaconda/core/threads.py", line 280, in run
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    threading.Thread.run(self)
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    ~~~~~~~~~~~~~~~~~~~~^^^^^^
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib64/python3.13/threading.py", line 992, in run
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    self._target(*self._args, **self._kwargs)
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib64/python3.13/site-packages/pyanaconda/modules/common/task/task.py", line 94, in _thread_run_callback
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    self._task_run_callback()
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    ~~~~~~~~~~~~~~~~~~~~~~~^^
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib64/python3.13/site-packages/pyanaconda/modules/common/task/task.py", line 107, in _task_run_callback
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    self._set_result(self.run())
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:                     ~~~~~~~~^^
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib64/python3.13/site-packages/pyanaconda/modules/storage/reset.py", line 64, in run
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    self._reset_storage(self._storage)
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib64/python3.13/site-packages/pyanaconda/modules/storage/reset.py", line 84, in _reset_storage
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    storage.reset()
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    ~~~~~~~~~~~~~^^
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib/python3.13/site-packages/blivet/threads.py", line 49, in run_with_lock
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    return m(*args, **kwargs)
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib64/python3.13/site-packages/pyanaconda/modules/storage/devicetree/model.py", line 265, in reset
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    super().reset(cleanup_only=cleanup_only)
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib/python3.13/site-packages/blivet/threads.py", line 49, in run_with_lock
01:51:52,043 WARNING org.fedoraproject.Anaconda.Modules.Storage:    return m(*args, **kwargs)
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib/python3.13/site-packages/blivet/blivet.py", line 155, in reset
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:    self.devicetree.populate(cleanup_only=cleanup_only)
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib/python3.13/site-packages/blivet/threads.py", line 49, in run_with_lock
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:    return m(*args, **kwargs)
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib/python3.13/site-packages/blivet/populator/populator.py", line 444, in populate
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:    self._populate()
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:    ~~~~~~~~~~~~~~^^
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib/python3.13/site-packages/blivet/threads.py", line 49, in run_with_lock
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:    return m(*args, **kwargs)
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib/python3.13/site-packages/blivet/populator/populator.py", line 457, in _populate
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:    blockdev.mpath.set_friendly_names(flags.multipath_friendly_names)
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:  File "/usr/lib64/python3.13/site-packages/gi/overrides/BlockDev.py", line 1272, in wrapped
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:    raise transform[1](msg)
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:gi.overrides.BlockDev.MpathError: Process reported exit code 1: Job for multipathd.service failed.
01:51:52,044 WARNING org.fedoraproject.Anaconda.Modules.Storage:See "systemctl status multipathd.service" and "journalctl -xeu multipathd.service" for details.
01:51:52,045 WARNING org.fedoraproject.Anaconda.Modules.Storage:INFO:anaconda.core.threads:Thread Done: AnaTaskThread-ScanDevicesTask-1 (140259463530176)
01:51:52,096 WARNING org.fedoraproject.Anaconda.Modules.Storage:WARNING:dasbus.server.handler:The call org.fedoraproject.Anaconda.Task.Finish has failed with an exception:

kstest.log virt-install.log

rvykydal avatar Oct 09 '24 06:10 rvykydal

Quite a prominent flake, but the rate 1/158 runs last week is not bad enough to cause double flakes. Attaching a recent log. virt-install.log

rvykydal avatar Nov 04 '24 11:11 rvykydal

This will be fixed in blivet, see https://github.com/storaged-project/blivet/pull/1326 There is no need to raise an exception just because we failed to set some multipath feature during reset.

vojtechtrefny avatar Dec 13 '24 12:12 vojtechtrefny