Investigate and fix image build failures
Our recent automatic image builds are frequently failing due to network errors, requiring multiple manual retries. This process takes too much time to get a usable build, which is unacceptable when we need to deploy an urgent fix.
Example: https://github.com/packit/packit-service/actions/runs/16138635756/job/45547324065
- [ ] Investigate the root cause of these transient network errors during the build process.
- [ ] Implement a fix or a reliable workaround to ensure stability, consider alternative solutions to what we use now
The errors seem to usually occur in the ansible dnf module that produces a traceback like this:
Traceback (most recent call last):
File "/src/files/~packit/.ansible/tmp/ansible-tmp-1751970564.7063925-154-159875683464028/AnsiballZ_dnf5.py", line 107, in <module>
_ansiballz_main()
~~~~~~~~~~~~~~~^^
File "/src/files/~packit/.ansible/tmp/ansible-tmp-1751970564.7063925-154-159875683464028/AnsiballZ_dnf5.py", line 99, in _ansiballz_main
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/src/files/~packit/.ansible/tmp/ansible-tmp-1751970564.7063925-154-159875683464028/AnsiballZ_dnf5.py", line 47, in invoke_module
runpy.run_module(mod_name='ansible.modules.dnf5', init_globals=dict(_module_fqn='ansible.modules.dnf5', _modlib_path=modlib_path),
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
run_name='__main__', alter_sys=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen runpy>", line 226, in run_module
File "<frozen runpy>", line 98, in _run_module_code
File "<frozen runpy>", line 88, in _run_code
File "/tmp/ansible_ansible.legacy.dnf5_payload_0aw0s5cy/ansible_ansible.legacy.dnf5_payload.zip/ansible/modules/dnf5.py", line 777, in <module>
File "/tmp/ansible_ansible.legacy.dnf5_payload_0aw0s5cy/ansible_ansible.legacy.dnf5_payload.zip/ansible/modules/dnf5.py", line 773, in main
File "/tmp/ansible_ansible.legacy.dnf5_payload_0aw0s5cy/ansible_ansible.legacy.dnf5_payload.zip/ansible/modules/dnf5.py", line 735, in run
File "/usr/lib64/python3.13/site-packages/libdnf5/base.py", line 867, in download
return _base.Transaction_download(self)
~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
libdnf5.exception.RepoPackageDownloadErrorNested: Failed to download packages
That doesn't give any details about the reason of the failure, we should try to figure out if we can enable some debug logging.
Try to use a simple dirty solution here. Create followup cards for a proper longterm fix.
I wasn't able to replicate this behaviour. Putting it back to the backlog.