[Build 2024111416-4.2] openQA test fails in upload_packages
Observation
openQA test in scenario qubesos-4.2-securedrop-x86_64-securedrop_install@64bit fails in upload_packages
curl: (28) Operation timed out after 10003 miliseconds with 0 bytes received
Test suite description
Install SecureDrop
Reproducible
Fails since (at least) Build 2024111416-4.2 (current job)
Expected result
Last good: 2024111309-4.2 (or more recent)
Further details
Always latest result in this scenario: latest
See more info at https://github.com/QubesOS/openqa-tests-qubesos/pull/25#issuecomment-2476280844
Adding some context captured on an old commit that won't make it into main:
In a specific set of circumstances dropped packets were causing the log uploads to take a long time and eventually fail. Even though the client would send retries, they would not reach the host.
A failok flag was added to the logs-uploading method, as a a temporary measure to let the tests proceed even though the log uploading failed.
@deeplow here is the issue I mentioned - if you hit such issue, add the issue URL as the job comment
Recent related failures:
| ?column? | t_finished | state | result | test | host |
|---|---|---|---|---|---|
| https://openqa.qubes-os.org/tests/159898 | 2025-11-19 10:08:45 | done | failed | system_tests_basic_vm_qrexec_gui_xfs | ramen |
| https://openqa.qubes-os.org/tests/159885 | 2025-11-19 07:25:53 | done | failed | system_tests_basic_vm_qrexec_gui_xfs | ramen |
| https://openqa.qubes-os.org/tests/159851 | 2025-11-19 06:24:35 | done | failed | system_tests_guivm_gui_interactive | herring |
| https://openqa.qubes-os.org/tests/159763 | 2025-11-18 13:25:23 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | herring |
| https://openqa.qubes-os.org/tests/159739 | 2025-11-18 12:51:48 | done | failed | system_tests_basic_vm_qrexec_gui | herring |
| https://openqa.qubes-os.org/tests/159757 | 2025-11-18 12:31:32 | done | failed | system_tests_guivm_vnc_gui_interactive | onigiri |
| https://openqa.qubes-os.org/tests/159670 | 2025-11-18 07:14:30 | done | failed | system_tests_qrexec | herring |
| https://openqa.qubes-os.org/tests/159681 | 2025-11-18 05:59:27 | done | failed | system_tests_update | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/159532 | 2025-11-16 04:43:17 | done | failed | system_tests_network_ipv6 | herring |
| https://openqa.qubes-os.org/tests/159561 | 2025-11-16 04:12:40 | done | failed | system_tests_basic_vm_qrexec_gui_btrfs | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/159562 | 2025-11-16 03:06:23 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | ramen |
| https://openqa.qubes-os.org/tests/159469 | 2025-11-14 17:16:55 | done | failed | system_tests_basic_vm_qrexec_gui_xfs | onigiri |
| https://openqa.qubes-os.org/tests/159468 | 2025-11-14 17:14:38 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | ramen |
| https://openqa.qubes-os.org/tests/159467 | 2025-11-14 17:11:09 | done | failed | system_tests_basic_vm_qrexec_gui_btrfs | ramen |
| https://openqa.qubes-os.org/tests/159350 | 2025-11-13 05:48:57 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | herring |
| https://openqa.qubes-os.org/tests/159160 | 2025-11-11 16:02:53 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | ramen |
| https://openqa.qubes-os.org/tests/159153 | 2025-11-11 13:46:03 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | herring |
| https://openqa.qubes-os.org/tests/159140 | 2025-11-11 12:15:19 | done | failed | system_tests_basic_vm_qrexec_gui_btrfs | herring |
| https://openqa.qubes-os.org/tests/159141 | 2025-11-11 12:05:07 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | ramen |
| https://openqa.qubes-os.org/tests/159152 | 2025-11-11 11:46:07 | done | failed | system_tests_guivm_vnc_gui_interactive | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/159130 | 2025-11-11 11:45:36 | done | failed | system_tests_backupdispvm | onigiri |
| https://openqa.qubes-os.org/tests/159021 | 2025-11-11 06:08:12 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | herring |
| https://openqa.qubes-os.org/tests/159060 | 2025-11-11 06:07:31 | done | failed | system_tests_update | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/158845 | 2025-11-10 14:32:03 | done | failed | system_tests_network | herring |
| https://openqa.qubes-os.org/tests/158811 | 2025-11-10 11:57:38 | done | failed | system_tests_update | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/158796 | 2025-11-09 13:25:30 | done | failed | system_tests_splitgpg | ramen |
| https://openqa.qubes-os.org/tests/158674 | 2025-11-05 16:59:43 | done | failed | system_tests_dispvm | herring |
| https://openqa.qubes-os.org/tests/158605 | 2025-11-05 05:35:32 | done | failed | system_tests_dispvm | herring |
| https://openqa.qubes-os.org/tests/158632 | 2025-11-05 04:42:32 | done | failed | system_tests_basic_vm_qrexec_gui_xfs | ramen |
| https://openqa.qubes-os.org/tests/158494 | 2025-11-04 05:29:51 | done | failed | system_tests_whonix | ramen |
| https://openqa.qubes-os.org/tests/158489 | 2025-11-03 22:28:56 | done | failed | system_tests_dispvm | onigiri |
| https://openqa.qubes-os.org/tests/158490 | 2025-11-03 22:15:46 | done | failed | system_tests_basic_vm_qrexec_gui_zfs | herring |
| https://openqa.qubes-os.org/tests/158456 | 2025-11-03 21:24:03 | done | failed | system_tests_dispvm | herring |
| https://openqa.qubes-os.org/tests/158483 | 2025-11-03 20:28:16 | done | failed | system_tests_basic_vm_qrexec_gui_xfs | onigiri |
| https://openqa.qubes-os.org/tests/158481 | 2025-11-03 20:23:46 | done | failed | system_tests_basic_vm_qrexec_gui_btrfs | ramen |
| https://openqa.qubes-os.org/tests/158443 | 2025-11-03 20:11:58 | done | failed | system_tests_network | ramen |
| https://openqa.qubes-os.org/tests/158004 | 2025-10-30 07:04:33 | done | passed | system_tests_gui_interactive_preloaded | onigiri |
| https://openqa.qubes-os.org/tests/157976 | 2025-10-30 06:59:25 | done | passed | system_tests_gui_interactive | ramen |
| https://openqa.qubes-os.org/tests/157787 | 2025-10-28 12:09:56 | done | failed | system_tests_extra | ramen |
| https://openqa.qubes-os.org/tests/157762 | 2025-10-28 07:08:39 | done | failed | system_tests_extra | ramen |
| https://openqa.qubes-os.org/tests/157755 | 2025-10-28 06:20:42 | done | failed | system_tests_basic_vm_qrexec_gui | ramen |
| https://openqa.qubes-os.org/tests/157757 | 2025-10-28 06:04:18 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | ramen |
| https://openqa.qubes-os.org/tests/157756 | 2025-10-28 05:58:34 | done | failed | system_tests_basic_vm_qrexec_gui_btrfs | ramen |
| https://openqa.qubes-os.org/tests/157737 | 2025-10-28 05:26:09 | done | failed | system_tests_whonix | onigiri |
| https://openqa.qubes-os.org/tests/157685 | 2025-10-27 08:05:05 | done | failed | system_tests_basic_vm_qrexec_gui_zfs | ramen |
| https://openqa.qubes-os.org/tests/157674 | 2025-10-27 06:01:33 | done | failed | system_tests_guivm_gui_interactive | ramen |
| https://openqa.qubes-os.org/tests/157379 | 2025-10-24 09:57:06 | done | failed | system_tests_basic_vm_qrexec_gui | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/157308 | 2025-10-23 15:16:53 | done | failed | system_tests_network_updates | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/157195 | 2025-10-22 20:53:59 | done | failed | system_tests_guivm_gui_interactive | ramen |
| https://openqa.qubes-os.org/tests/156736 | 2025-10-21 08:37:34 | done | failed | system_tests_network_updates | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/156730 | 2025-10-21 08:25:51 | done | failed | system_tests_extra | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/156734 | 2025-10-21 07:07:23 | done | failed | system_tests_network | onigiri |
| https://openqa.qubes-os.org/tests/156725 | 2025-10-21 06:36:55 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | ramen |
| https://openqa.qubes-os.org/tests/156427 | 2025-10-16 22:17:16 | done | failed | system_tests_network | ramen |
| https://openqa.qubes-os.org/tests/156227 | 2025-10-14 19:27:09 | done | failed | system_tests_basic_vm_qrexec_gui_zfs | ramen |
| https://openqa.qubes-os.org/tests/156228 | 2025-10-14 17:32:03 | done | failed | system_tests_guivm_vnc_gui_interactive | onigiri |
| https://openqa.qubes-os.org/tests/156170 | 2025-10-14 07:40:39 | done | failed | system_tests_network_updates | ramen |
| https://openqa.qubes-os.org/tests/156157 | 2025-10-14 06:31:44 | done | failed | system_tests_basic_vm_qrexec_gui | ramen |
| https://openqa.qubes-os.org/tests/155873 | 2025-10-11 10:19:25 | done | failed | system_tests_network | ramen |
| https://openqa.qubes-os.org/tests/155571 | 2025-10-07 17:34:21 | done | failed | system_tests_guivm_vnc_gui_interactive | ramen |
| https://openqa.qubes-os.org/tests/155519 | 2025-10-07 09:06:53 | done | failed | system_tests_guivm_vnc_gui_interactive | onigiri |
| https://openqa.qubes-os.org/tests/155192 | 2025-10-05 01:24:30 | done | failed | system_tests_splitgpg | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/155193 | 2025-10-05 01:14:46 | done | failed | system_tests_extra | onigiri |
| https://openqa.qubes-os.org/tests/154818 | 2025-10-02 20:41:32 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | onigiri |
| https://openqa.qubes-os.org/tests/154792 | 2025-10-02 20:17:36 | done | failed | system_tests_dispvm | onigiri |
| https://openqa.qubes-os.org/tests/154503 | 2025-09-30 06:48:46 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | onigiri |
| https://openqa.qubes-os.org/tests/154175 | 2025-09-23 06:31:14 | done | failed | system_tests_basic_vm_qrexec_gui_btrfs | ramen |
| https://openqa.qubes-os.org/tests/154176 | 2025-09-23 06:10:21 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | ramen |
| https://openqa.qubes-os.org/tests/153865 | 2025-09-19 01:59:23 | done | failed | system_tests_basic_vm_qrexec_gui_xfs | onigiri |
| https://openqa.qubes-os.org/tests/153729 | 2025-09-18 08:00:15 | done | failed | system_tests_basic_vm_qrexec_gui_btrfs | onigiri |
| https://openqa.qubes-os.org/tests/153696 | 2025-09-18 07:34:39 | done | failed | system_tests_extra | onigiri |
| https://openqa.qubes-os.org/tests/153589 | 2025-09-17 21:53:16 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | onigiri |
| https://openqa.qubes-os.org/tests/153581 | 2025-09-17 19:10:07 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | ramen |
| https://openqa.qubes-os.org/tests/153555 | 2025-09-17 18:31:17 | done | failed | system_tests_dispvm | ramen |
| https://openqa.qubes-os.org/tests/153547 | 2025-09-17 18:18:36 | done | failed | system_tests_extra | ramen |
| https://openqa.qubes-os.org/tests/153362 | 2025-09-16 07:51:41 | done | failed | system_tests_network_updates | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/153364 | 2025-09-16 07:17:08 | done | failed | system_tests_qrexec | ramen |
| https://openqa.qubes-os.org/tests/153351 | 2025-09-16 06:39:40 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | onigiri |
| https://openqa.qubes-os.org/tests/153350 | 2025-09-16 06:23:58 | done | failed | system_tests_basic_vm_qrexec_gui_btrfs | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/153340 | 2025-09-16 05:35:07 | done | failed | system_tests_guivm_vnc_gui_interactive | ramen |
| https://openqa.qubes-os.org/tests/153286 | 2025-09-16 03:54:16 | done | failed | system_tests_network | ramen |
| https://openqa.qubes-os.org/tests/153266 | 2025-09-15 19:16:16 | done | failed | system_tests_basic_vm_qrexec_gui_xfs | onigiri |
| https://openqa.qubes-os.org/tests/153236 | 2025-09-15 18:27:09 | done | failed | system_tests_qrexec | ramen |
| https://openqa.qubes-os.org/tests/152831 | 2025-09-12 08:28:13 | done | failed | system_tests_network_updates | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/152833 | 2025-09-12 07:45:15 | done | failed | system_tests_dispvm | ramen |
| https://openqa.qubes-os.org/tests/152828 | 2025-09-12 07:10:09 | done | failed | system_tests_usbproxy | ramen |
| https://openqa.qubes-os.org/tests/152238 | 2025-09-07 02:52:05 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/152212 | 2025-09-07 01:35:18 | done | failed | system_tests_backupdispvm | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/152218 | 2025-09-07 01:28:12 | done | failed | system_tests_basic_vm_qrexec_gui_zfs | onigiri |
| https://openqa.qubes-os.org/tests/152199 | 2025-09-07 00:15:42 | done | failed | system_tests_basic_vm_qrexec_gui | salmon.qubes-os.org |
| https://openqa.qubes-os.org/tests/152150 | 2025-09-06 04:51:24 | done | failed | system_tests_basic_vm_qrexec_gui_ext4 | onigiri |
| https://openqa.qubes-os.org/tests/152130 | 2025-09-06 04:35:24 | done | failed | system_tests_basic_vm_qrexec_gui_zfs | ramen |
| https://openqa.qubes-os.org/tests/152111 | 2025-09-06 03:38:04 | done | failed | system_tests_basic_vm_qrexec_gui | ramen |
In a very limited testing, it appears as ethtool -K ens6 sg off in sys-net helps.
The ethtool change helps quite a lot, but the issue still happens, especially in collect_logs stage at the end of the the test job.