openQA icon indicating copy to clipboard operation
openQA copied to clipboard

Test works, but live streaming return 504

Open trdthg opened this issue 1 year ago • 10 comments

Hi, I'm trying to use openQA in container (I have used it in openSUSE VM before), after I solved unable to upgrade ws to command server, test started successful, I can see the current module xxx at xxx changes, and get all testresult after the end. But streaming (as well as livelog) just return 504 after a while. How should I debug? 😵

trdthg avatar Dec 03 '23 14:12 trdthg

Does the upload of the logs work at the end of the test run? To debug this, I'd have a look at the logs on the web UI services. I think openqa-webui.service is the most relevant one.

Martchus avatar Dec 04 '23 10:12 Martchus

Does the upload of the logs work at the end of the test run?

Yes, all logs, video and shortcuts

image image

logs on the web UI services

I run a mini test case, here is a screenshot(docker-compsoe, webui and worker) image

test case:

# Copyright 2014-2018 SUSE LLC
# SPDX-License-Identifier: GPL-2.0-or-later

use base 'basetest';
use strict;
use testapi;

sub run {
    # wait for bootloader to appear
    assert_screen 'bootloader';

    # conclude test early if there's no boot media
    return undef if match_has_tag 'no-boot-media';

    # press enter to boot right away
    send_key 'ret';

    # wait for the desktop to appear
    assert_screen 'desktop', 300;
}

1;

all logs:

  • Video: nothing
  • autoinst-log.txt
    penQA
    All Tests](http://192.168.0.76/tests)
    Job Groups](http://192.168.0.76/tests/55/logfile?filename=autoinst-log.txt#)
    ype to search
    Logged in as Demo ](http://192.168.0.76/tests/55/logfile?filename=autoinst-log.txt#)
    
    utoinst-log.txt
    
    2023-12-08T09:04:15.533543Z] [info] [pid:46] +++ setup notes +++
    2023-12-08T09:04:15.533859Z] [info] [pid:46] Running on 6670f76993d0:1 (Linux 6.1.62 #1-NixOS SMP PREEMPT_DYNAMIC Wed Nov  8 13:11:05 UTC 2023 x86_64)
    2023-12-08T09:04:15.537571Z] [debug] [pid:49] +++ worker notes +++
    2023-12-08T09:04:15.807747Z] [debug] [pid:49] Current version is 4.6.1701427827.ab6274c [interface v40]
    2023-12-08T09:04:15.819858Z] [debug] [pid:49] git hash in demohw: d0857d708878e6f1558f66ba29a6b642c534c186
    2023-12-08T09:04:15.824358Z] [debug] [pid:49] scheduling boot tests/boot.pm
    2023-12-08T09:04:15.828093Z] [info] [pid:58] cmdsrv: daemon reachable under [http://*:20013/RB81NKvnfvSxjXbQ/](http://%2A:20013/RB81NKvnfvSxjXbQ/)
    2023-12-08T09:04:15.829296Z] [info] [pid:58] Listening at "[http://0.0.0.0:20013"](http://0.0.0.0:20013/)
    eb application available at http://0.0.0.0:20013/
    2023-12-08T09:04:15.836565Z] [debug] [pid:49] git hash in demohw/needles: d0857d708878e6f1558f66ba29a6b642c534c186
    2023-12-08T09:04:15.836653Z] [debug] [pid:49] init needles from demohw/needles
    2023-12-08T09:04:15.837210Z] [debug] [pid:49] loaded 1 needles
    2023-12-08T09:04:16.024957Z] [debug] [pid:49] 49: channel_out 15, channel_in 14
    2023-12-08T09:04:16.051618Z] [debug] [pid:96] 96: cmdpipe 13, rsppipe 16
    2023-12-08T09:04:16.051760Z] [debug] [pid:96] started mgmt loop with pid 96
    2023-12-08T09:04:16.171920Z] [debug] [pid:96] qemu version detected: 7.1.0
    2023-12-08T09:04:16.173485Z] [debug] [pid:96] running `/usr/bin/chattr +C /var/lib/openqa/pool/1/raid`
    2023-12-08T09:04:16.182562Z] [debug] [pid:96] Fatal error in command `/usr/bin/chattr +C /var/lib/openqa/pool/1/raid`: open3: exec of /usr/bin/chattr +C /var/lib/openqa/pool/1/raid failed: No such file or directory at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 127.
    
    2023-12-08T09:04:16.182711Z] [debug] [pid:96] Configuring storage controllers and block devices
    2023-12-08T09:04:16.183432Z] [debug] [pid:96] Initializing block device images
    2023-12-08T09:04:16.183648Z] [debug] [pid:96] running `/usr/bin/qemu-img create -f qcow2 /var/lib/openqa/pool/1/raid/hd0 100G`
    2023-12-08T09:04:16.261746Z] [debug] [pid:96] Formatting '/var/lib/openqa/pool/1/raid/hd0', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=107374182400 lazy_refcounts=off refcount_bits=16
    2023-12-08T09:04:16.261872Z] [debug] [pid:96] init_blockdev_images: Finished creating block devices
    2023-12-08T09:04:16.262802Z] [debug] [pid:96] starting: /usr/bin/qemu-system-x86_64 -device VGA,edid=on,xres=1024,yres=768 -only-migratable -chardev ringbuf,id=serial0,logfile=serial0,logappend=on -serial chardev:serial0 -audiodev none,id=snd0 -device intel-hda -device hda-output,audiodev=snd0 -global isa-fdc.fdtypeA=none -m 1024 -netdev user,id=qanet0 -device virtio-net,netdev=qanet0,mac=52:54:00:12:34:56 -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 -boot once=d -device qemu-xhci -device usb-tablet -smp 1 -enable-kvm -no-shutdown -vnc :91,share=force-shared -device virtio-serial -chardev pipe,id=virtio_console,path=virtio_console,logfile=virtio_console.log,logappend=on -device virtconsole,chardev=virtio_console,name=org.openqa.console.virtio_console -chardev pipe,id=virtio_console_user,path=virtio_console_user,logfile=virtio_console_user.log,logappend=on -device virtconsole,chardev=virtio_console_user,name=org.openqa.console.virtio_console_user -chardev socket,path=qmp_socket,server=on,wait=off,id=qmp_socket,logfile=qmp_socket.log,logappend=on -qmp chardev:qmp_socket -S -device virtio-scsi-pci,id=scsi0 -blockdev driver=file,node-name=hd0-file,filename=/var/lib/openqa/pool/1/raid/hd0,cache.no-flush=on -blockdev driver=qcow2,node-name=hd0,file=hd0-file,cache.no-flush=on,discard=unmap -device virtio-blk,id=hd0-device,drive=hd0,serial=hd0
    2023-12-08T09:04:16.264745Z] [debug] [pid:96] Waiting for 0 attempts
    2023-12-08T09:04:17.265662Z] [debug] [pid:96] Waiting for 1 attempts
    2023-12-08T09:04:18.266435Z] [debug] [pid:96] Finished after 2 attempts
    2023-12-08T09:04:18.278296Z] [debug] [pid:96] Establishing VNC connection to localhost:5991
    2023-12-08T09:04:18.331380Z] [debug] [pid:96] pointer type 0 0 640 480 -257
    2023-12-08T09:04:18.331563Z] [debug] [pid:96] led state 0 1 1 -261
    2023-12-08T09:04:18.334663Z] [debug] [pid:96] Start CPU
    2023-12-08T09:04:18.334899Z] [debug] [pid:96] EVENT {"data":{"client":{"family":"ipv4","host":"127.0.0.1","service":"49318","websocket":false},"server":{"auth":"none","family":"ipv4","host":"0.0.0.0","service":"5991","websocket":false}},"event":"VNC_CONNECTED","timestamp":{"microseconds":278852,"seconds":1702026258}}
    2023-12-08T09:04:18.334960Z] [debug] [pid:96] EVENT {"data":{"client":{"family":"ipv4","host":"127.0.0.1","service":"49318","websocket":false},"server":{"auth":"none","family":"ipv4","host":"0.0.0.0","service":"5991","websocket":false}},"event":"VNC_INITIALIZED","timestamp":{"microseconds":279801,"seconds":1702026258}}
    2023-12-08T09:04:18.335145Z] [debug] [pid:96] EVENT {"event":"RESUME","timestamp":{"microseconds":335075,"seconds":1702026258}}
    OT GO
    
    2023-12-08T09:04:18.338441Z] [debug] [pid:96] QEMU: QEMU emulator version 7.1.0 (SUSE Linux Enterprise 15)
    2023-12-08T09:04:18.338505Z] [debug] [pid:96] QEMU: Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers
    2023-12-08T09:04:18.341054Z] [debug] [pid:66] Snapshots are supported
    2023-12-08T09:04:18.342482Z] [debug] [pid:66] ||| starting boot tests/boot.pm
    2023-12-08T09:04:18.344598Z] [debug] [pid:66] tests/boot.pm:10 called testapi::assert_screen
    2023-12-08T09:04:18.345040Z] [debug] [pid:66] <<< testapi::assert_screen(mustmatch="bootloader", timeout=30)
    2023-12-08T09:04:19.346304Z] [debug] [pid:96] no match: 29.0s, best candidate: bootloader (0.00)
    2023-12-08T09:04:20.340941Z] [debug] [pid:96] no match: 28.0s, best candidate: bootloader (0.00)
    2023-12-08T09:04:21.340683Z] [debug] [pid:96] no match: 27.0s, best candidate: bootloader (0.00)
    2023-12-08T09:04:22.341846Z] [debug] [pid:96] no match: 26.0s, best candidate: bootloader (0.00)
    2023-12-08T09:04:23.364623Z] [debug] [pid:96] no match: 25.0s, best candidate: bootloader (0.00)
    2023-12-08T09:04:24.343855Z] [debug] [pid:96] no match: 24.0s, best candidate: bootloader (0.00)
    2023-12-08T09:04:25.345611Z] [debug] [pid:96] no match: 23.0s, best candidate: bootloader (0.00)
    2023-12-08T09:04:26.346836Z] [debug] [pid:96] no match: 22.0s, best candidate: bootloader (0.00)
    2023-12-08T09:04:27.347840Z] [debug] [pid:96] no match: 21.0s, best candidate: bootloader (0.00)
    2023-12-08T09:04:28.349501Z] [debug] [pid:96] no match: 19.9s, best candidate: bootloader (0.00)
    2023-12-08T09:04:29.391914Z] [debug] [pid:66] >>> testapi::_handle_found_needle: found bootloader, similarity 1.00 @ 0/370
    2023-12-08T09:04:29.394513Z] [debug] [pid:66] ||| finished boot tests (runtime: 11 s)
    2023-12-08T09:04:29.396599Z] [debug] [pid:49] stopping autotest process 66
    2023-12-08T09:04:29.400070Z] [debug] [pid:49] [autotest] process exited: 0
    2023-12-08T09:04:29.501085Z] [debug] [pid:49] done with autotest process
    2023-12-08T09:04:29.501781Z] [debug] [pid:49] stopping command server 58 because test execution ended
    2023-12-08T09:04:29.501851Z] [debug] [pid:49] isotovideo: informing websocket clients before stopping command server: [http://127.0.0.1:20013/RB81NKvnfvSxjXbQ/broadcast](http://127.0.0.1:20013/RB81NKvnfvSxjXbQ/broadcast%3C/span%3E)
    2023-12-08T09:04:29.508933Z] [debug] [pid:49] commands process exited: 0
    2023-12-08T09:04:29.609619Z] [debug] [pid:49] done with command server
    2023-12-08T09:04:29.609776Z] [debug] [pid:49] isotovideo done
    2023-12-08T09:04:29.611094Z] [debug] [pid:96] QEMU status is not 'shutdown', it is 'running'
    2023-12-08T09:04:29.611244Z] [debug] [pid:49] backend shutdown state: 
    2023-12-08T09:04:29.611564Z] [info] [pid:96] ::: OpenQA::Qemu::Proc::save_state: Saving QEMU state to qemu_state.json
    2023-12-08T09:04:30.652420Z] [debug] [pid:96] Passing remaining frames to the video encoder
    2023-12-08T09:04:30.667883Z] [debug] [pid:96] Waiting for video encoder to finalize the video
    2023-12-08T09:04:30.667952Z] [debug] [pid:96] The built-in video encoder (pid 127) terminated
    2023-12-08T09:04:30.668285Z] [debug] [pid:96] QEMU: qemu-system-x86_64: terminating on signal 15 from pid 96 (/usr/bin/isotovideo: backen)
    2023-12-08T09:04:30.668828Z] [debug] [pid:96] sending magic and exit
    2023-12-08T09:04:30.669063Z] [debug] [pid:49] received magic close
    2023-12-08T09:04:30.673521Z] [debug] [pid:49] backend process exited: 0
    2023-12-08T09:04:30.774468Z] [debug] [pid:49] stopping backend process 96
    2023-12-08T09:04:30.774561Z] [debug] [pid:49] done with backend process
    9: EXIT 0
    2023-12-08T09:04:30.785963Z] [info] [pid:46] Isotovideo exit status: 0
    2023-12-08T09:04:30.801592Z] [info] [pid:46] +++ worker notes +++
    2023-12-08T09:04:30.801752Z] [info] [pid:46] End time: 2023-12-08 09:04:30
    2023-12-08T09:04:30.801835Z] [info] [pid:46] Result: done
    2023-12-08T09:04:30.804992Z] [info] [pid:171] Uploading video.ogv
    2023-12-08T09:04:30.825799Z] [info] [pid:171] Uploading autoinst-log.txt
    penQA is licensed [GPL-2.0](https://github.com/os-autoinst/openQA) - Version [4.6.1701437900.fbad594](http://192.168.0.76/changelog)
    
  • worker-log.txt
    2023-12-08T09:04:15.533136Z] [debug] [pid:46] Preparing Mojo::IOLoop::ReadWriteProcess::Session
    2023-12-08T09:04:15.534243Z] [debug] [pid:46] Job settings:
       ARCH=x86_64
       BACKEND=qemu
       DESKTOP=xfce
       DISTRI=demohw
       FLAVOR=generalhw
       GENERAL_HW_CMD_DIR=/var/lib/openqa/share/tests/openeuler/data/generalhw_scripts
       GENERAL_HW_REBOOT_ARGS=192.168.0.76
       GENERAL_HW_REBOOT_CMD=reboot_SUT_via_SSH.sh
       GENERAL_HW_SOL_ARGS=ttyUSB1    #如果需要连接 vnc 没连串口也要配置
       GENERAL_HW_SOL_CMD=get_sol_dev.sh    #如果需要连接 vnc 没连串口也要配置
       GENERAL_HW_VNC_IP=192.168.0.76    #桌面测试时需要配置
       GENERAL_HW_VNC_PASSWORD=[redacted]
       GENERAL_HW_VNC_PORT=5901    #桌面测试时需要配置
       HDDSIZEGB=100
       INSTALL=1
       JOBTOKEN=RB81NKvnfvSxjXbQ
       LOG_LEVEL=debug
       MACHINE=demohw
       NAME=00000055-demohw-v1-generalhw-x86_64-demohw@demohw
       NICTYPE=user
       OPENQA_HOSTNAME=172.18.0.10
       OPENQA_URL=http://172.18.0.10/
       PRJDIR=/var/lib/openqa/share
       QEMUPORT=20012
       RETRY_DELAY=5
       RETRY_DELAY_IF_WEBUI_BUSY=60
       SSH_XTERM_WAIT_SUT_ALIVE_TIMEOUT=60
       SUT_IP=192.168.0.76
       TEST=demohw
       TEST_SUITE_NAME=demohw
       VERSION=v1
       VIDEOMODE=vnc
       VNC=91
       VNC_TYPING_LIMIT=7
       WORKER_CLASS=demohw
       WORKER_HOSTNAME=worker
       WORKER_ID=12
       WORKER_INSTANCE=1
       YAML_SCHEDULE=schedule/demohw.yaml
    2023-12-08T09:04:15.534564Z] [debug] [pid:46] Symlinked from "/var/lib/openqa/share/tests/demohw" to "/var/lib/openqa/pool/1/demohw"
    2023-12-08T09:04:15.534824Z] [info] [pid:46] Preparing cgroup to start isotovideo
    2023-12-08T09:04:15.535341Z] [warn] [pid:46] Disabling cgroup usage because cgroup creation failed: mkdir /sys/fs/cgroup/systemd: Permission denied at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/File.pm line 85.
    2023-12-08T09:04:15.535420Z] [info] [pid:46] You can define a custom slice with OPENQA_CGROUP_SLICE or indicating the base mount with MOJO_CGROUP_FS.
    2023-12-08T09:04:15.535667Z] [info] [pid:46] Starting isotovideo container
    2023-12-08T09:04:15.537317Z] [debug] [pid:46] Registered process:49
    2023-12-08T09:04:15.537432Z] [info] [pid:49] 49: WORKING 55
    2023-12-08T09:04:15.537584Z] [info] [pid:46] isotovideo has been started (PID: 49)
    2023-12-08T09:04:15.537966Z] [debug] [pid:46] Running job 55 from [http://172.18.0.10:](http://172.18.0.10/) 00000055-demohw-v1-generalhw-x86_64-demohw@demohw.
    2023-12-08T09:04:15.539013Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:15.575914Z] [debug] [pid:46] Upload concluded (no current module)
    2023-12-08T09:04:16.121766Z] [debug] [pid:46] Starting livelog
    2023-12-08T09:04:16.122787Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:16.174977Z] [debug] [pid:46] Upload concluded (no current module)
    2023-12-08T09:04:17.176886Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:17.212788Z] [debug] [pid:46] Upload concluded (no current module)
    2023-12-08T09:04:18.214767Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:18.250836Z] [debug] [pid:46] Upload concluded (no current module)
    2023-12-08T09:04:19.254017Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:19.309617Z] [debug] [pid:46] Upload concluded (at boot)
    2023-12-08T09:04:20.312391Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:20.356129Z] [debug] [pid:46] Upload concluded (at boot)
    2023-12-08T09:04:21.358374Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:21.400272Z] [debug] [pid:46] Upload concluded (at boot)
    2023-12-08T09:04:22.402617Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:22.444202Z] [debug] [pid:46] Upload concluded (at boot)
    2023-12-08T09:04:23.446569Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:23.487366Z] [debug] [pid:46] Upload concluded (at boot)
    2023-12-08T09:04:24.489705Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:24.531815Z] [debug] [pid:46] Upload concluded (at boot)
    2023-12-08T09:04:25.534946Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:25.578051Z] [debug] [pid:46] Upload concluded (at boot)
    2023-12-08T09:04:26.581191Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:26.627307Z] [debug] [pid:46] Upload concluded (at boot)
    2023-12-08T09:04:27.628754Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:27.671306Z] [debug] [pid:46] Upload concluded (at boot)
    2023-12-08T09:04:28.674420Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:28.713008Z] [debug] [pid:46] Upload concluded (at boot)
    2023-12-08T09:04:29.715004Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:29.769226Z] [debug] [pid:46] Upload concluded (up to boot)
    2023-12-08T09:04:30.771556Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:30.786213Z] [debug] [pid:46] Stopping job 55 from [http://172.18.0.10:](http://172.18.0.10/) 00000055-demohw-v1-generalhw-x86_64-demohw@demohw - reason: done
    2023-12-08T09:04:30.786665Z] [debug] [pid:46] REST-API call: POST http://172.18.0.10/api/v1/jobs/55/status
    2023-12-08T09:04:30.804862Z] [info] [pid:171] Uploading video.ogv
    2023-12-08T09:04:30.805151Z] [debug] [pid:171] Uploading artefact video.ogv
    2023-12-08T09:04:30.807118Z] [error] [pid:171] REST-API error (POST http://172.18.0.10/api/v1/jobs/55/status): Connection error: Premature connection close (remaining tries: 59)
    2023-12-08T09:04:30.808678Z] [debug] [pid:46] Upload concluded (no current module)
    2023-12-08T09:04:30.825733Z] [info] [pid:171] Uploading autoinst-log.txt
    2023-12-08T09:04:30.825869Z] [debug] [pid:171] Uploading artefact autoinst-log.txt
    2023-12-08T09:04:30.840811Z] [info] [pid:171] Uploading worker-log.txt
    2023-12-08T09:04:30.840945Z] [debug] [pid:171] Uploading artefact worker-log.txt
    
  • serial0.txt: nothing

trdthg avatar Dec 08 '23 09:12 trdthg

I guess the logs of openqa-webui.service and possible the other web UI services would have been more interesting than what's going on on the worker-side.

EDIT: Oh, that screenshot contains web UI logs. It also contains worker logs which where a bit distracting. Unfortunately I couldn't find anything useful in these log excerpts, likely everything that comes after the worker registration would be more useful. It is possible that this is a problem of the docker-compose setup (and it would work when setting things up via plain old RPMs.)

Martchus avatar Dec 08 '23 09:12 Martchus

Thank you for your attention.

everything that comes after the worker registration

I use docker-compose up --build start a new worker, rerun the test, here is the log:

although the livestreaming was canceled prematurely due to the short test run, instead of returning 504

worker-1  | TEST_DISTRI_DEPS: 
worker-1  | Loading repository data...
worker-1  | Reading installed packages...
worker-1  | Resolving package dependencies...
worker-1  | 
worker-1  | The following 60 NEW packages are going to be installed:
worker-1  |   lsof os-autoinst-distri-opensuse-deps perl-Archive-Zip perl-B-Hooks-EndOfScope perl-B-Keywords perl-Class-Inspector perl-Class-Singleton perl-Class-Tiny perl-Clone perl-Code-DRY perl-DateTime perl-DateTime-Locale perl-DateTime-TimeZone perl-Dist-CheckConflicts perl-Eval-Closure perl-Exception-Base perl-File-Path perl-File-ShareDir perl-IO-String perl-JSON perl-Lingua-EN-Inflect perl-List-SomeUtils perl-List-SomeUtils-XS perl-Module-Build perl-Module-Pluggable perl-NetAddr-IP perl-PPI perl-PPIx-QuoteLike perl-PPIx-Regexp perl-PPIx-Utilities perl-Package-Stash perl-Package-Stash-XS perl-Params-ValidationCompiler perl-Path-Tiny perl-Perl-Critic perl-Perl-Critic-Community perl-Perl-Critic-Policy-Variables-ProhibitLoopOnHash perl-Perl-Critic-Pulp perl-Perl-Tidy perl-Pod-MinimumVersion perl-Pod-Spell perl-Readonly perl-Selenium-Remote-Driver perl-SemVer perl-Specio perl-String-Format perl-Sub-Identify perl-Symbol-Util perl-Task-Weaken perl-Test-Assert perl-Test-Fatal perl-Test-LongString perl-Tie-IxHash perl-Variable-Magic perl-XML-SAX-Expat perl-XML-Simple perl-XML-Writer perl-constant-boolean perl-namespace-autoclean perl-namespace-clean
worker-1  | 
worker-1  | 60 new packages to install.
worker-1  | Overall download size: 8.9 MiB. Already cached: 0 B. After the operation, additional 36.2 MiB will be used.
worker-1  | Continue? [y/n/v/...? shows all options] (y): y
worker-1  | Retrieving: perl-Perl-Tidy-20230912.0.0-lp155.2.1.noarch (devel_openQA_Leap) (1/60), 768.9 KiB    
worker-1  | Retrieving: perl-Perl-Tidy-20230912.0.0-lp155.2.1.noarch.rpm [...................................done (497.4 KiB/s)]
worker-1  | Retrieving: lsof-4.91-1.11.x86_64 (Main Repository) (2/60), 330.9 KiB    
worker-1  | Retrieving: lsof-4.91-1.11.x86_64.rpm [....done (89.8 KiB/s)]
worker-1  | Retrieving: perl-Archive-Zip-1.60-3.3.1.noarch (Main Repository) (3/60), 103.4 KiB    
worker-1  | Retrieving: perl-Archive-Zip-1.60-3.3.1.noarch.rpm [..done (4.1 KiB/s)]
worker-1  | Retrieving: perl-B-Keywords-1.24-bp155.1.5.noarch (Main Repository) (4/60),  21.2 KiB    
worker-1  | Retrieving: perl-B-Keywords-1.24-bp155.1.5.noarch.rpm [..done (794 B/s)]
worker-1  | Retrieving: perl-Class-Inspector-1.32-1.22.noarch (Main Repository) (5/60),  32.0 KiB    
worker-1  | Retrieving: perl-Class-Inspector-1.32-1.22.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Class-Singleton-1.5-1.24.noarch (Main Repository) (6/60),  19.3 KiB    
worker-1  | Retrieving: perl-Class-Singleton-1.5-1.24.noarch.rpm [.....done (18.6 KiB/s)]
worker-1  | Retrieving: perl-Class-Tiny-1.006-1.24.noarch (Main Repository) (7/60),  30.8 KiB    
worker-1  | Retrieving: perl-Class-Tiny-1.006-1.24.noarch.rpm [.......done (30.1 KiB/s)]
worker-1  | Retrieving: perl-Clone-0.46-bp155.1.5.x86_64 (Main Repository) (8/60),  21.9 KiB    
worker-1  | Retrieving: perl-Clone-0.46-bp155.1.5.x86_64.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Dist-CheckConflicts-0.11-1.23.noarch (Main Repository) (9/60),  24.8 KiB    
worker-1  | Retrieving: perl-Dist-CheckConflicts-0.11-1.23.noarch.rpm [....done (4.3 KiB/s)]
worker-1  | Retrieving: perl-Exception-Base-0.2501-bp155.2.8.noarch (Main Repository) (10/60),  52.6 KiB    
worker-1  | Retrieving: perl-Exception-Base-0.2501-bp155.2.8.noarch.rpm [...done (4.2 KiB/s)]
worker-1  | Retrieving: perl-File-Path-2.150000-1.22.noarch (Main Repository) (11/60),  39.5 KiB    
worker-1  | Retrieving: perl-File-Path-2.150000-1.22.noarch.rpm [...done]
worker-1  | Retrieving: perl-IO-String-1.08-152.24.x86_64 (Main Repository) (12/60),  17.0 KiB    
worker-1  | Retrieving: perl-IO-String-1.08-152.24.x86_64.rpm [..done (798 B/s)]
worker-1  | Retrieving: perl-JSON-2.97001-1.18.noarch (Main Repository) (13/60),  92.4 KiB    
worker-1  | Retrieving: perl-JSON-2.97001-1.18.noarch.rpm [..done (2.2 KiB/s)]
worker-1  | Retrieving: perl-Lingua-EN-Inflect-1.905-bp155.2.12.noarch (Main Repository) (14/60),  60.6 KiB    
worker-1  | Retrieving: perl-Lingua-EN-Inflect-1.905-bp155.2.12.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-List-SomeUtils-XS-0.56-1.17.x86_64 (Main Repository) (15/60),  41.3 KiB    
worker-1  | Retrieving: perl-List-SomeUtils-XS-0.56-1.17.x86_64.rpm [......done (32.3 KiB/s)]
worker-1  | Retrieving: perl-Module-Build-0.422400-1.24.noarch (Main Repository) (16/60), 259.4 KiB    
worker-1  | Retrieving: perl-Module-Build-0.422400-1.24.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Module-Pluggable-5.2-1.24.noarch (Main Repository) (17/60),  33.6 KiB    
worker-1  | Retrieving: perl-Module-Pluggable-5.2-1.24.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-NetAddr-IP-4.079-1.27.x86_64 (Main Repository) (18/60), 120.4 KiB    
worker-1  | Retrieving: perl-NetAddr-IP-4.079-1.27.x86_64.rpm [....done (3.5 KiB/s)]
worker-1  | Retrieving: perl-Package-Stash-XS-0.28-1.26.x86_64 (Main Repository) (19/60),  30.9 KiB    
worker-1  | Retrieving: perl-Package-Stash-XS-0.28-1.26.x86_64.rpm [...done (251.7 KiB/s)]
worker-1  | Retrieving: perl-Readonly-2.05-1.22.noarch (Main Repository) (20/60),  36.5 KiB    
worker-1  | Retrieving: perl-Readonly-2.05-1.22.noarch.rpm [...done]
worker-1  | Retrieving: perl-SemVer-0.10.0-bp155.1.4.noarch (Main Repository) (21/60),  20.4 KiB    
worker-1  | Retrieving: perl-SemVer-0.10.0-bp155.1.4.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-String-Format-1.18-bp155.2.12.noarch (Main Repository) (22/60),  23.1 KiB    
worker-1  | Retrieving: perl-String-Format-1.18-bp155.2.12.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Sub-Identify-0.14-1.26.x86_64 (Main Repository) (23/60),  20.2 KiB    
worker-1  | Retrieving: perl-Sub-Identify-0.14-1.26.x86_64.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Symbol-Util-0.0203-bp155.2.12.noarch (Main Repository) (24/60),  30.7 KiB    
worker-1  | Retrieving: perl-Symbol-Util-0.0203-bp155.2.12.noarch.rpm [.....done (30.7 KiB/s)]
worker-1  | Retrieving: perl-Task-Weaken-1.06-2.13.noarch (Main Repository) (25/60),  24.0 KiB    
worker-1  | Retrieving: perl-Task-Weaken-1.06-2.13.noarch.rpm [....done (24.0 KiB/s)]
worker-1  | Retrieving: perl-Test-Fatal-0.014-1.23.noarch (Main Repository) (26/60),  24.1 KiB    
worker-1  | Retrieving: perl-Test-Fatal-0.014-1.23.noarch.rpm [...done]
worker-1  | Retrieving: perl-Test-LongString-0.17-bp155.2.12.noarch (Main Repository) (27/60),  17.8 KiB    
worker-1  | Retrieving: perl-Test-LongString-0.17-bp155.2.12.noarch.rpm [..done (788 B/s)]
worker-1  | Retrieving: perl-Tie-IxHash-1.23-1.23.noarch (Main Repository) (28/60),  18.4 KiB    
worker-1  | Retrieving: perl-Tie-IxHash-1.23-1.23.noarch.rpm [..done (799 B/s)]
worker-1  | Retrieving: perl-Variable-Magic-0.62-1.19.x86_64 (Main Repository) (29/60),  62.2 KiB    
worker-1  | Retrieving: perl-Variable-Magic-0.62-1.19.x86_64.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-XML-SAX-Expat-0.51-1.22.noarch (Main Repository) (30/60),  16.1 KiB    
worker-1  | Retrieving: perl-XML-SAX-Expat-0.51-1.22.noarch.rpm [...done (4.1 KiB/s)]
worker-1  | Retrieving: perl-XML-Writer-0.625-1.24.noarch (Main Repository) (31/60),  34.6 KiB    
worker-1  | Retrieving: perl-XML-Writer-0.625-1.24.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Eval-Closure-0.14-1.22.noarch (Main Repository) (32/60),  25.2 KiB    
worker-1  | Retrieving: perl-Eval-Closure-0.14-1.22.noarch.rpm [...done]
worker-1  | Retrieving: perl-File-ShareDir-1.104-1.22.noarch (Main Repository) (33/60),  27.0 KiB    
worker-1  | Retrieving: perl-File-ShareDir-1.104-1.22.noarch.rpm [.....done (548.4 KiB/s)]
worker-1  | Retrieving: perl-Path-Tiny-0.104-1.22.noarch (Main Repository) (34/60),  77.7 KiB    
worker-1  | Retrieving: perl-Path-Tiny-0.104-1.22.noarch.rpm [....done (4.1 KiB/s)]
worker-1  | Retrieving: perl-Pod-MinimumVersion-50-bp155.3.10.noarch (Main Repository) (35/60),  35.0 KiB    
worker-1  | Retrieving: perl-Pod-MinimumVersion-50-bp155.3.10.noarch.rpm [...done]
worker-1  | Retrieving: perl-List-SomeUtils-0.56-1.20.noarch (Main Repository) (36/60),  46.9 KiB    
worker-1  | Retrieving: perl-List-SomeUtils-0.56-1.20.noarch.rpm [...done (1.7 MiB/s)]
worker-1  | Retrieving: perl-Code-DRY-0.10-bp155.2.10.x86_64 (Main Repository) (37/60),  44.0 KiB    
worker-1  | Retrieving: perl-Code-DRY-0.10-bp155.2.10.x86_64.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Package-Stash-0.37-1.23.noarch (Main Repository) (38/60),  35.7 KiB    
worker-1  | Retrieving: perl-Package-Stash-0.37-1.23.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-constant-boolean-0.02-bp155.2.8.noarch (Main Repository) (39/60),  23.6 KiB    
worker-1  | Retrieving: perl-constant-boolean-0.02-bp155.2.8.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-PPI-1.276-bp155.1.4.noarch (Main Repository) (40/60), 349.9 KiB    
worker-1  | Retrieving: perl-PPI-1.276-bp155.1.4.noarch.rpm [....done (3.9 KiB/s)]
worker-1  | Retrieving: perl-B-Hooks-EndOfScope-0.21-1.22.noarch (Main Repository) (41/60),  30.3 KiB    
worker-1  | Retrieving: perl-B-Hooks-EndOfScope-0.21-1.22.noarch.rpm [..done (791 B/s)]
worker-1  | Retrieving: perl-XML-Simple-2.24-1.22.noarch (Main Repository) (42/60),  81.4 KiB    
worker-1  | Retrieving: perl-XML-Simple-2.24-1.22.noarch.rpm [...done]
worker-1  | Retrieving: perl-Specio-0.42-1.17.noarch (Main Repository) (43/60), 160.8 KiB    
worker-1  | Retrieving: perl-Specio-0.42-1.17.noarch.rpm [..done (2.2 KiB/s)]
worker-1  | Retrieving: perl-Params-ValidationCompiler-0.26-1.17.noarch (Main Repository) (44/60),  34.0 KiB    
worker-1  | Retrieving: perl-Params-ValidationCompiler-0.26-1.17.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Pod-Spell-1.25-bp155.1.5.noarch (Main Repository) (45/60),  44.2 KiB    
worker-1  | Retrieving: perl-Pod-Spell-1.25-bp155.1.5.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Test-Assert-0.0504-bp155.2.8.noarch (Main Repository) (46/60),  34.6 KiB    
worker-1  | Retrieving: perl-Test-Assert-0.0504-bp155.2.8.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-PPIx-Utilities-1.001000-bp155.3.9.noarch (Main Repository) (47/60),  36.2 KiB    
worker-1  | Retrieving: perl-PPIx-Utilities-1.001000-bp155.3.9.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-PPIx-Regexp-0.086-bp155.1.5.noarch (Main Repository) (48/60), 329.3 KiB    
worker-1  | Retrieving: perl-PPIx-Regexp-0.086-bp155.1.5.noarch.rpm [..done (792 B/s)]
worker-1  | Retrieving: perl-PPIx-QuoteLike-0.023-bp155.1.5.noarch (Main Repository) (49/60),  86.5 KiB    
worker-1  | Retrieving: perl-PPIx-QuoteLike-0.023-bp155.1.5.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-namespace-clean-0.27-1.22.noarch (Main Repository) (50/60),  24.0 KiB    
worker-1  | Retrieving: perl-namespace-clean-0.27-1.22.noarch.rpm [....done (3.5 KiB/s)]
worker-1  | Retrieving: perl-Perl-Critic-1.148-bp155.1.4.noarch (Main Repository) (51/60), 894.5 KiB    
worker-1  | Retrieving: perl-Perl-Critic-1.148-bp155.1.4.noarch.rpm [....done (5.3 MiB/s)]
worker-1  | Retrieving: perl-namespace-autoclean-0.28-1.22.noarch (Main Repository) (52/60),  27.7 KiB    
worker-1  | Retrieving: perl-namespace-autoclean-0.28-1.22.noarch.rpm [...done (165.2 KiB/s)]
worker-1  | Retrieving: perl-Perl-Critic-Pulp-99-bp155.2.8.x86_64 (Main Repository) (53/60), 238.3 KiB    
worker-1  | Retrieving: perl-Perl-Critic-Pulp-99-bp155.2.8.x86_64.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Perl-Critic-Policy-Variables-ProhibitLoopOnHash-0.008-bp155.2.8.noarch (Main Repository) (54/60),  21.4 KiB    
worker-1  | Retrieving: perl-Perl-Critic-Policy-Variables-ProhibitLoopOnHash-0.008-bp155.2.8.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-DateTime-Locale-1.170000-1.17.noarch (Main Repository) (55/60),   2.5 MiB    
worker-1  | Retrieving: perl-DateTime-Locale-1.170000-1.17.noarch.rpm [............done (2.3 MiB/s)]
worker-1  | Retrieving: perl-Perl-Critic-Community-1.0.3-bp155.1.4.noarch (Main Repository) (56/60), 160.0 KiB    
worker-1  | Retrieving: perl-Perl-Critic-Community-1.0.3-bp155.1.4.noarch.rpm [..done (782 B/s)]
worker-1  | Retrieving: perl-Selenium-Remote-Driver-1.49-bp155.2.3.1.x86_64 (Update repository of openSUSE Backports) (57/60), 816.3 KiB    
worker-1  | Retrieving: perl-Selenium-Remote-Driver-1.49-bp155.2.3.1.x86_64.rpm [.............done (2.9 MiB/s)]
worker-1  | Retrieving: perl-DateTime-TimeZone-2.60-150000.3.3.1.noarch (Update repository with updates from SUSE Linux Enterprise 15) (58/60), 354.3 KiB    
worker-1  | Retrieving: perl-DateTime-TimeZone-2.60-150000.3.3.1.noarch.rpm [......done (2.1 MiB/s)]
worker-1  | Retrieving: perl-DateTime-1.45-1.16.x86_64 (Main Repository) (59/60), 143.0 KiB    
worker-1  | Retrieving: perl-DateTime-1.45-1.16.x86_64.rpm [...done (3.9 KiB/s)]
worker-1  | Retrieving: os-autoinst-distri-opensuse-deps-1.1702033475.f69e77d2-lp155.13340.1.noarch (devel_openQA) (60/60),   8.2 KiB    
worker-1  | Retrieving: os-autoinst-distri-opensuse-deps-1.1702033475.f69e77d2-lp155.13340.1.noarch.rpm [........done (17.0 KiB/s)]
worker-1  | 
worker-1  | Checking for file conflicts: [..done]
worker-1  | ( 1/60) Installing: perl-Perl-Tidy-20230912.0.0-lp155.2.1.noarch [..done]
worker-1  | ( 2/60) Installing: lsof-4.91-1.11.x86_64 [..done]
worker-1  | ( 3/60) Installing: perl-Archive-Zip-1.60-3.3.1.noarch [..done]
worker-1  | ( 4/60) Installing: perl-B-Keywords-1.24-bp155.1.5.noarch [..done]
worker-1  | ( 5/60) Installing: perl-Class-Inspector-1.32-1.22.noarch [..done]
worker-1  | ( 6/60) Installing: perl-Class-Singleton-1.5-1.24.noarch [..done]
worker-1  | ( 7/60) Installing: perl-Class-Tiny-1.006-1.24.noarch [..done]
worker-1  | ( 8/60) Installing: perl-Clone-0.46-bp155.1.5.x86_64 [..done]
worker-1  | ( 9/60) Installing: perl-Dist-CheckConflicts-0.11-1.23.noarch [..done]
worker-1  | (10/60) Installing: perl-Exception-Base-0.2501-bp155.2.8.noarch [..done]
worker-1  | (11/60) Installing: perl-File-Path-2.150000-1.22.noarch [..done]
worker-1  | (12/60) Installing: perl-IO-String-1.08-152.24.x86_64 [..done]
worker-1  | (13/60) Installing: perl-JSON-2.97001-1.18.noarch [..done]
worker-1  | (14/60) Installing: perl-Lingua-EN-Inflect-1.905-bp155.2.12.noarch [..done]
worker-1  | (15/60) Installing: perl-List-SomeUtils-XS-0.56-1.17.x86_64 [..done]
worker-1  | (16/60) Installing: perl-Module-Build-0.422400-1.24.noarch [..done]
worker-1  | (17/60) Installing: perl-Module-Pluggable-5.2-1.24.noarch [..done]
worker-1  | (18/60) Installing: perl-NetAddr-IP-4.079-1.27.x86_64 [..done]
worker-1  | (19/60) Installing: perl-Package-Stash-XS-0.28-1.26.x86_64 [..done]
worker-1  | (20/60) Installing: perl-Readonly-2.05-1.22.noarch [..done]
worker-1  | (21/60) Installing: perl-SemVer-0.10.0-bp155.1.4.noarch [..done]
worker-1  | (22/60) Installing: perl-String-Format-1.18-bp155.2.12.noarch [..done]
worker-1  | (23/60) Installing: perl-Sub-Identify-0.14-1.26.x86_64 [..done]
worker-1  | (24/60) Installing: perl-Symbol-Util-0.0203-bp155.2.12.noarch [..done]
worker-1  | (25/60) Installing: perl-Task-Weaken-1.06-2.13.noarch [..done]
worker-1  | (26/60) Installing: perl-Test-Fatal-0.014-1.23.noarch [..done]
worker-1  | (27/60) Installing: perl-Test-LongString-0.17-bp155.2.12.noarch [..done]
worker-1  | (28/60) Installing: perl-Tie-IxHash-1.23-1.23.noarch [..done]
worker-1  | (29/60) Installing: perl-Variable-Magic-0.62-1.19.x86_64 [..done]
worker-1  | (30/60) Installing: perl-XML-SAX-Expat-0.51-1.22.noarch [..done]
worker-1  | (31/60) Installing: perl-XML-Writer-0.625-1.24.noarch [..done]
worker-1  | (32/60) Installing: perl-Eval-Closure-0.14-1.22.noarch [..done]
worker-1  | (33/60) Installing: perl-File-ShareDir-1.104-1.22.noarch [..done]
worker-1  | (34/60) Installing: perl-Path-Tiny-0.104-1.22.noarch [..done]
worker-1  | (35/60) Installing: perl-Pod-MinimumVersion-50-bp155.3.10.noarch [..done]
worker-1  | (36/60) Installing: perl-List-SomeUtils-0.56-1.20.noarch [..done]
worker-1  | (37/60) Installing: perl-Code-DRY-0.10-bp155.2.10.x86_64 [..done]
worker-1  | (38/60) Installing: perl-Package-Stash-0.37-1.23.noarch [..done]
worker-1  | (39/60) Installing: perl-constant-boolean-0.02-bp155.2.8.noarch [..done]
worker-1  | (40/60) Installing: perl-PPI-1.276-bp155.1.4.noarch [..done]
worker-1  | (41/60) Installing: perl-B-Hooks-EndOfScope-0.21-1.22.noarch [..done]
worker-1  | (42/60) Installing: perl-XML-Simple-2.24-1.22.noarch [..done]
worker-1  | (43/60) Installing: perl-Specio-0.42-1.17.noarch [..done]
worker-1  | (44/60) Installing: perl-Params-ValidationCompiler-0.26-1.17.noarch [..done]
worker-1  | (45/60) Installing: perl-Pod-Spell-1.25-bp155.1.5.noarch [..done]
worker-1  | (46/60) Installing: perl-Test-Assert-0.0504-bp155.2.8.noarch [..done]
worker-1  | (47/60) Installing: perl-PPIx-Utilities-1.001000-bp155.3.9.noarch [..done]
worker-1  | (48/60) Installing: perl-PPIx-Regexp-0.086-bp155.1.5.noarch [..done]
worker-1  | (49/60) Installing: perl-PPIx-QuoteLike-0.023-bp155.1.5.noarch [..done]
worker-1  | (50/60) Installing: perl-namespace-clean-0.27-1.22.noarch [..done]
worker-1  | (51/60) Installing: perl-Perl-Critic-1.148-bp155.1.4.noarch [..done]
worker-1  | (52/60) Installing: perl-namespace-autoclean-0.28-1.22.noarch [..done]
worker-1  | (53/60) Installing: perl-Perl-Critic-Pulp-99-bp155.2.8.x86_64 [..done]
worker-1  | (54/60) Installing: perl-Perl-Critic-Policy-Variables-ProhibitLoopOnHash-0.008-bp155.2.8.noarch [..done]
worker-1  | (55/60) Installing: perl-DateTime-Locale-1.170000-1.17.noarch [.....done]
worker-1  | (56/60) Installing: perl-Perl-Critic-Community-1.0.3-bp155.1.4.noarch [..done]
worker-1  | (57/60) Installing: perl-Selenium-Remote-Driver-1.49-bp155.2.3.1.x86_64 [..done]
worker-1  | (58/60) Installing: perl-DateTime-TimeZone-2.60-150000.3.3.1.noarch [..done]
worker-1  | (59/60) Installing: perl-DateTime-1.45-1.16.x86_64 [..done]
worker-1  | (60/60) Installing: os-autoinst-distri-opensuse-deps-1.1702033475.f69e77d2-lp155.13340.1.noarch [..done]
worker-1  | /run_openqa_worker.sh: line 34:     9 Terminated              qemu-system-x86_64 -S
worker-1  | [info] [pid:180] worker 1:
worker-1  |  - config file:                      /etc/openqa/workers.ini
worker-1  |  - name used to register:            93239f7bc06e
worker-1  |  - worker address (WORKER_HOSTNAME): worker
worker-1  |  - isotovideo version:               40
worker-1  |  - websocket API version:            1
worker-1  |  - web UI hosts:                     http://172.18.0.9
worker-1  |  - class:                            demohw
worker-1  |  - no cleanup:                       no
worker-1  |  - pool directory:                   /var/lib/openqa/pool/1
worker-1  | [info] [pid:180] Project dir for host http://172.18.0.9 is /var/lib/openqa/share
worker-1  | [info] [pid:180] Registering with openQA http://172.18.0.9
worker-1  | [info] [pid:180] Establishing ws connection via ws://172.18.0.9/api/v1/ws/14
worker-1  | [info] [pid:180] Registered and connected via websockets with openQA host http://172.18.0.9 and worker ID 14
worker-1  | [warn] [pid:180] Websocket connection to http://172.18.0.9/api/v1/ws/14 finished by remote side with code 1006, no reason - trying again in 10 seconds
worker-1  | [info] [pid:180] Registering with openQA http://172.18.0.9
worker-1  | [info] [pid:180] Establishing ws connection via ws://172.18.0.9/api/v1/ws/14
worker-1  | [info] [pid:180] Registered and connected via websockets with openQA host http://172.18.0.9 and worker ID 14
worker-1  | [warn] [pid:180] Websocket connection to http://172.18.0.9/api/v1/ws/14 finished by remote side with code 1006, no reason - trying again in 10 seconds
worker-1  | [info] [pid:180] Registering with openQA http://172.18.0.9
worker-1  | [info] [pid:180] Establishing ws connection via ws://172.18.0.9/api/v1/ws/14
worker-1  | [info] [pid:180] Registered and connected via websockets with openQA host http://172.18.0.9 and worker ID 14
worker-1  | [warn] [pid:180] Websocket connection to http://172.18.0.9/api/v1/ws/14 finished by remote side with code 1006, no reason - trying again in 10 seconds
worker-1  | [info] [pid:180] Registering with openQA http://172.18.0.9
worker-1  | [info] [pid:180] Establishing ws connection via ws://172.18.0.9/api/v1/ws/14
worker-1  | [info] [pid:180] Registered and connected via websockets with openQA host http://172.18.0.9 and worker ID 14
worker-1  | [debug] [pid:180] Accepting job 58 from http://172.18.0.9.
worker-1  | [debug] [pid:180] Setting job 58 from http://172.18.0.9 up
worker-1  | [debug] [pid:180] Preparing Mojo::IOLoop::ReadWriteProcess::Session
worker-1  | [info] [pid:180] +++ setup notes +++
worker-1  | [info] [pid:180] Running on 93239f7bc06e:1 (Linux 6.1.62 #1-NixOS SMP PREEMPT_DYNAMIC Wed Nov  8 13:11:05 UTC 2023 x86_64)
worker-1  | [debug] [pid:180] Job settings:
worker-1  |     ARCH=x86_64
worker-1  |     BACKEND=qemu
worker-1  |     DESKTOP=xfce
worker-1  |     DISTRI=demohw
worker-1  |     FLAVOR=generalhw
worker-1  |     GENERAL_HW_CMD_DIR=/var/lib/openqa/share/tests/openeuler/data/generalhw_scripts
worker-1  |     GENERAL_HW_REBOOT_ARGS=192.168.0.76
worker-1  |     GENERAL_HW_REBOOT_CMD=reboot_SUT_via_SSH.sh
worker-1  |     GENERAL_HW_SOL_ARGS=ttyUSB1    #如果需要连接 vnc 没连串口也要配置
worker-1  |     GENERAL_HW_SOL_CMD=get_sol_dev.sh    #如果需要连接 vnc 没连串口也要配置
worker-1  |     GENERAL_HW_VNC_IP=192.168.0.76    #桌面测试时需要配置
worker-1  |     GENERAL_HW_VNC_PASSWORD=[redacted]
worker-1  |     GENERAL_HW_VNC_PORT=5901    #桌面测试时需要配置
worker-1  |     HDDSIZEGB=100
worker-1  |     INSTALL=1
worker-1  |     JOBTOKEN=FaE9jRmaYObvjmKG
worker-1  |     LOG_LEVEL=debug
worker-1  |     MACHINE=demohw
worker-1  |     NAME=00000058-demohw-v1-generalhw-x86_64-demohw@demohw
worker-1  |     NICTYPE=user
worker-1  |     OPENQA_HOSTNAME=172.18.0.9
worker-1  |     OPENQA_URL=http://172.18.0.9
worker-1  |     PRJDIR=/var/lib/openqa/share
worker-1  |     QEMUPORT=20012
worker-1  |     RETRY_DELAY=5
worker-1  |     RETRY_DELAY_IF_WEBUI_BUSY=60
worker-1  |     SSH_XTERM_WAIT_SUT_ALIVE_TIMEOUT=60
worker-1  |     SUT_IP=192.168.0.76
worker-1  |     TEST=demohw
worker-1  |     TEST_SUITE_NAME=demohw
worker-1  |     VERSION=v1
worker-1  |     VIDEOMODE=vnc
worker-1  |     VNC=91
worker-1  |     VNC_TYPING_LIMIT=7
worker-1  |     WORKER_CLASS=demohw
worker-1  |     WORKER_HOSTNAME=worker
worker-1  |     WORKER_ID=14
worker-1  |     WORKER_INSTANCE=1
worker-1  |     YAML_SCHEDULE=schedule/demohw.yaml
worker-1  | [debug] [pid:180] Symlinked from "/var/lib/openqa/share/tests/demohw" to "/var/lib/openqa/pool/1/demohw"
worker-1  | [info] [pid:180] Preparing cgroup to start isotovideo
worker-1  | [warn] [pid:180] Disabling cgroup usage because cgroup creation failed: mkdir /sys/fs/cgroup/systemd: Permission denied at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/File.pm line 85.
worker-1  | [info] [pid:180] You can define a custom slice with OPENQA_CGROUP_SLICE or indicating the base mount with MOJO_CGROUP_FS.
worker-1  | [info] [pid:180] Starting isotovideo container
worker-1  | [debug] [pid:180] Registered process:183
worker-1  | [info] [pid:180] isotovideo has been started (PID: 183)
worker-1  | [info] [pid:183] 183: WORKING 58
worker-1  | [debug] [pid:183] +++ worker notes +++
worker-1  | [debug] [pid:180] Running job 58 from http://172.18.0.9: 00000058-demohw-v1-generalhw-x86_64-demohw@demohw.
worker-1  | [debug] [pid:180] REST-API call: POST http://172.18.0.9/api/v1/jobs/58/status
worker-1  | [debug] [pid:180] Upload concluded (no current module)
worker-1  | [debug] [pid:180] Starting livelog
worker-1  | [debug] [pid:180] REST-API call: POST http://172.18.0.9/api/v1/jobs/58/status
worker-1  | [debug] [pid:180] Upload concluded (no current module)
worker-1  | [debug] [pid:180] REST-API call: POST http://172.18.0.9/api/v1/jobs/58/status
worker-1  | [debug] [pid:180] Upload concluded (at boot)
worker-1  | [debug] [pid:180] REST-API call: POST http://172.18.0.9/api/v1/jobs/58/status
worker-1  | [debug] [pid:180] Upload concluded (at boot)
worker-1  | [debug] [pid:180] REST-API call: POST http://172.18.0.9/api/v1/jobs/58/status
worker-1  | [debug] [pid:180] Upload concluded (at boot)
worker-1  | [debug] [pid:180] REST-API call: POST http://172.18.0.9/api/v1/jobs/58/status
worker-1  | [debug] [pid:180] Upload concluded (up to boot)
worker-1  | [info] [pid:180] Isotovideo exit status: 0
worker-1  | [debug] [pid:180] Stopping job 58 from http://172.18.0.9: 00000058-demohw-v1-generalhw-x86_64-demohw@demohw - reason: done
worker-1  | [debug] [pid:180] REST-API call: POST http://172.18.0.9/api/v1/jobs/58/status
worker-1  | [info] [pid:180] +++ worker notes +++
worker-1  | [info] [pid:180] End time: 2023-12-09 13:50:39
worker-1  | [info] [pid:180] Result: done
worker-1  | [info] [pid:303] Uploading video.ogv
worker-1  | [debug] [pid:303] Uploading artefact video.ogv
worker-1  | [info] [pid:303] Uploading autoinst-log.txt
worker-1  | [debug] [pid:303] Uploading artefact autoinst-log.txt
worker-1  | [info] [pid:303] Uploading worker-log.txt
worker-1  | [debug] [pid:303] Uploading artefact worker-log.txt
worker-1  | [warn] [pid:303] Skipping upload of vars.json because an error occurred when stripping credentials: Wide character in syswrite at /usr/lib/perl5/5.26.1/x86_64-linux-thread-multi/IO/Handle.pm line 483.
worker-1  | 	IO::Handle::syswrite(GLOB(0x562a88816bc8), "{\x{a}   \"ARCH\" : \"x86_64\",\x{a}   \"ASSETDIR\" : \"/var/lib/openqa/shar"...) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/File.pm line 145
worker-1  | 	Mojo::File::spew(Mojo::File=SCALAR(0x562a88848c50), "{\x{a}   \"ARCH\" : \"x86_64\",\x{a}   \"ASSETDIR\" : \"/var/lib/openqa/shar"...) called at /usr/share/openqa/script/../lib/OpenQA/Log.pm line 217
worker-1  | 	OpenQA::Log::redact_settings_in_file("/var/lib/openqa/pool/1/vars.json") called at /usr/share/openqa/script/../lib/OpenQA/Worker/Job.pm line 394
worker-1  | 	eval {...} called at /usr/share/openqa/script/../lib/OpenQA/Worker/Job.pm line 394
worker-1  | 	OpenQA::Worker::Job::_redact_file("/var/lib/openqa/pool/1/vars.json", "vars.json") called at /usr/share/openqa/script/../lib/OpenQA/Worker/Job.pm line 462
worker-1  | 	OpenQA::Worker::Job::__ANON__(Mojo::IOLoop::Subprocess=HASH(0x562a887e0508)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/Subprocess.pm line 53
worker-1  | 	eval {...} called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/Subprocess.pm line 53
worker-1  | 	eval {...} called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/Subprocess.pm line 51
worker-1  | 	Mojo::IOLoop::Subprocess::_start(Mojo::IOLoop::Subprocess=HASH(0x562a887e0508), CODE(0x562a887ef648), CODE(0x562a887ef8b8)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/Subprocess.pm line 21
worker-1  | 	Mojo::IOLoop::Subprocess::__ANON__(Mojo::IOLoop=HASH(0x562a86b841e0)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop.pm line 72
worker-1  | 	Mojo::IOLoop::__ANON__(Mojo::Reactor::Poll=HASH(0x562a853cf518)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/Reactor/Poll.pm line 129
worker-1  | 	Mojo::Reactor::Poll::_next(Mojo::Reactor::Poll=HASH(0x562a853cf518)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/Reactor/Poll.pm line 141
worker-1  | 	eval {...} called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/Reactor/Poll.pm line 141
worker-1  | 	Mojo::Reactor::Poll::_try(Mojo::Reactor::Poll=HASH(0x562a853cf518), "Timer", CODE(0x562a86b7c690)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/Reactor/Poll.pm line 83
worker-1  | 	Mojo::Reactor::Poll::one_tick(Mojo::Reactor::Poll=HASH(0x562a853cf518)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/Reactor/Poll.pm line 101
worker-1  | 	Mojo::Reactor::Poll::start(Mojo::Reactor::Poll=HASH(0x562a853cf518)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop.pm line 134
worker-1  | 	Mojo::IOLoop::start(Mojo::IOLoop=HASH(0x562a86b841e0)) called at /usr/share/openqa/script/../lib/OpenQA/Worker.pm line 364
worker-1  | 	OpenQA::Worker::exec(OpenQA::Worker=HASH(0x562a886fce70)) called at /usr/share/openqa/script/worker line 125
worker-1  | 
worker-1  | [info] [pid:303] Uploading serial0.txt
worker-1  | [debug] [pid:303] Uploading artefact serial0.txt
worker-1  | [info] [pid:303] Uploading video_time.vtt
worker-1  | [debug] [pid:303] Uploading artefact video_time.vtt
worker-1  | [info] [pid:303] Uploading serial_terminal.txt
worker-1  | [debug] [pid:303] Uploading artefact serial_terminal.txt
worker-1  | [info] [pid:303] Uploading serial_terminal_user.txt
worker-1  | [debug] [pid:303] Uploading artefact serial_terminal_user.txt
worker-1  | [debug] [pid:180] Considering job 58 as regularly done
worker-1  | [debug] [pid:180] REST-API call: POST http://172.18.0.9/api/v1/jobs/58/status
worker-1  | [debug] [pid:180] Upload concluded (no current module)
worker-1  | [debug] [pid:180] REST-API call: POST http://172.18.0.9/api/v1/jobs/58/set_done?worker_id=14
worker-1  | [debug] [pid:180] Job 58 from http://172.18.0.9 finished - reason: done
worker-1  | [debug] [pid:180] Cleaning up for next job

Other stuff I've done besides:

  1. delete replicas

    deploy: 
    	replicas: $XXX
    

    parts in both webui and worker docker-compose.yaml,

  2. use webui_default network in worker

    networks:
      default:
        name: webui_default
        driver: bridge
        external: true
    
  3. set in HOST to http://172.18.0.9 in worker.ini and client.conf

    It is the ip of webui-nginx-1, I get this from docker inspect

  4. set GENERAL_HW_CMD_DIR, ... in worker.ini, I was trying to test another running desktop, they should have no effect, since I've switched the backend to qemu.

btw, is generalhw the backend I need if i just want to test using ssh(for server), vnc(for server and desktop)?

trdthg avatar Dec 09 '23 14:12 trdthg

everything that comes after the worker registration

I use docker-compose up --build start a new worker, rerun the test, here is the log:

I still meant on the web UI side.

Maybe I'll find the time next week to revisit our docker compose setup and possibly reproduce the issue. I would recommend to setup openQA using plain old RPMs in the meantime.


btw, is generalhw the backend I need if i just want to test using ssh(for server), vnc(for server and desktop)?

I'm not sure what you mean by that. If you really don't want to test using a VM but using real hardware instead then generalhw might be the right backend, indeed (see https://github.com/os-autoinst/os-autoinst/blob/master/doc/backend_vars.asciidoc for relevant variables and e.g. https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/data/generalhw_scripts/raspberry_pi_hardware_testing_setup.md for and example using a Raspberry Pi).

Martchus avatar Dec 09 '23 20:12 Martchus

@trdthg maybe not the original problem but also suspicious are the entries in the logs with the non-ASCII characters and the according warning, see

worker-1  |     GENERAL_HW_SOL_ARGS=ttyUSB1    #������ vnc 没�串����置
worker-1  |     GENERAL_HW_SOL_CMD=get_sol_dev.sh    #������ vnc 没�串����置
worker-1  |     GENERAL_HW_VNC_IP=192.168.0.76    #��������置
worker-1  |     GENERAL_HW_VNC_PASSWORD=[redacted]
worker-1  |     GENERAL_HW_VNC_PORT=5901    #��������置
...
worker-1  | [warn] [pid:303] Skipping upload of vars.json because an error occurred when stripping credentials: Wide character in syswrite at /usr/lib/perl5/5.26.1/x86_64-linux-thread-multi/IO/Handle.pm line 483.

Could it be that your VNC password has non-ASCII characters included?

okurz avatar Dec 11 '23 07:12 okurz

Could it be that your VNC password has non-ASCII characters included?

U_U Maybe not, my vnc password is just 123456

EDIT: and vnc connection is ok

trdthg avatar Dec 11 '23 09:12 trdthg

Then I don't know where those characters come from but that's not how it normally looks :)

okurz avatar Dec 11 '23 10:12 okurz

just some comment in Chinese, I wrote those myself, (just checked that it's legal to use # or ; as a comment in .ini). I will delete them and try again later.

trdthg avatar Dec 11 '23 12:12 trdthg

I will delete them and try again later.

Didn't see anything new.

webui log

websockets-1  | [info] Listening at "http://0.0.0.0:9527"
scheduler-1   | [info] Listening at "http://0.0.0.0:9529"
nginx-1       | NGINX configuration:
websockets-1  | Web application available at http://0.0.0.0:9527
nginx-1       | events {
nginx-1       | }
websockets-1  | [info] Worker 15 websocket connection closed - 1006
websockets-1  | [info] Worker 15 websocket connection closed - 1006
nginx-1       | 
websockets-1  | [info] Worker 15 websocket connection closed - 1006
nginx-1       | http {
nginx-1       |     upstream webui {
nginx-1       |         server webui:9526;
nginx-1       |     }
nginx-1       | 
nginx-1       |     map $http_upgrade $connection_upgrade {
websockets-1  | [info] Worker 15 websocket connection closed - 1006
webui-1       | Waiting for DB creation
nginx-1       |         default upgrade;
websockets-1  | [info] Worker 15 websocket connection closed - 1006
nginx-1       |         '' close;
webui-1       | [info] Listening at "http://0.0.0.0:9526"
websockets-1  | [info] Worker 15 websocket connection closed - 1006
db-1          | 
websockets-1  | [info] Worker 15 websocket connection closed - 1006
nginx-1       |     }
webui-1       | Web application available at http://0.0.0.0:9526
webui-1       | [info] Manager 19 started
livehandler-1  | Waiting for DB creation
db-1           | PostgreSQL Database directory appears to contain a database; Skipping initialization
webui_db_init-1  | Waiting for DB creation
nginx-1          | 
gru-1            | Waiting for DB creation
livehandler-1    | [info] Listening at "http://0.0.0.0:9528"
nginx-1          |     server {
webui_db_init-1  | psql: error: connection to server at "db" (172.18.0.2), port 5432 failed: Connection refused
webui-1          | [info] Worker 21 started
gru-1            | [info] Resetting all leftover Gru locks after restart
db-1             | 
webui_db_init-1  | 	Is the server running on that host and accepting TCP/IP connections?
nginx-1          |         listen       80;
webui-1          | [info] Worker 22 started
webui-1          | [info] Worker 23 started
nginx-1          |         listen       443 ssl;
webui_db_init-1  | [info] Listening at "http://0.0.0.0:9526"
webui_db_init-1  | Web application available at http://0.0.0.0:9526
nginx-1          |         listen       9526;
webui-1          | [info] Worker 24 started
webui-1          | [info] Worker 25 started
nginx-1          |         server_name  localhost;
webui_db_init-1  | [info] Manager 26 started
webui-1          | [info] Worker 26 started
nginx-1          |         client_max_body_size 0;
webui_db_init-1  | [info] Worker 28 started
webui-1          | [info] Worker 27 started
webui_db_init-1  | [info] Worker 29 started
nginx-1          | 
webui-1          | [info] Worker 28 started
nginx-1          |         ssl_certificate     /etc/ssl/certs/openqa.crt;
webui_db_init-1  | [info] Worker 30 started
webui-1          | [info] Worker 29 started
nginx-1          |         ssl_certificate_key /etc/ssl/certs/openqa.key;
webui_db_init-1  | [info] Worker 31 started
webui-1          | [info] Worker 30 started
nginx-1          |         ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
webui_db_init-1  | [info] Worker 32 started
db-1             | 2023-12-11 12:15:40.094 UTC [1] LOG:  starting PostgreSQL 15.1 (Debian 15.1-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
gru-1            | [info] Worker 21 started
webui-1          | [info] Worker 31 started
nginx-1          |         ssl_ciphers         HIGH:!aNULL:!MD5;
webui_db_init-1  | [info] Worker 33 started
db-1             | 2023-12-11 12:15:40.095 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
nginx-1          | 
webui-1          | [info] Worker 32 started
webui_db_init-1  | [info] Worker 34 started
db-1             | 2023-12-11 12:15:40.095 UTC [1] LOG:  listening on IPv6 address "::", port 5432
nginx-1          |         location /api/v1/ws/ {
webui-1          | [info] Worker 33 started
webui_db_init-1  | [info] Worker 35 started
db-1             | 2023-12-11 12:15:40.100 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
webui-1          | [info] Worker 34 started
webui_db_init-1  | [info] Worker 36 started
db-1             | 2023-12-11 12:15:40.109 UTC [28] LOG:  database system was shut down at 2023-12-11 12:15:36 UTC
webui_db_init-1  | [info] Worker 37 started
webui-1          | [info] Worker 35 started
webui-1          | [info] Worker 36 started
db-1             | 2023-12-11 12:15:40.121 UTC [1] LOG:  database system is ready to accept connections
webui_db_init-1  | [info] Worker 38 started
nginx-1          |             proxy_pass http://websockets:9527;
webui_db_init-1  | [info] Worker 39 started
webui-1          | [info] Worker 37 started
nginx-1          |             proxy_http_version 1.1;
webui_db_init-1  | [info] Worker 40 started
webui-1          | [info] Worker 38 started
nginx-1          |             proxy_set_header Upgrade $http_upgrade;
webui_db_init-1  | [info] Worker 41 started
webui-1          | [info] Worker 39 started
nginx-1          |             proxy_set_header Connection $connection_upgrade;
webui_db_init-1  | [info] Worker 42 started
webui-1          | [info] Worker 40 started
nginx-1          |             proxy_set_header Host $host;
webui_db_init-1  | [info] Worker 43 started
webui-1          | [info] Worker 41 started
nginx-1          |         }
webui_db_init-1  | [info] Worker 44 started
db-1             | 2023-12-11 12:17:54.015 UTC [1] LOG:  received fast shutdown request
nginx-1          | 
webui-1          | [info] Worker 42 started
webui_db_init-1  | [info] Worker 45 started
webui-1          | [info] Worker 43 started
nginx-1          |         location /liveviewhandler/ {
db-1             | 2023-12-11 12:17:54.018 UTC [1] LOG:  aborting any active transactions
webui_db_init-1  | [info] Worker 46 started
webui-1          | [info] Worker 44 started
nginx-1          |             proxy_pass http://livehandler:9528;
db-1             | 2023-12-11 12:17:54.019 UTC [1] LOG:  background worker "logical replication launcher" (PID 31) exited with exit code 1
webui_db_init-1  | [info] Worker 47 started
webui-1          | [info] Worker 45 started
nginx-1          |             proxy_http_version 1.1;
webui_db_init-1  | [info] Worker 48 started
db-1             | 2023-12-11 12:17:54.019 UTC [26] LOG:  shutting down
webui-1          | [info] Worker 46 started
nginx-1          |             proxy_set_header Upgrade $http_upgrade;
webui_db_init-1  | [info] Worker 49 started
webui-1          | [info] Worker 47 started
db-1             | 2023-12-11 12:17:54.022 UTC [26] LOG:  checkpoint starting: shutdown immediate
nginx-1          |             proxy_set_header Connection $connection_upgrade;
webui_db_init-1  | [info] Worker 50 started
webui-1          | [info] Worker 48 started
db-1             | 2023-12-11 12:17:54.064 UTC [26] LOG:  checkpoint complete: wrote 43 buffers (0.3%); 0 WAL file(s) added, 0 removed, 1 recycled; write=0.010 s, sync=0.009 s, total=0.045 s; sync files=28, longest=0.006 s, average=0.001 s; distance=158 kB, estimate=158 kB
nginx-1          |             proxy_set_header Host $host;
webui_db_init-1  | [info] Worker 51 started
webui-1          | [info] Worker 49 started
db-1             | 2023-12-11 12:17:54.067 UTC [1] LOG:  database system is shut down
nginx-1          |         }
webui_db_init-1  | [info] Worker 52 started
webui-1          | [info] Creating process id file "/tmp/prefork.pid"
db-1             | 
nginx-1          | 
webui_db_init-1  | [info] Worker 53 started
webui-1          | [info] Worker 50 started
db-1             | PostgreSQL Database directory appears to contain a database; Skipping initialization
nginx-1          |         location /assets/ {
webui_db_init-1  | [info] Worker 54 started
nginx-1          |             root /usr/share/openqa/assets;
db-1             | 
webui_db_init-1  | [info] Worker 55 started
db-1             | 2023-12-11 12:27:10.601 UTC [1] LOG:  starting PostgreSQL 15.1 (Debian 15.1-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
nginx-1          |         }
webui_db_init-1  | [info] Worker 56 started
nginx-1          | 
db-1             | 2023-12-11 12:27:10.602 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
webui_db_init-1  | [info] Worker 57 started
nginx-1          |         location / {
db-1             | 2023-12-11 12:27:10.602 UTC [1] LOG:  listening on IPv6 address "::", port 5432
webui_db_init-1  | [info] Creating process id file "/tmp/prefork.pid"
nginx-1          |             proxy_set_header X-Forwarded-Host $host:$server_port;
db-1             | 2023-12-11 12:27:10.607 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
nginx-1          |             proxy_set_header X-Forwarded-Server $host;
db-1             | 2023-12-11 12:27:10.615 UTC [28] LOG:  database system was shut down at 2023-12-11 12:17:54 UTC
nginx-1          |             proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
db-1             | 2023-12-11 12:27:10.628 UTC [1] LOG:  database system is ready to accept connections
nginx-1          |             proxy_pass http://webui;
nginx-1          |         }
nginx-1          |     }
nginx-1          | }
db-1             | 2023-12-11 12:32:10.664 UTC [26] LOG:  checkpoint starting: time
db-1             | 2023-12-11 12:32:15.030 UTC [26] LOG:  checkpoint complete: wrote 45 buffers (0.3%); 0 WAL file(s) added, 0 removed, 0 recycled; write=4.330 s, sync=0.017 s, total=4.367 s; sync files=17, longest=0.008 s, average=0.001 s; distance=169 kB, estimate=169 kB
db-1             | 2023-12-11 12:37:10.127 UTC [26] LOG:  checkpoint starting: time
db-1             | 2023-12-11 12:37:14.585 UTC [26] LOG:  checkpoint complete: wrote 45 buffers (0.3%); 0 WAL file(s) added, 0 removed, 0 recycled; write=4.423 s, sync=0.014 s, total=4.458 s; sync files=21, longest=0.008 s, average=0.001 s; distance=139 kB, estimate=166 kB
db-1             | 2023-12-11 12:42:10.663 UTC [26] LOG:  checkpoint starting: time
db-1             | 2023-12-11 12:42:20.849 UTC [26] LOG:  checkpoint complete: wrote 102 buffers (0.6%); 0 WAL file(s) added, 0 removed, 0 recycled; write=10.148 s, sync=0.017 s, total=10.187 s; sync files=75, longest=0.009 s, average=0.001 s; distance=283 kB, estimate=283 kB

worker log

worker-1  | Retrieving repository 'devel_openQA' metadata [.............done]
worker-1  | Building repository 'devel_openQA' cache [...done]
worker-1  | Retrieving repository 'Update repository of openSUSE Backports' metadata [...............done]
worker-1  | Building repository 'Update repository of openSUSE Backports' cache [....done]
worker-1  | Retrieving repository 'Update repository with updates from SUSE Linux Enterprise 15' metadata [...................................................................................................................................................................................................................done]
worker-1  | Building repository 'Update repository with updates from SUSE Linux Enterprise 15' cache [....done]
worker-1  | Loading repository data...
worker-1  | Reading installed packages...
worker-1  | Resolving package dependencies...
worker-1  | 
worker-1  | The following 60 NEW packages are going to be installed:
worker-1  |   lsof os-autoinst-distri-opensuse-deps perl-Archive-Zip perl-B-Hooks-EndOfScope perl-B-Keywords perl-Class-Inspector perl-Class-Singleton perl-Class-Tiny perl-Clone perl-Code-DRY perl-DateTime perl-DateTime-Locale perl-DateTime-TimeZone perl-Dist-CheckConflicts perl-Eval-Closure perl-Exception-Base perl-File-Path perl-File-ShareDir perl-IO-String perl-JSON perl-Lingua-EN-Inflect perl-List-SomeUtils perl-List-SomeUtils-XS perl-Module-Build perl-Module-Pluggable perl-NetAddr-IP perl-PPI perl-PPIx-QuoteLike perl-PPIx-Regexp perl-PPIx-Utilities perl-Package-Stash perl-Package-Stash-XS perl-Params-ValidationCompiler perl-Path-Tiny perl-Perl-Critic perl-Perl-Critic-Community perl-Perl-Critic-Policy-Variables-ProhibitLoopOnHash perl-Perl-Critic-Pulp perl-Perl-Tidy perl-Pod-MinimumVersion perl-Pod-Spell perl-Readonly perl-Selenium-Remote-Driver perl-SemVer perl-Specio perl-String-Format perl-Sub-Identify perl-Symbol-Util perl-Task-Weaken perl-Test-Assert perl-Test-Fatal perl-Test-LongString perl-Tie-IxHash perl-Variable-Magic perl-XML-SAX-Expat perl-XML-Simple perl-XML-Writer perl-constant-boolean perl-namespace-autoclean perl-namespace-clean
worker-1  | 
worker-1  | 60 new packages to install.
worker-1  | Overall download size: 8.9 MiB. Already cached: 0 B. After the operation, additional 36.2 MiB will be used.
worker-1  | Continue? [y/n/v/...? shows all options] (y): y
worker-1  | Retrieving: perl-Perl-Tidy-20230912.0.0-lp155.2.1.noarch (devel_openQA_Leap) (1/60), 768.9 KiB    
worker-1  | Retrieving: perl-Perl-Tidy-20230912.0.0-lp155.2.1.noarch.rpm [......................done (473.6 KiB/s)]
worker-1  | Retrieving: lsof-4.91-1.11.x86_64 (Main Repository) (2/60), 330.9 KiB    
worker-1  | Retrieving: lsof-4.91-1.11.x86_64.rpm [.....done (2.8 MiB/s)]
worker-1  | Retrieving: perl-Archive-Zip-1.60-3.3.1.noarch (Main Repository) (3/60), 103.4 KiB    
worker-1  | Retrieving: perl-Archive-Zip-1.60-3.3.1.noarch.rpm [..done (2.2 KiB/s)]
worker-1  | Retrieving: perl-B-Keywords-1.24-bp155.1.5.noarch (Main Repository) (4/60),  21.2 KiB    
worker-1  | Retrieving: perl-B-Keywords-1.24-bp155.1.5.noarch.rpm [..done (3.9 KiB/s)]
worker-1  | Retrieving: perl-Class-Inspector-1.32-1.22.noarch (Main Repository) (5/60),  32.0 KiB    
worker-1  | Retrieving: perl-Class-Inspector-1.32-1.22.noarch.rpm [..done (2.7 KiB/s)]
worker-1  | Retrieving: perl-Class-Singleton-1.5-1.24.noarch (Main Repository) (6/60),  19.3 KiB    
worker-1  | Retrieving: perl-Class-Singleton-1.5-1.24.noarch.rpm [..done (2.2 KiB/s)]
worker-1  | Retrieving: perl-Class-Tiny-1.006-1.24.noarch (Main Repository) (7/60),  30.8 KiB    
worker-1  | Retrieving: perl-Class-Tiny-1.006-1.24.noarch.rpm [..done (2.2 KiB/s)]
worker-1  | Retrieving: perl-Clone-0.46-bp155.1.5.x86_64 (Main Repository) (8/60),  21.9 KiB    
worker-1  | Retrieving: perl-Clone-0.46-bp155.1.5.x86_64.rpm [..done (2.2 KiB/s)]
worker-1  | Retrieving: perl-Dist-CheckConflicts-0.11-1.23.noarch (Main Repository) (9/60),  24.8 KiB    
worker-1  | Retrieving: perl-Dist-CheckConflicts-0.11-1.23.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Exception-Base-0.2501-bp155.2.8.noarch (Main Repository) (10/60),  52.6 KiB    
worker-1  | Retrieving: perl-Exception-Base-0.2501-bp155.2.8.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-File-Path-2.150000-1.22.noarch (Main Repository) (11/60),  39.5 KiB    
worker-1  | Retrieving: perl-File-Path-2.150000-1.22.noarch.rpm [...done (4.0 KiB/s)]
worker-1  | Retrieving: perl-IO-String-1.08-152.24.x86_64 (Main Repository) (12/60),  17.0 KiB    
worker-1  | Retrieving: perl-IO-String-1.08-152.24.x86_64.rpm [....done (3.8 KiB/s)]
worker-1  | Retrieving: perl-JSON-2.97001-1.18.noarch (Main Repository) (13/60),  92.4 KiB    
worker-1  | Retrieving: perl-JSON-2.97001-1.18.noarch.rpm [...done]
worker-1  | Retrieving: perl-Lingua-EN-Inflect-1.905-bp155.2.12.noarch (Main Repository) (14/60),  60.6 KiB    
worker-1  | Retrieving: perl-Lingua-EN-Inflect-1.905-bp155.2.12.noarch.rpm [...done (4.1 KiB/s)]
worker-1  | Retrieving: perl-List-SomeUtils-XS-0.56-1.17.x86_64 (Main Repository) (15/60),  41.3 KiB    
worker-1  | Retrieving: perl-List-SomeUtils-XS-0.56-1.17.x86_64.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Module-Build-0.422400-1.24.noarch (Main Repository) (16/60), 259.4 KiB    
worker-1  | Retrieving: perl-Module-Build-0.422400-1.24.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Module-Pluggable-5.2-1.24.noarch (Main Repository) (17/60),  33.6 KiB    
worker-1  | Retrieving: perl-Module-Pluggable-5.2-1.24.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-NetAddr-IP-4.079-1.27.x86_64 (Main Repository) (18/60), 120.4 KiB    
worker-1  | Retrieving: perl-NetAddr-IP-4.079-1.27.x86_64.rpm [...done (1.4 MiB/s)]
worker-1  | Retrieving: perl-Package-Stash-XS-0.28-1.26.x86_64 (Main Repository) (19/60),  30.9 KiB    
worker-1  | Retrieving: perl-Package-Stash-XS-0.28-1.26.x86_64.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Readonly-2.05-1.22.noarch (Main Repository) (20/60),  36.5 KiB    
worker-1  | Retrieving: perl-Readonly-2.05-1.22.noarch.rpm [.....done (8.7 KiB/s)]
worker-1  | Retrieving: perl-SemVer-0.10.0-bp155.1.4.noarch (Main Repository) (21/60),  20.4 KiB    
worker-1  | Retrieving: perl-SemVer-0.10.0-bp155.1.4.noarch.rpm [...done]
worker-1  | Retrieving: perl-String-Format-1.18-bp155.2.12.noarch (Main Repository) (22/60),  23.1 KiB    
worker-1  | Retrieving: perl-String-Format-1.18-bp155.2.12.noarch.rpm [...done (4.0 KiB/s)]
worker-1  | Retrieving: perl-Sub-Identify-0.14-1.26.x86_64 (Main Repository) (23/60),  20.2 KiB    
worker-1  | Retrieving: perl-Sub-Identify-0.14-1.26.x86_64.rpm [..done (2.2 KiB/s)]
worker-1  | Retrieving: perl-Symbol-Util-0.0203-bp155.2.12.noarch (Main Repository) (24/60),  30.7 KiB    
worker-1  | Retrieving: perl-Symbol-Util-0.0203-bp155.2.12.noarch.rpm [...done]
worker-1  | Retrieving: perl-Task-Weaken-1.06-2.13.noarch (Main Repository) (25/60),  24.0 KiB    
worker-1  | Retrieving: perl-Task-Weaken-1.06-2.13.noarch.rpm [.....done (2.0 KiB/s)]
worker-1  | Retrieving: perl-Test-Fatal-0.014-1.23.noarch (Main Repository) (26/60),  24.1 KiB    
worker-1  | Retrieving: perl-Test-Fatal-0.014-1.23.noarch.rpm [...done (3.9 KiB/s)]
worker-1  | Retrieving: perl-Test-LongString-0.17-bp155.2.12.noarch (Main Repository) (27/60),  17.8 KiB    
worker-1  | Retrieving: perl-Test-LongString-0.17-bp155.2.12.noarch.rpm [...done (4.0 KiB/s)]
worker-1  | Retrieving: perl-Tie-IxHash-1.23-1.23.noarch (Main Repository) (28/60),  18.4 KiB    
worker-1  | Retrieving: perl-Tie-IxHash-1.23-1.23.noarch.rpm [..done (2.2 KiB/s)]
worker-1  | Retrieving: perl-Variable-Magic-0.62-1.19.x86_64 (Main Repository) (29/60),  62.2 KiB    
worker-1  | Retrieving: perl-Variable-Magic-0.62-1.19.x86_64.rpm [...done]
worker-1  | Retrieving: perl-XML-SAX-Expat-0.51-1.22.noarch (Main Repository) (30/60),  16.1 KiB    
worker-1  | Retrieving: perl-XML-SAX-Expat-0.51-1.22.noarch.rpm [...done (270.4 KiB/s)]
worker-1  | Retrieving: perl-XML-Writer-0.625-1.24.noarch (Main Repository) (31/60),  34.6 KiB    
worker-1  | Retrieving: perl-XML-Writer-0.625-1.24.noarch.rpm [..done (2.2 KiB/s)]
worker-1  | Retrieving: perl-Eval-Closure-0.14-1.22.noarch (Main Repository) (32/60),  25.2 KiB    
worker-1  | Retrieving: perl-Eval-Closure-0.14-1.22.noarch.rpm [....done (4.0 KiB/s)]
worker-1  | Retrieving: perl-File-ShareDir-1.104-1.22.noarch (Main Repository) (33/60),  27.0 KiB    
worker-1  | Retrieving: perl-File-ShareDir-1.104-1.22.noarch.rpm [...done (3.5 KiB/s)]
worker-1  | Retrieving: perl-Path-Tiny-0.104-1.22.noarch (Main Repository) (34/60),  77.7 KiB    
worker-1  | Retrieving: perl-Path-Tiny-0.104-1.22.noarch.rpm [.....done]
worker-1  | Retrieving: perl-Pod-MinimumVersion-50-bp155.3.10.noarch (Main Repository) (35/60),  35.0 KiB    
worker-1  | Retrieving: perl-Pod-MinimumVersion-50-bp155.3.10.noarch.rpm [...done]
worker-1  | Retrieving: perl-List-SomeUtils-0.56-1.20.noarch (Main Repository) (36/60),  46.9 KiB    
worker-1  | Retrieving: perl-List-SomeUtils-0.56-1.20.noarch.rpm [...done]
worker-1  | Retrieving: perl-Code-DRY-0.10-bp155.2.10.x86_64 (Main Repository) (37/60),  44.0 KiB    
worker-1  | Retrieving: perl-Code-DRY-0.10-bp155.2.10.x86_64.rpm [..done (2.2 KiB/s)]
worker-1  | Retrieving: perl-Package-Stash-0.37-1.23.noarch (Main Repository) (38/60),  35.7 KiB    
worker-1  | Retrieving: perl-Package-Stash-0.37-1.23.noarch.rpm [..done]
worker-1  | Retrieving: perl-constant-boolean-0.02-bp155.2.8.noarch (Main Repository) (39/60),  23.6 KiB    
worker-1  | Retrieving: perl-constant-boolean-0.02-bp155.2.8.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-PPI-1.276-bp155.1.4.noarch (Main Repository) (40/60), 349.9 KiB    
worker-1  | Retrieving: perl-PPI-1.276-bp155.1.4.noarch.rpm [...done (3.3 MiB/s)]
worker-1  | Retrieving: perl-B-Hooks-EndOfScope-0.21-1.22.noarch (Main Repository) (41/60),  30.3 KiB    
worker-1  | Retrieving: perl-B-Hooks-EndOfScope-0.21-1.22.noarch.rpm [.....done]
worker-1  | Retrieving: perl-XML-Simple-2.24-1.22.noarch (Main Repository) (42/60),  81.4 KiB    
worker-1  | Retrieving: perl-XML-Simple-2.24-1.22.noarch.rpm [......done (799.8 KiB/s)]
worker-1  | Retrieving: perl-Specio-0.42-1.17.noarch (Main Repository) (43/60), 160.8 KiB    
worker-1  | Retrieving: perl-Specio-0.42-1.17.noarch.rpm [....done (3.8 KiB/s)]
worker-1  | Retrieving: perl-Params-ValidationCompiler-0.26-1.17.noarch (Main Repository) (44/60),  34.0 KiB    
worker-1  | Retrieving: perl-Params-ValidationCompiler-0.26-1.17.noarch.rpm [...done]
worker-1  | Retrieving: perl-Pod-Spell-1.25-bp155.1.5.noarch (Main Repository) (45/60),  44.2 KiB    
worker-1  | Retrieving: perl-Pod-Spell-1.25-bp155.1.5.noarch.rpm [....done (3.9 KiB/s)]
worker-1  | Retrieving: perl-Test-Assert-0.0504-bp155.2.8.noarch (Main Repository) (46/60),  34.6 KiB    
worker-1  | Retrieving: perl-Test-Assert-0.0504-bp155.2.8.noarch.rpm [.....done]
worker-1  | Retrieving: perl-PPIx-Utilities-1.001000-bp155.3.9.noarch (Main Repository) (47/60),  36.2 KiB    
worker-1  | Retrieving: perl-PPIx-Utilities-1.001000-bp155.3.9.noarch.rpm [...done]
worker-1  | Retrieving: perl-PPIx-Regexp-0.086-bp155.1.5.noarch (Main Repository) (48/60), 329.3 KiB    
worker-1  | Retrieving: perl-PPIx-Regexp-0.086-bp155.1.5.noarch.rpm [....done (3.1 MiB/s)]
worker-1  | Retrieving: perl-PPIx-QuoteLike-0.023-bp155.1.5.noarch (Main Repository) (49/60),  86.5 KiB    
worker-1  | Retrieving: perl-PPIx-QuoteLike-0.023-bp155.1.5.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-namespace-clean-0.27-1.22.noarch (Main Repository) (50/60),  24.0 KiB    
worker-1  | Retrieving: perl-namespace-clean-0.27-1.22.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Perl-Critic-1.148-bp155.1.4.noarch (Main Repository) (51/60), 894.5 KiB    
worker-1  | Retrieving: perl-Perl-Critic-1.148-bp155.1.4.noarch.rpm [.....done (2.0 MiB/s)]
worker-1  | Retrieving: perl-namespace-autoclean-0.28-1.22.noarch (Main Repository) (52/60),  27.7 KiB    
worker-1  | Retrieving: perl-namespace-autoclean-0.28-1.22.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Perl-Critic-Pulp-99-bp155.2.8.x86_64 (Main Repository) (53/60), 238.3 KiB    
worker-1  | Retrieving: perl-Perl-Critic-Pulp-99-bp155.2.8.x86_64.rpm [....done]
worker-1  | Retrieving: perl-Perl-Critic-Policy-Variables-ProhibitLoopOnHash-0.008-bp155.2.8.noarch (Main Repository) (54/60),  21.4 KiB    
worker-1  | Retrieving: perl-Perl-Critic-Policy-Variables-ProhibitLoopOnHash-0.008-bp155.2.8.noarch.rpm [...done]
worker-1  | Retrieving: perl-DateTime-Locale-1.170000-1.17.noarch (Main Repository) (55/60),   2.5 MiB    
worker-1  | Retrieving: perl-DateTime-Locale-1.170000-1.17.noarch.rpm [.........done (4.2 MiB/s)]
worker-1  | Retrieving: perl-Perl-Critic-Community-1.0.3-bp155.1.4.noarch (Main Repository) (56/60), 160.0 KiB    
worker-1  | Retrieving: perl-Perl-Critic-Community-1.0.3-bp155.1.4.noarch.rpm [..done (2.1 KiB/s)]
worker-1  | Retrieving: perl-Selenium-Remote-Driver-1.49-bp155.2.3.1.x86_64 (Update repository of openSUSE Backports) (57/60), 816.3 KiB    
worker-1  | Retrieving: perl-Selenium-Remote-Driver-1.49-bp155.2.3.1.x86_64.rpm [........done (3.4 MiB/s)]
worker-1  | Retrieving: perl-DateTime-TimeZone-2.60-150000.3.3.1.noarch (Update repository with updates from SUSE Linux Enterprise 15) (58/60), 354.3 KiB    
worker-1  | Retrieving: perl-DateTime-TimeZone-2.60-150000.3.3.1.noarch.rpm [....done (184.2 KiB/s)]
worker-1  | Retrieving: perl-DateTime-1.45-1.16.x86_64 (Main Repository) (59/60), 143.0 KiB    
worker-1  | Retrieving: perl-DateTime-1.45-1.16.x86_64.rpm [..done (2.2 KiB/s)]
worker-1  | Retrieving: os-autoinst-distri-opensuse-deps-1.1702280286.cc5583fe-lp155.13341.1.noarch (devel_openQA) (60/60),   8.2 KiB    
worker-1  | Retrieving: os-autoinst-distri-opensuse-deps-1.1702280286.cc5583fe-lp155.13341.1.noarch.rpm [......done (5.1 KiB/s)]
worker-1  | 
worker-1  | Checking for file conflicts: [..done]
worker-1  | ( 1/60) Installing: perl-Perl-Tidy-20230912.0.0-lp155.2.1.noarch [..done]
worker-1  | ( 2/60) Installing: lsof-4.91-1.11.x86_64 [..done]
worker-1  | ( 3/60) Installing: perl-Archive-Zip-1.60-3.3.1.noarch [..done]
worker-1  | ( 4/60) Installing: perl-B-Keywords-1.24-bp155.1.5.noarch [..done]
worker-1  | ( 5/60) Installing: perl-Class-Inspector-1.32-1.22.noarch [..done]
worker-1  | ( 6/60) Installing: perl-Class-Singleton-1.5-1.24.noarch [..done]
worker-1  | ( 7/60) Installing: perl-Class-Tiny-1.006-1.24.noarch [..done]
worker-1  | ( 8/60) Installing: perl-Clone-0.46-bp155.1.5.x86_64 [..done]
worker-1  | ( 9/60) Installing: perl-Dist-CheckConflicts-0.11-1.23.noarch [..done]
worker-1  | (10/60) Installing: perl-Exception-Base-0.2501-bp155.2.8.noarch [..done]
worker-1  | (11/60) Installing: perl-File-Path-2.150000-1.22.noarch [..done]
worker-1  | (12/60) Installing: perl-IO-String-1.08-152.24.x86_64 [..done]
worker-1  | (13/60) Installing: perl-JSON-2.97001-1.18.noarch [..done]
worker-1  | (14/60) Installing: perl-Lingua-EN-Inflect-1.905-bp155.2.12.noarch [..done]
worker-1  | (15/60) Installing: perl-List-SomeUtils-XS-0.56-1.17.x86_64 [..done]
worker-1  | (16/60) Installing: perl-Module-Build-0.422400-1.24.noarch [..done]
worker-1  | (17/60) Installing: perl-Module-Pluggable-5.2-1.24.noarch [..done]
worker-1  | (18/60) Installing: perl-NetAddr-IP-4.079-1.27.x86_64 [..done]
worker-1  | (19/60) Installing: perl-Package-Stash-XS-0.28-1.26.x86_64 [..done]
worker-1  | (20/60) Installing: perl-Readonly-2.05-1.22.noarch [..done]
worker-1  | (21/60) Installing: perl-SemVer-0.10.0-bp155.1.4.noarch [..done]
worker-1  | (22/60) Installing: perl-String-Format-1.18-bp155.2.12.noarch [..done]
worker-1  | (23/60) Installing: perl-Sub-Identify-0.14-1.26.x86_64 [..done]
worker-1  | (24/60) Installing: perl-Symbol-Util-0.0203-bp155.2.12.noarch [..done]
worker-1  | (25/60) Installing: perl-Task-Weaken-1.06-2.13.noarch [..done]
worker-1  | (26/60) Installing: perl-Test-Fatal-0.014-1.23.noarch [..done]
worker-1  | (27/60) Installing: perl-Test-LongString-0.17-bp155.2.12.noarch [..done]
worker-1  | (28/60) Installing: perl-Tie-IxHash-1.23-1.23.noarch [..done]
worker-1  | (29/60) Installing: perl-Variable-Magic-0.62-1.19.x86_64 [..done]
worker-1  | (30/60) Installing: perl-XML-SAX-Expat-0.51-1.22.noarch [..done]
worker-1  | (31/60) Installing: perl-XML-Writer-0.625-1.24.noarch [..done]
worker-1  | (32/60) Installing: perl-Eval-Closure-0.14-1.22.noarch [..done]
worker-1  | (33/60) Installing: perl-File-ShareDir-1.104-1.22.noarch [..done]
worker-1  | (34/60) Installing: perl-Path-Tiny-0.104-1.22.noarch [..done]
worker-1  | (35/60) Installing: perl-Pod-MinimumVersion-50-bp155.3.10.noarch [..done]
worker-1  | (36/60) Installing: perl-List-SomeUtils-0.56-1.20.noarch [..done]
worker-1  | (37/60) Installing: perl-Code-DRY-0.10-bp155.2.10.x86_64 [..done]
worker-1  | (38/60) Installing: perl-Package-Stash-0.37-1.23.noarch [..done]
worker-1  | (39/60) Installing: perl-constant-boolean-0.02-bp155.2.8.noarch [..done]
worker-1  | (40/60) Installing: perl-PPI-1.276-bp155.1.4.noarch [..done]
worker-1  | (41/60) Installing: perl-B-Hooks-EndOfScope-0.21-1.22.noarch [..done]
worker-1  | (42/60) Installing: perl-XML-Simple-2.24-1.22.noarch [..done]
worker-1  | (43/60) Installing: perl-Specio-0.42-1.17.noarch [..done]
worker-1  | (44/60) Installing: perl-Params-ValidationCompiler-0.26-1.17.noarch [..done]
worker-1  | (45/60) Installing: perl-Pod-Spell-1.25-bp155.1.5.noarch [..done]
worker-1  | (46/60) Installing: perl-Test-Assert-0.0504-bp155.2.8.noarch [..done]
worker-1  | (47/60) Installing: perl-PPIx-Utilities-1.001000-bp155.3.9.noarch [..done]
worker-1  | (48/60) Installing: perl-PPIx-Regexp-0.086-bp155.1.5.noarch [..done]
worker-1  | (49/60) Installing: perl-PPIx-QuoteLike-0.023-bp155.1.5.noarch [..done]
worker-1  | (50/60) Installing: perl-namespace-clean-0.27-1.22.noarch [..done]
worker-1  | (51/60) Installing: perl-Perl-Critic-1.148-bp155.1.4.noarch [..done]
worker-1  | (52/60) Installing: perl-namespace-autoclean-0.28-1.22.noarch [..done]
worker-1  | (53/60) Installing: perl-Perl-Critic-Pulp-99-bp155.2.8.x86_64 [..done]
worker-1  | (54/60) Installing: perl-Perl-Critic-Policy-Variables-ProhibitLoopOnHash-0.008-bp155.2.8.noarch [..done]
worker-1  | (55/60) Installing: perl-DateTime-Locale-1.170000-1.17.noarch [.....done]
worker-1  | (56/60) Installing: perl-Perl-Critic-Community-1.0.3-bp155.1.4.noarch [..done]
worker-1  | (57/60) Installing: perl-Selenium-Remote-Driver-1.49-bp155.2.3.1.x86_64 [..done]
worker-1  | (58/60) Installing: perl-DateTime-TimeZone-2.60-150000.3.3.1.noarch [..done]
worker-1  | (59/60) Installing: perl-DateTime-1.45-1.16.x86_64 [..done]
worker-1  | (60/60) Installing: os-autoinst-distri-opensuse-deps-1.1702280286.cc5583fe-lp155.13341.1.noarch [..done]
worker-1  | /run_openqa_worker.sh: line 34:     9 Terminated              qemu-system-x86_64 -S
worker-1  | [info] [pid:287] worker 1:
worker-1  |  - config file:                      /etc/openqa/workers.ini
worker-1  |  - name used to register:            de1427a126e7
worker-1  |  - worker address (WORKER_HOSTNAME): worker
worker-1  |  - isotovideo version:               40
worker-1  |  - websocket API version:            1
worker-1  |  - web UI hosts:                     http://172.18.0.9
worker-1  |  - class:                            demohw
worker-1  |  - no cleanup:                       no
worker-1  |  - pool directory:                   /var/lib/openqa/pool/1
worker-1  | [info] [pid:287] Project dir for host http://172.18.0.9 is /var/lib/openqa/share
worker-1  | [info] [pid:287] Registering with openQA http://172.18.0.9
worker-1  | [info] [pid:287] Establishing ws connection via ws://172.18.0.9/api/v1/ws/15
worker-1  | [info] [pid:287] Registered and connected via websockets with openQA host http://172.18.0.9 and worker ID 15
worker-1  | [warn] [pid:287] Websocket connection to http://172.18.0.9/api/v1/ws/15 finished by remote side with code 1006, no reason - trying again in 10 seconds
worker-1  | [info] [pid:287] Registering with openQA http://172.18.0.9
worker-1  | [info] [pid:287] Establishing ws connection via ws://172.18.0.9/api/v1/ws/15
worker-1  | [info] [pid:287] Registered and connected via websockets with openQA host http://172.18.0.9 and worker ID 15
worker-1  | [warn] [pid:287] Websocket connection to http://172.18.0.9/api/v1/ws/15 finished by remote side with code 1006, no reason - trying again in 10 seconds
worker-1  | [info] [pid:287] Registering with openQA http://172.18.0.9
worker-1  | [info] [pid:287] Establishing ws connection via ws://172.18.0.9/api/v1/ws/15
worker-1  | [info] [pid:287] Registered and connected via websockets with openQA host http://172.18.0.9 and worker ID 15
worker-1  | [warn] [pid:287] Websocket connection to http://172.18.0.9/api/v1/ws/15 finished by remote side with code 1006, no reason - trying again in 10 seconds
worker-1  | [info] [pid:287] Registering with openQA http://172.18.0.9
worker-1  | [info] [pid:287] Establishing ws connection via ws://172.18.0.9/api/v1/ws/15
worker-1  | [info] [pid:287] Registered and connected via websockets with openQA host http://172.18.0.9 and worker ID 15
worker-1  | [debug] [pid:287] Accepting job 59 from http://172.18.0.9.
worker-1  | [debug] [pid:287] Setting job 59 from http://172.18.0.9 up
worker-1  | [debug] [pid:287] Preparing Mojo::IOLoop::ReadWriteProcess::Session
worker-1  | [info] [pid:287] +++ setup notes +++
worker-1  | [info] [pid:287] Running on de1427a126e7:1 (Linux 6.1.65 #1-NixOS SMP PREEMPT_DYNAMIC Sun Dec  3 06:32:13 UTC 2023 x86_64)
worker-1  | [debug] [pid:287] Job settings:
worker-1  |     ARCH=x86_64
worker-1  |     BACKEND=qemu
worker-1  |     DESKTOP=xfce
worker-1  |     DISTRI=demohw
worker-1  |     FLAVOR=generalhw
worker-1  |     HDDSIZEGB=100
worker-1  |     INSTALL=1
worker-1  |     JOBTOKEN=qbgKDWmM_1PE9jOP
worker-1  |     LOG_LEVEL=debug
worker-1  |     MACHINE=demohw
worker-1  |     NAME=00000059-demohw-v1-generalhw-x86_64-demohw@demohw
worker-1  |     NICTYPE=user
worker-1  |     OPENQA_HOSTNAME=172.18.0.9
worker-1  |     OPENQA_URL=http://172.18.0.9
worker-1  |     PRJDIR=/var/lib/openqa/share
worker-1  |     QEMUPORT=20012
worker-1  |     RETRY_DELAY=5
worker-1  |     RETRY_DELAY_IF_WEBUI_BUSY=60
worker-1  |     SSH_XTERM_WAIT_SUT_ALIVE_TIMEOUT=60
worker-1  |     TEST=demohw
worker-1  |     TEST_SUITE_NAME=demohw
worker-1  |     VERSION=v1
worker-1  |     VIDEOMODE=vnc
worker-1  |     VNC=91
worker-1  |     VNC_TYPING_LIMIT=7
worker-1  |     WORKER_CLASS=demohw
worker-1  |     WORKER_HOSTNAME=worker
worker-1  |     WORKER_ID=15
worker-1  |     WORKER_INSTANCE=1
worker-1  |     YAML_SCHEDULE=schedule/demohw.yaml
worker-1  | [debug] [pid:287] Symlinked from "/var/lib/openqa/share/tests/demohw" to "/var/lib/openqa/pool/1/demohw"
worker-1  | [info] [pid:287] Preparing cgroup to start isotovideo
worker-1  | [warn] [pid:287] Disabling cgroup usage because cgroup creation failed: mkdir /sys/fs/cgroup/systemd: Permission denied at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/File.pm line 85.
worker-1  | [info] [pid:287] You can define a custom slice with OPENQA_CGROUP_SLICE or indicating the base mount with MOJO_CGROUP_FS.
worker-1  | [info] [pid:287] Starting isotovideo container
worker-1  | [debug] [pid:287] Registered process:290
worker-1  | [info] [pid:287] isotovideo has been started (PID: 290)
worker-1  | [info] [pid:290] 290: WORKING 59
worker-1  | [debug] [pid:290] +++ worker notes +++
worker-1  | [debug] [pid:287] Running job 59 from http://172.18.0.9: 00000059-demohw-v1-generalhw-x86_64-demohw@demohw.
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (no current module)
worker-1  | [debug] [pid:287] Starting livelog
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (no current module)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (no current module)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (no current module)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (at boot)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (at boot)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (at boot)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (at boot)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (at boot)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (at boot)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (at boot)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (at boot)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (at boot)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (at boot)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (up to boot)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (no current module)
worker-1  | [info] [pid:287] Isotovideo exit status: 0
worker-1  | [debug] [pid:287] Stopping job 59 from http://172.18.0.9: 00000059-demohw-v1-generalhw-x86_64-demohw@demohw - reason: done
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [info] [pid:287] +++ worker notes +++
worker-1  | [info] [pid:287] End time: 2023-12-11 12:40:20
worker-1  | [info] [pid:287] Result: done
worker-1  | [info] [pid:413] Uploading video.ogv
worker-1  | [debug] [pid:413] Uploading artefact video.ogv
worker-1  | [info] [pid:413] Uploading autoinst-log.txt
worker-1  | [debug] [pid:413] Uploading artefact autoinst-log.txt
worker-1  | [info] [pid:413] Uploading worker-log.txt
worker-1  | [debug] [pid:413] Uploading artefact worker-log.txt
worker-1  | [info] [pid:413] Uploading vars.json
worker-1  | [debug] [pid:413] Uploading artefact vars.json
worker-1  | [info] [pid:413] Uploading serial0.txt
worker-1  | [debug] [pid:413] Uploading artefact serial0.txt
worker-1  | [info] [pid:413] Uploading video_time.vtt
worker-1  | [debug] [pid:413] Uploading artefact video_time.vtt
worker-1  | [info] [pid:413] Uploading serial_terminal.txt
worker-1  | [debug] [pid:413] Uploading artefact serial_terminal.txt
worker-1  | [info] [pid:413] Uploading serial_terminal_user.txt
worker-1  | [debug] [pid:413] Uploading artefact serial_terminal_user.txt
worker-1  | [debug] [pid:287] Considering job 59 as regularly done
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/status
worker-1  | [debug] [pid:287] Upload concluded (no current module)
worker-1  | [debug] [pid:287] REST-API call: POST http://172.18.0.9/api/v1/jobs/59/set_done?worker_id=15
worker-1  | [debug] [pid:287] Job 59 from http://172.18.0.9 finished - reason: done
worker-1  | [debug] [pid:287] Cleaning up for next job
worker-1  | [warn] [pid:287] Websocket connection to http://172.18.0.9/api/v1/ws/15 finished by remote side with code 1006, no reason - trying again in 10 seconds
worker-1  | [info] [pid:287] Registering with openQA http://172.18.0.9
worker-1  | [info] [pid:287] Establishing ws connection via ws://172.18.0.9/api/v1/ws/15
worker-1  | [info] [pid:287] Registered and connected via websockets with openQA host http://172.18.0.9 and worker ID 15
worker-1  | [warn] [pid:287] Websocket connection to http://172.18.0.9/api/v1/ws/15 finished by remote side with code 1006, no reason - trying again in 10 seconds
worker-1  | [info] [pid:287] Registering with openQA http://172.18.0.9
worker-1  | [info] [pid:287] Establishing ws connection via ws://172.18.0.9/api/v1/ws/15
worker-1  | [info] [pid:287] Registered and connected via websockets with openQA host http://172.18.0.9 and worker ID 15
worker-1  | [warn] [pid:287] Websocket connection to http://172.18.0.9/api/v1/ws/15 finished by remote side with code 1006, no reason - trying again in 10 seconds
worker-1  | [info] [pid:287] Registering with openQA http://172.18.0.9
worker-1  | [info] [pid:287] Establishing ws connection via ws://172.18.0.9/api/v1/ws/15
worker-1  | [info] [pid:287] Registered and connected via websockets with openQA host http://172.18.0.9 and worker ID 15

i change:

-    assert_screen 'bootloader';
+    assert_screen 'bootloaderr', timeout => 100;

to try to reproduce 504, here is the screenshot.

image

trdthg avatar Dec 11 '23 13:12 trdthg