coreos-assembler icon indicating copy to clipboard operation
coreos-assembler copied to clipboard

Multi-arch upgrade tests fail to handle parent build with missing arch

Open jlebon opened this issue 3 years ago • 0 comments

Our bump-lockfile job hit the following when running the aarch64 upgrade test:

[2022-07-28T21:04:40.149Z] + set -xeuo pipefail
[2022-07-28T21:04:40.149Z] ++ umask
[2022-07-28T21:04:40.149Z] + '[' 0000 = 0000 ']'
[2022-07-28T21:04:40.149Z] + umask 0022
[2022-07-28T21:04:40.149Z] + cosa kola --rerun --upgrades --no-test-exit-error
[2022-07-28T21:04:41.068Z] kola -p qemu-unpriv --output-dir tmp/kola-upgrade run-upgrade --rerun --no-test-exit-error --qemu-image-dir tmp/kola-qemu-cache -v --find-parent-image
[2022-07-28T21:04:41.068Z] 2022-07-28T21:04:40Z cli: Started logging at level INFO
[2022-07-28T21:04:41.068Z] 2022-07-28T21:04:40Z cli: Started logging at level INFO
[2022-07-28T21:04:41.323Z] Error: Received a 403 error in http response for: https://builds.coreos.fedoraproject.org/prod/streams/testing-devel/builds/36.20220727.20.1/aarch64/meta.json
[2022-07-28T21:04:41.323Z] 2022-07-28T21:04:41Z cli: Received a 403 error in http response for: https://builds.coreos.fedoraproject.org/prod/streams/testing-devel/builds/36.20220727.20.1/aarch64/meta.json
[2022-07-28T21:04:41.323Z] error: failed to execute cmd-kola: exit status 1
script returned exit code 1

The reason it failed is that aarch64 artifacts don't exist for this build.

We should do something similar to the pipeline here to find the parent commit: https://github.com/coreos/fedora-coreos-pipeline/blob/fc3532704a0c3082810d34ca461d52747a53150a/jobs/build.Jenkinsfile#L186-L203.

The kola area of interest is here: https://github.com/coreos/coreos-assembler/blob/73bdf387a9d25b461f718c76fa3455902acfddfb/mantle/cmd/kola/kola.go#L620-L639

(The comment there needs to be updated; bump-lockfile also falls in this category.) Instead of using the last release, it'd do the equivalent of the pipeline Groovy code to keep going backwards.

jlebon avatar Jul 29 '22 19:07 jlebon