osbuild-composer
osbuild-composer copied to clipboard
osbuild
test suite should use a fixed version of cloud-image-val
Currently, the latest tag is used, see https://github.com/osbuild/osbuild-composer/blob/7bf02771750dc7a48b752d05d04d2a45353a1845/test/cases/aws.sh#L219.
This is error-prone because any updates to cloud-image-val can cause osbuild-composer CI to start failing. This is especially true considering that cloud-image-val CI is usually red these days, see https://github.com/osbuild/cloud-image-val/commits/main Additionally, if I understand it correctly, this civ commit https://github.com/osbuild/cloud-image-val/commit/28797c0cc591284907b9a7380fa11b5f377d8dcd broke composer's CI and https://github.com/osbuild/osbuild-composer/pull/2932 was needed to fix it.
Therefore, we should just use a fixed version and update it from time to time (a bot can do it).
What do you think about the suggestion we made with @thozza in #2932 ? Only release updated cloud-image-val container once tests on main pass there. It's using the latest osbuild-composer rpms from main for the tests there.
i'm slightly concerned about keeping the CI bits in sync between these two projects. When we add a new distro in composer, it might be missing from cloud-image-val which might cause issues in the newly added distros.
Also, using a fixed environment is how we do most stuff (the whole manifest-db concept, fixed images, fixed repositories) so it makes sense from the consistency point of view.
Using a fixed version of CIV requires a lot of manual work, specially now that CIV is getting new important tests al the time. The errors caused in rhel 9 are an exception and were caused because I'm changing the way the CI uploads images. With this PR (https://github.com/osbuild/cloud-image-val/pull/113), no changed introduced in CIV should cause an error on osbuild-composer.
The only problem I see is what you said about syncing when you introduce a new release.
With respect of CIV CI being red. This only applies to what we called published images, where all the tests apply. For IB test only a subset of tests is being executed (test without the pub tag).
We plan to get this tests green, but other important tasks were prioritized.