bootc icon indicating copy to clipboard operation
bootc copied to clipboard
verified publisher icon containers

test: some updates on test

Open henrywang opened this issue 1 year ago • 1 comments

  1. add bootc swtich test. Switch to local folder /mnt in this case
  2. add bootc install to-disk test
  3. add case installing SELinux-enabled targets from SELinux-disabled hosts for https://github.com/containers/bootc/issues/419
  4. cover more distros, like rhel 9.5, fedora 40 and fedora 41(rawhide)
  5. upgrade testing-farm-as-github-action to v2

henrywang avatar May 20 '24 02:05 henrywang

(Huh, weird that you hit cargo fmt on docgen.rs, but not elsewhere. Anyways the fix is in https://github.com/containers/bootc/pull/547 )

cgwalters avatar May 20 '24 12:05 cgwalters

We're running a lot of tests per PR today, I'm not sure always need to all these tests per PR. Maybe we can move some of them to post-merge testing? For example testing both c9s and rhel9.4 makes sense, but I'm not sure about every PR. Similarly both x86_64 and aarch64.

I'd like to be able to opt-in (via a comment or label on a PR) to full testing, but my instinct here is to ensure we have baseline sanity testing to start...and what do you think about ensuring that if say we choose to say only test aarch64 on a PR for c9s (skipping x86_64) we are diligently watching post-merge testing and can quickly identify any breakage?

cgwalters avatar May 23 '24 16:05 cgwalters

I totally agree with you. Our daily test covered much more. We can catch failure with daily test. What about the plan like this:

  • rhel9.4: x86_64
  • rhel9.5: x86_64, aarch64
  • cs9: aarch64
  • fedora 40: x86_64
  • fedora 41: aarch64

henrywang avatar May 23 '24 23:05 henrywang

Let's drop fedora 41 there: I don't want to gate PRs on fedora rawhide stability. Or to future proof it the matrix would just have fedora $stable. We can also pretty confidently drop 9.4 I think for PR testing, the chance that we pass on 9.5 but break on 9.4 is currently very close to zero.

cgwalters avatar May 24 '24 00:05 cgwalters

This is a huge PR. Is it possible to break it into smaller PRs such that it is less risky and easier to review?

lmilbaum avatar May 24 '24 12:05 lmilbaum

This is a huge PR. Is it possible to break it into smaller PRs such that it is less risky and easier to review?

It's not easy to split this PR now. :-) I run this PR with testing farm. That can avoid test issue imported by this PR.

  • rhel 9.5 (x86_64): https://artifacts.osci.redhat.com/testing-farm/41ee2c13-5089-4578-91bb-92cb56f8e40f
  • rhel 9.5 (aarch64): https://artifacts.osci.redhat.com/testing-farm/d553cd14-cce6-4340-903c-ee2b38fa5feb
  • centos stream 9 (aarch64): https://artifacts.osci.redhat.com/testing-farm/25a63390-4026-4959-9e5a-495dfa0801a3
  • fedora 40 (x86_64): https://artifacts.osci.redhat.com/testing-farm/338cafff-eee7-46f0-af50-2e8f9ef23a11

This PR is ready to merge now.

henrywang avatar May 26 '24 14:05 henrywang