buildah icon indicating copy to clipboard operation
buildah copied to clipboard

tests: stop using ubi8 (this is the "0x3c" flake)

Open edsantiago opened this issue 3 years ago • 2 comments

Use of ubi8 images is causing one of our big flakes, mostly in podman (because it runs many more CI jobs than buildah) but also in buildah CI:

$ buildah build -f Containerfile --pull=false -q --arch=amd64 -t image-amd --signature-policy /var/tmp/go/src/github.com/containers/buildah/tests/./policy.json /var/tmp/buildah_tests.sufdm8/my-dir
error creating build container: copying system image from manifest list:
     reading signatures:
     parsing signature https://access.redhat.com/webassets/docker/content/sigstore/ubi8-minimal@sha256=SHA/signature-7:
     unrecognized signature format, starting with binary 0x3c

I think this is also the cause of the horrid multiple-platform-no-run flake, or at least one of the causes.

We can't rely on flaky registries. Can all tests that use ubi8-* be fixed to use a different image?

[Integration] 295 bud --pull=false --arch test

  • [unknown image] : Integration fedora-35 w/ vfs
  • [unknown image] : Integration rootless fedora-35 w/ overlay
  • [unknown image] : Integration rootless ubuntu-2204 w/ overlay

[Containerized] 295 bud --pull=false --arch test

...and, in podman-land:

[bud] 277 bud --pull=false --arch test

  • fedora-36 : bud podman fedora-36 root host
  • fedora-36 : bud remote fedora-36 root host [remote]

[bud] 268 bud --pull=false --arch test

[bud] 261 bud --pull=false --arch test

edsantiago avatar Oct 05 '22 12:10 edsantiago

Every. Single. Integration. Job. In. My. PR.

edsantiago avatar Oct 05 '22 19:10 edsantiago

For the record: This is the signature server serving 200 OK with HTML content; it is supposed to serve 200 OK with a signature, or 404 — or some non-200 error at least.

mtrmac avatar Oct 10 '22 15:10 mtrmac

All CI (buildah and podman), all gating tests, are now broken because ubi8 changed something.

✗ bud --pull=false --arch test
   #/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
   #|     FAIL: buildah inspect --format {{ index .Docker.Config.Labels "architecture" }} image-arm
   #| expected: =~ 'arm64'
   #|   actual:    'aarch64'
   #\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When we use images we don't control, on registries we don't control, bad things happen.

edsantiago avatar Oct 26 '22 02:10 edsantiago

Fixed in #4384 and #4397.

edsantiago avatar Nov 03 '22 22:11 edsantiago