toolbox .zuul, playbooks: Run the CI on Arch Linux

https://github.com/containers/toolbox/issues/1438

Nov 05 '24 18:11 debarshiray

/cc @Foxboron

Nov 05 '24 19:11 debarshiray

Cool, this looks good to me

Nov 05 '24 20:11 Foxboron

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/93d834d5d12a43cabeb4911480753050

Nov 05 '24 20:11 softwarefactory-project-zuul[bot]

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/d2c652cccb984d439c8dfffbb32190f9

Nov 06 '24 02:11 softwarefactory-project-zuul[bot]

recheck

Nov 06 '24 11:11 debarshiray

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/56047e4f12334e94985197ae6ed387df

Nov 06 '24 12:11 softwarefactory-project-zuul[bot]

recheck

Nov 07 '24 10:11 danpawlik

Updated image once again.

Nov 07 '24 11:11 danpawlik

recheck

Nov 07 '24 11:11 danpawlik

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/84aed52fb1c046f69095d8f7bddcd698

Nov 07 '24 12:11 softwarefactory-project-zuul[bot]

recheck

Nov 07 '24 13:11 debarshiray

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/7bb927156e43405286c621d43b91daf6

Nov 07 '24 14:11 softwarefactory-project-zuul[bot]

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/e468fd9a7a0b40a295e2867efa42c662

Nov 12 '24 15:11 softwarefactory-project-zuul[bot]

@Foxboron It seems that installing Podman or Toolbx on an Arch Linux host doesn't install fuse-overlayfs. Am I reading it correctly?

We had recently removed fuse-overlayfs as a weak dependency of Podman, through the containers-common package, in Fedora as part of an effort to deprecate it, only to later give up and restored it.

Users with Podman containers and images that predate the use of the Linux kernel's overlay file system still need fuse-overlayfs. Otherwise it leads to: https://discussion.fedoraproject.org/t/rpm-ostree-update-breaks-toolbox-fedora-40/120095 https://github.com/containers/toolbox/issues/1512

Another problem is if Podman is supposed to be usable on a live media that itself uses the Linux kernel's overlayfs. It's not possible to use overlayfs on top of another overlayfs.

This led to a discussion that ended up restoring fuse-overlayfs: https://github.com/coreos/fedora-coreos-tracker/issues/1749 https://github.com/containers/common/pull/2203

It's fine to drop fuse-overlayfs if these scenarios don't matter for Arch Linux. Otherwise, it might be important.

Nov 12 '24 16:11 debarshiray

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/2351bee194874fb4be3d9bd061ae921a

Nov 12 '24 18:11 softwarefactory-project-zuul[bot]

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/ce5be7f5eb9f42c38ec6fcdfde14feee

Nov 12 '24 22:11 softwarefactory-project-zuul[bot]

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/091bb6e2bff24890bb59e865f5bfc399

Nov 14 '24 03:11 softwarefactory-project-zuul[bot]

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/a3d4739c6ba8449ca52660dfa07fb6a6

Nov 15 '24 22:11 softwarefactory-project-zuul[bot]

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/1a5782ab98344213b1315e5873efba0e

Nov 17 '24 12:11 softwarefactory-project-zuul[bot]

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/5a46c0fb8c554447a62e38615083372f

Nov 17 '24 17:11 softwarefactory-project-zuul[bot]

The runtime-environment tests are failing with:

arch | not ok 3 network: /etc/resolv.conf inside the default container
arch | # tags: arch-fedora runtime-environment
arch | # (from function `assert_line' in file test/system/libs/bats-assert/src/assert.bash, line 488,
arch | #  in test file test/system/203-network.bats, line 72)
arch | #   `assert_line --index 0 "/run/host/etc/resolv.conf"' failed
arch | # ~ ~/src/github.com/containers/toolbox
arch | #
arch | # -- line differs --
arch | # index    : 0
arch | # expected : /run/host/etc/resolv.conf
arch | # actual   : /run/host/run/systemd/resolve/stub-resolv.conf
arch | # --
arch | #
...
...
arch | not ok 36 ulimit: real-time non-blocking time (hard)
arch | # tags: arch-fedora runtime-environment
arch | # (from function `assert_line' in file test/system/libs/bats-assert/src/assert.bash, line 488,
arch | #  in test file test/system/210-ulimit.bats, line 45)
arch | #   `assert_line --index 0 "$limit"' failed
arch | #
arch | # -- line differs --
arch | # index    : 0
arch | # expected : unlimited
arch | # actual   :
arch | # --
arch | #

I might have to install an Arch Linux host to see how things are on the host.

Nov 17 '24 18:11 debarshiray

Build succeeded. https://softwarefactory-project.io/zuul/t/local/buildset/edfb38c2e94d462eb31121c12489b72c

Nov 17 '24 20:11 softwarefactory-project-zuul[bot]

The runtime-environment tests are failing with:

arch | not ok 3 network: /etc/resolv.conf inside the default container
arch | # tags: arch-fedora runtime-environment
arch | # (from function `assert_line' in file test/system/libs/bats-assert/src/assert.bash, line 488,
arch | #  in test file test/system/203-network.bats, line 72)
arch | #   `assert_line --index 0 "/run/host/etc/resolv.conf"' failed
arch | # ~ ~/src/github.com/containers/toolbox
arch | #
arch | # -- line differs --
arch | # index    : 0
arch | # expected : /run/host/etc/resolv.conf
arch | # actual   : /run/host/run/systemd/resolve/stub-resolv.conf
arch | # --
arch | #

The problem with /etc/resolv.conf is that the images generated by arch-boxes have /etc/resolv.conf as an absolute symbolic link, instead of relative.

I have now added a note to the Toolbx tests to detect such cases without failing. I will send a merge request to arch-boxes once I get access to gitlab.archlinux.org.

Nov 18 '24 20:11 debarshiray

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/84d623e695d14b88b651d513fe4cf719

Nov 18 '24 21:11 softwarefactory-project-zuul[bot]

There's also this failure that I didn't notice before:

arch | not ok 7 network: DNS inside the default container
arch | # tags: arch-fedora runtime-environment
arch | # (from function `assert_success' in file test/system/libs/bats-assert/src/assert.bash, line 114,
arch | #  in test file test/system/203-network.bats, line 228)
arch | #   `assert_success' failed
arch | # ~ ~/src/github.com/containers/toolbox
arch | #
arch | # -- command failed --
arch | # status : 127
arch | # output :
arch | # --
arch | #

Nov 18 '24 22:11 debarshiray

I need to context switch from this for a few days. So, I have merged the finished patches from this pull request through https://github.com/containers/toolbox/pull/1597

Nov 18 '24 22:11 debarshiray

The runtime-environment tests are failing with:
arch | not ok 3 network: /etc/resolv.conf inside the default container
arch | # tags: arch-fedora runtime-environment
arch | # (from function `assert_line' in file test/system/libs/bats-assert/src/assert.bash, line 488,
arch | #  in test file test/system/203-network.bats, line 72)
arch | #   `assert_line --index 0 "/run/host/etc/resolv.conf"' failed
arch | # ~ ~/src/github.com/containers/toolbox
arch | #
arch | # -- line differs --
arch | # index    : 0
arch | # expected : /run/host/etc/resolv.conf
arch | # actual   : /run/host/run/systemd/resolve/stub-resolv.conf
arch | # --
arch | #
The problem with /etc/resolv.conf is that the images generated by arch-boxes have /etc/resolv.conf as an absolute symbolic link, instead of relative.

I have now added a note to the Toolbx tests to detect such cases without failing. I will send a merge request to arch-boxes once I get access to gitlab.archlinux.org.

There was already an attempt to change /etc/resolv.conf into a relative symbolic link: https://gitlab.archlinux.org/archlinux/arch-boxes/-/merge_requests/194 https://gitlab.archlinux.org/archlinux/arch-boxes/-/commit/ff22bfc49ac60001849fcce73fe91e5fbabcf03a

... but it had to be reverted: https://gitlab.archlinux.org/archlinux/arch-install-scripts/-/issues/55 https://gitlab.archlinux.org/archlinux/arch-boxes/-/commit/5768351a8f93e102fb8ab5d248fd35fbe53863ff

Nov 21 '24 14:11 debarshiray