ebuildtester icon indicating copy to clipboard operation
ebuildtester copied to clipboard

Experiments with libpod/podman

Open ZeroPointEnergy opened this issue 6 years ago • 12 comments

Hello

I started to experiment with ebuildtester and possibly using it with podman/libpod, because it supports rootless containers with user namespaces. The CLI is equivalent to dockers, so it should ideally be a drop-in replacement.

Here is what I did:

  • Installed and configured libpod with the crun runner (the default runc does not support cgroups v2 yet)
  • removed the SYS_ADMIN cap
  • created a symlink docker -> podman

Results:

  • It seems to work at least in parts. The container gets created and starts. It can install gentoolkit etc.
  • I get a lot of messages on stderr: Unable to unshare: EPERM (for FEATURES="ipc-sandbox network-sandbox")
  • It breaks on package glibc because that checks if /dev/pts is mounted with gid=5 which it isn't in the container.

So what do you think? is this worth investigating more?

ZeroPointEnergy avatar Apr 27 '20 10:04 ZeroPointEnergy

Hi @ZeroPointEnergy ! Thanks for the detailed information! Yes, I think it's worth investigating. I have been toying with the idea of using lxc which would be yet another container solution. Maybe what we should consider is to abstract the container calls so that we can modularize the actual container backend. The user could then choose what backend to use (i.e. docker, podman, lxc, etc.). This would also allow us to use libvirt for a full VM backend approach. What do you think?

nicolasbock avatar Apr 27 '20 12:04 nicolasbock

I think this would be great if there was the possibility to have multiple backends to run the tests on. Something like podman would be practically identical to docker, but lxc probably needs completely different commands.

ZeroPointEnergy avatar Apr 28 '20 06:04 ZeroPointEnergy

this is amazing, really nice that someone else has the same line of thought.

The dev problem is very common one for a lot of chroot and container based systems.

MAKEDEV from redhat is one way, another is to mount /dev/ as we do during the gentoo install.

I am inclined to choose MAKEDEV as mounting as a slave has infinitely many bad security implications plus is not doable as an unprivileged user.

comments?

epsilon-0 avatar May 16 '20 12:05 epsilon-0

Hmm, yes, to be able to be selective (as in MAKEDEV) sounds a lot saner to me :smile: I am all for it. This would also help the docker implementation.

nicolasbock avatar May 16 '20 13:05 nicolasbock

an aisde: there is another python app called jailkit which does the selective dev creation using pure python. I totally do not recommend using jailkit here, I use it for secure deployments in OpenBSD. I just wanted to show that there are interesting options which are platform independent and look cute :smile_cat:

epsilon-0 avatar May 16 '20 13:05 epsilon-0

what gentoo ~~recommends~~ has as an alternative instead of MAKEDEV or udev: mdev

~~At the point where systemd is not used, udev becomes a liability and redundancy.~~

I shouldn't write personal opinions, will just stick to facts

epsilon-0 avatar May 16 '20 13:05 epsilon-0

The lack of systemd in docker is kind of a pain to be honest. lxc for example handles this without issues. It would be nice if we could also test systemd in the container.

nicolasbock avatar May 16 '20 13:05 nicolasbock

Nice that someone else is interested in this as well :smile:

Testing seems to work fine mostly without any modification to ebuildtester if you just create a symlink docker -> podman.

Apart from the weird error messages which seem to have no consequences, the only real issue I had was the one with glibc which checks if devpts is mounted with gid=5. This isn't the case in the rootless container, but this is probably a bug as it shows the host uid and not the one that it maps.

I went to the podman IRC and talked to the developers about it. They said they can reproduce the problem without podman, so it probably is a kernel issue.

ZeroPointEnergy avatar May 16 '20 15:05 ZeroPointEnergy

This issue is stale because it has been open for 90 days with no activity. Remove the stale label or comment or this will be closed in 30 days.

github-actions[bot] avatar Jun 09 '22 13:06 github-actions[bot]

This issue was closed because it has been stale for 90 days with no activity.

github-actions[bot] avatar Jul 10 '22 03:07 github-actions[bot]

Seems some users using podman are having serious problems with ebuildtester. Here is a log reported on IRC gentoo-dev-help channel.

$ ebuildtester --docker-command /usr/bin/podman --docker-image docker.io/gentoo/stage3 --rm --portage-dir /home/user/gentoo --atom net-p2p/deluge-2.2.0
2025-06-26 22:03:36,734 - logging at /tmp/ebuildtester-net-p2p-deluge.log
2025-06-26 22:03:36,734 - *** please note that all necessary licenses will be accepted ***
2025-06-26 22:03:36,734 - creating container
2025-06-26 22:03:36,734 - creating docker container with: /usr/bin/podman create --tty --cap-add CAP_SYS_ADMIN --cap-add CAP_MKNOD --cap-add CAP_NET_ADMIN --security-opt apparmor:unconfined --device /dev/fuse --workdir /root --volume /home/user/gentoo:/var/db/repos/gentoo --volume /home/user/gentoo/distfiles:/var/cache/distfiles --volume /home/user/gentoo/packages:/var/cache/binpkgs docker.io/gentoo/stage3
Error: statfs /home/user/gentoo/distfiles: no such file or directory
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.13/ebuildtester", line 8, in <module>
    sys.exit(main())
             ~~~~^^
  File "/usr/lib/python3.13/site-packages/ebuildtester/main.py", line 30, in main
    container = Docker(
        os.path.abspath(os.path.expanduser(options.OPTIONS.portage_dir)),
        [os.path.abspath(p) for p in options.OPTIONS.overlay_dir])
  File "/usr/lib/python3.13/site-packages/ebuildtester/docker.py", line 28, in __init__
    self._create_container(docker_image, local_portage,
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                           zip(overlay_dirs, overlay_mountpoints))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/ebuildtester/docker.py", line 186, in _create_container
    raise Exception("failure creating docker container")
Exception: failure creating docker container

stkw0 avatar Jun 27 '25 09:06 stkw0

Fixed by #229.

kuraga avatar Jul 11 '25 19:07 kuraga