youki
youki copied to clipboard
containers
Unable to start rootless container under Podman when Youki used as runtime
When I try to start a rootless podman container with Youki used as runtime, it tries to create a /run/youki directory, which, as not running as root, fails with permission denied.
Youki version: 0.0.2 (commit: 0.0.2-0-73dc75c)
% podman create --runtime /home/ondra/.cargo/bin/youki --name fedora fedora
128fed7131b2a59030b9691108977dd0b4fc4c1fcc6b064c6f4c16d615637b96
% podman start fedora
Error: failed to create directory /run/youki
Caused by:
Permission denied (os error 13)
ERRO[0000] Error removing container 128fed7131b2a59030b9691108977dd0b4fc4c1fcc6b064c6f4c16d615637b96 from runtime after creation failed
Error: unable to start container "128fed7131b2a59030b9691108977dd0b4fc4c1fcc6b064c6f4c16d615637b96": Permission denied (os error 13): OCI permission denied
@ondra05 Thanks for your report. But, I couldn't reproduce this error in my environment.
$ podman create --runtime /home/utam0k/ghq/github.com/utam0k/youki/youki --name fedora fedora
WARN[0000] "/" is not a shared mount, this could cause issues or missing mounts with rootless containers
Resolved "fedora" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull registry.fedoraproject.org/fedora:latest...
Getting image source signatures
Copying blob c6183d119aa8 done
Copying config e417cd49a8 done
Writing manifest to image destination
Storing signatures
e60cafa77762fe924e97d1d9e9ce2cdf75315517ed511b1eb53e0058ba0bfbe0
Please tell me the result of youki info?
Sure thing!
Version 0.0.2
Commit 73dc75c
Kernel-Release 5.16.8-200.fc35.x86_64
Kernel-Version #1 SMP PREEMPT Tue Feb 8 20:58:59 UTC 2022
Architecture x86_64
Operating System Fedora Linux 35 (Workstation Edition)
Cores 8
Total Memory 15699
Cgroup setup unified
Cgroup mounts
unified /sys/fs/cgroup
[WARN crates/libcgroups/src/v2/util.rs:41] 2022-02-20T10:19:40.888719702+01:00 Controller misc is not yet implemented.
CGroup v2 controllers
cpu attached
cpuset attached
hugetlb attached
io attached
memory attached
pids attached
device attached
Namespaces enabled
mount enabled
uts enabled
ipc enabled
user enabled
pid enabled
network enabled
cgroup enabled
Well, the issue happnens when I tried to start the container, creation of the container worked just fine.
I have the same issue with Podman 3.4.2 and Youki 0.0.3 on Ubuntu 21.10:
$ podman run --runtime youki --rm -it ubuntu
Error: failed to create directory /run/youki
Caused by:
Permission denied (os error 13)
ERRO[0000] Error removing container 98fd9fee35115f1f92adff042897f827271ca4bcf6e607c6d4666a4860caef65 from runtime after creation failed
Error: Permission denied (os error 13): OCI permission denied
$ youki info
Version 0.0.3
Commit 541bf4a
Kernel-Release 5.13.0-39-generic
Kernel-Version #44-Ubuntu SMP Thu Mar 24 15:35:05 UTC 2022
Architecture x86_64
Operating System Ubuntu 21.10
Cores 4
Total Memory 15767
Cgroup setup unified
Cgroup mounts
unified /sys/fs/cgroup
[WARN libcgroups/src/v2/util.rs:41] 2022-04-14T16:09:49.230724310+02:00 Controller rdma is not yet implemented.
[WARN libcgroups/src/v2/util.rs:41] 2022-04-14T16:09:49.230868203+02:00 Controller misc is not yet implemented.
CGroup v2 controllers
cpu attached
cpuset attached
hugetlb attached
io attached
memory attached
pids attached
device attached
Namespaces enabled
mount enabled
uts enabled
ipc enabled
user enabled
pid enabled
network enabled
cgroup enabled
Hi,
I encountered the same issue on Fedora 36 with podman v4.1.0. It seems that mkdir /etc/youki got EACCESS.
$ podman --runtime=$(pwd)/youki run --rm --name test hello-world
[DEBUG crates/youki/src/main.rs:92] 2022-05-17T17:36:43.548188063+09:00 started by user 0 with ArgsOs { inner: ["/home/ori/devel/src/github.com/containers/youki/work/youki", "delete", "--force", "8d38990d6a0ded9fb3797bb996a7b5ec43a9b411a65fec108034a1e83c387c64"] }
Error: failed to create directory /run/youki
Caused by:
Permission denied (os error 13)
ERRO[0000] Removing container 8d38990d6a0ded9fb3797bb996a7b5ec43a9b411a65fec108034a1e83c387c64 from runtime after creation failed
Error: /home/ori/devel/src/github.com/containers/youki/work/youki: Permission denied (os error 13): OCI permission denied
$ ./youki info
[DEBUG crates/youki/src/main.rs:92] 2022-05-17T17:36:07.126714051+09:00 started by user 1000 with ArgsOs { inner: ["./youki", "info"] }
Version 0.0.3
Commit 05ce5c6
Kernel-Release 5.17.7-300.fc36.x86_64
Kernel-Version #1 SMP PREEMPT Thu May 12 14:56:44 UTC 2022
Architecture x86_64
Operating System Fedora Linux 36 (Workstation Edition)
Cores 12
Total Memory 64043
Cgroup setup unified
Cgroup mounts
unified /sys/fs/cgroup
[WARN crates/libcgroups/src/v2/util.rs:41] 2022-05-17T17:36:07.156835792+09:00 Controller misc is not yet implemented.
CGroup v2 controllers
cpu attached
cpuset attached
hugetlb attached
io attached
memory attached
pids attached
device attached
Namespaces enabled
mount enabled
uts enabled
ipc enabled
user enabled
pid enabled
network enabled
cgroup enabled
Hmm .. I tried some "dbg!-ing" and found out that the check for rootless_required() in determine_root_path() behaves weirdly.
When running e.g. ./youki info it returns true (UID is 1000):
$ ./youki info
[DEBUG crates/youki/src/main.rs:92] 2022-05-18T23:21:42.556071673+02:00 started by user 1000 with ArgsOs { inner: ["./youki", "info"] }
[crates/youki/src/main.rs:133] getuid().as_raw() = 1000
[crates/youki/src/main.rs:135] root_path = None
[crates/youki/src/main.rs:143] rootless_required() = true
[crates/youki/src/main.rs:151] Path::new(&path).join("youki") = "/run/user/1000/youki"
Version 0.0.3
Commit 05ce5c6
Kernel-Release 5.15.0-30-generic
Kernel-Version #31-Ubuntu SMP Thu May 5 10:00:34 UTC 2022
Architecture x86_64
Operating System Ubuntu 22.04 LTS
Cores 4
Total Memory 15767
Cgroup setup unified
Cgroup mounts
unified /sys/fs/cgroup
[WARN crates/libcgroups/src/v2/util.rs:41] 2022-05-18T23:21:42.621769976+02:00 Controller rdma is not yet implemented.
[WARN crates/libcgroups/src/v2/util.rs:41] 2022-05-18T23:21:42.621803261+02:00 Controller misc is not yet implemented.
CGroup v2 controllers
cpu attached
cpuset attached
hugetlb attached
io attached
memory attached
pids attached
device attached
Namespaces enabled
mount enabled
uts enabled
ipc enabled
user enabled
pid enabled
network enabled
cgroup enabled
But when running it though podman (as a non-root user) it returns false (UID is 0, but should be 1000) :raised_eyebrow: :
$ podman run --runtime $PWD/youki --rm hello-world
[DEBUG crates/youki/src/main.rs:92] 2022-05-18T23:22:05.152662158+02:00 started by user 0 with ArgsOs { inner: ["/home/riyad/src/youki/youki", "delete", "--force", "cc560217ade908d275148f4b162e35e6ac668106460a232853afe0fca505f504"] }
[crates/youki/src/main.rs:133] getuid().as_raw() = 0
[crates/youki/src/main.rs:135] root_path = None
[crates/youki/src/main.rs:143] rootless_required() = false
[crates/youki/src/main.rs:144] get_default_not_rootless_path() = "/run/youki"
Error: failed to create directory /run/youki
Caused by:
Permission denied (os error 13)
time="2022-05-18T23:22:05+02:00" level=error msg="Error removing container cc560217ade908d275148f4b162e35e6ac668106460a232853afe0fca505f504 from runtime after creation failed"
Error: Permission denied (os error 13): OCI permission denied
Hi, @Furisto Do you have time to check this because you assigned it by yourself. But if you don't have time, please let me. I'll check.
I wrapped youki in a scrip that set the environment variable YOUKI_USE_ROOTLESS=true, which allowed this to progress farther. It then complained "rootless container requires valid user namespace definition", which I seemed to have resolved by passing --userns=auto to podman run.
This results in another failure, where youki attempts to connect to the system dbus (instead of the session dbus). It looks like it has the same effective uid check as determine_root_path:
https://github.com/containers/youki/blob/a72a33b3e946a33d83d555f210e89401ef57cef6/crates/libcgroups/src/common.rs#L230
The behavior I am seeing is weird. Podman is not started as root, but youki seems to be. If I create a rootless container with youki without podman it works and I am seeing the correct (unprivileged) uid, so this does not seem to be a bug with the detection of the user in youki.
The config.json that podman provides to youki does not contain a user namespace either, which is further evidence that podman is not trying to create a rootless container. Youki also has no problems creating the /run/youki directory when started standalone, only when started by podman. Maybe podman has a security policy that prevents it? Will look further into this.
It seems to me that checking nix::unistd::geteuid().is_root() only is not enough to determine rootless.
Maybe we should consult /proc/self/uid_map also?
@orimanabu Thanks. @Furisto WDYT? If you don't have time, please tell me, and I can take over
My previous comment was just a guess, but I did some more research.
When in rootless mode, podman creates user namespace in the very early phase. (userns is created in PersistentPreRunE of rootCmd.) So When youki is invoked by podman, youki runs as root, uid mapped in the userns.
runc and crun seem to check if /proc/self/uid_map contains "4294967295" whether they run in rootless.
Should youki do the same, maybe in rootless_required() ?
@ondra05 this took a long time, but with current main, this should be resolved. Can you verify once, otherwise I'll close this in few days, Thanks!
