sysbox
sysbox copied to clipboard
nestybox
writing /etc/resolv.conf caused: open /etc/resolv.conf: permission denied: unknown without "userns-remap":"sysbox"; kernel version is fine
Despite it's said in https://github.com/nestybox/sysbox/blob/master/docs/user-guide/install-package.md#docker-userns-remap that "Starting with Sysbox v0.5.0, configuring Docker in userns-remap mode is no longer required, even if the host has no support for shiftfs. However, without shiftfs, you will need a host with kernel >= 5.12 so that Sysbox can use the kernel's ID-mapped mounts feature." - it doesn't work unfortunately.
Full error message:
$ docker compose up ci-runner
[+] Running 1/0
✔ Container ci-runner-ci-runner-1 Recreated Attaching to ci-runner-1
Error response from daemon: failed to create task for container: failed to create shim task:
OCI runtime create failed: container_linux.go:439: starting container process caused:
process_linux.go:608: container init caused: switching Docker DNS: rootfs_linux.go:1420:
writing /etc/resolv.conf caused: open /etc/resolv.conf: permission denied: unknown
# uname -a
Linux ip-10-40-83-20 5.15.0-1009-aws #11-Ubuntu SMP Thu May 26 19:39:49 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
# dpkg -l | grep sysbox
ii sysbox-ce 0.6.4.linux
# dpkg -l | grep docker
ii docker-buildx-plugin 0.14.0-1~ubuntu.22.04~jammy arm64
ii docker-ce 5:26.1.1-1~ubuntu.22.04~jammy arm64
ii docker-ce-cli 5:26.1.1-1~ubuntu.22.04~jammy arm64
ii docker-ce-rootless-extras 5:26.1.1-1~ubuntu.22.04~jammy arm64
ii docker-compose-plugin 2.27.0-1~ubuntu.22.04~jammy arm64
# cat /etc/docker/daemon.json
{
"runtimes": {
"sysbox-runc": {
"path": "/usr/bin/sysbox-runc"
}
},
"default-runtime": "sysbox-runc",
"bip": "172.20.0.1/16",
"default-address-pools": [
{
"base": "172.25.0.0/16",
"size": 24
}
]
}
When I add "userns-remap": "sysbox", it starts working.
The container image is based on "ubuntu:22.04".
I wanted some minimally changed configuration to work, since it's said that sysbox version and kernel version are new enough for it all to work without userns-remap.
Hi @dimikot,
Thanks for giving Sysbox a shot!
Can you share your docker-compose YAML so I can recreate locally please?
With Kernel 5.15 you shouldn't need to run Docker in userns-remap mode, as Sysbox will leverage the kernel's ID-mapped-mounts feature to ensure files show up with proper ownership inside the rootless Sysbox container. However we still recommend shiftfs until kernel 5.19 which adds overlayfs on ID-mapped-mounts support.
Thanks!
@ctalledo Thanks! Try using the following compose.yml, I think it's self-sufficient:
services:
ci-storage:
image: ghcr.io/dimikot/ci-storage:main
ports:
- 26022:22
environment:
- TZ
volumes:
- ci-storage-mnt:/mnt
volumes:
ci-storage-mnt:
external: false
See my linux kernel version in the above uname -a listing. It's 5.15.0-1009-aws.
I actually don't want shiftfs in sake of just performance, since its installation is not so trivial (compiling from source code etc.); I want the setup to be as vanilla as possible.
Hi @dimikot,
Apologies for the late reply.
I am not able to repro on an AWS Ubuntu Focal host with kernel 5.15:
root@sysbox-test:~# docker compose up
[+] Running 6/6
✔ ci-storage Pulled 7.9s
✔ a8b1c5f80c2d Pull complete 2.0s
✔ c47bb6d4174f Pull complete 7.1s
✔ cc39bbef79d9 Pull complete 7.1s
✔ 85d1c32a4d0e Pull complete 7.1s
✔ 4f4fb700ef54 Pull complete 7.2s
[+] Running 3/3
✔ Network root_default Created 0.0s
✔ Volume "root_ci-storage-mnt" Created 0.0s
✔ Container root-ci-storage-1 Created 3.3s Attaching to ci-storage-1
ci-storage-1 | 2024-05-24 02:06:05 UTC
ci-storage-1 | To access this container over SSH, a secret CI_STORAGE_PUBLIC_KEY or a mounted file /run/secrets/CI_STORAGE_PUBLIC_KEY should exist. The container will start, but it's not accessible, which may be fine in dev environment.
ci-storage-1 | Starting SSH server...
root@sysbox-test:~/nestybox/sysbox# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
07cd2f49e140 ghcr.io/dimikot/ci-storage:main "/root/entrypoint.sh" 48 seconds ago Up 44 seconds 0.0.0.0:26022->22/tcp root-ci-storage-1
root@sysbox-test:~/nestybox/sysbox# docker inspect root-ci-storage-1 | grep Runtime
"Runtime": "sysbox-runc",
root@sysbox-test:~/nestybox/sysbox# uname -a
Linux sysbox-test 5.15.0-1061-aws #67~20.04.1-Ubuntu SMP Wed Apr 17 15:09:54 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Not quite sure why if fails in your case ...
I also used Sysbox v0.6.4:
● sysbox.service - Sysbox container runtime
Loaded: loaded (/lib/systemd/system/sysbox.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2024-05-24 02:10:57 UTC; 5min ago
Docs: https://github.com/nestybox/sysbox
Main PID: 26146 (sh)
Tasks: 2 (limit: 8898)
Memory: 348.0K
CGroup: /system.slice/sysbox.service
├─26146 /bin/sh -c /usr/bin/sysbox-runc --version && /usr/bin/sysbox-mgr --version && /usr/bin/sysbox-fs --version && /bin/sleep infinity
└─26165 /bin/sleep infinity
May 24 02:10:57 ip-172-31-39-41 systemd[1]: Started Sysbox container runtime.
May 24 02:10:57 ip-172-31-39-41 sh[26147]: sysbox-runc
May 24 02:10:57 ip-172-31-39-41 sh[26147]: edition: Community Edition (CE)
May 24 02:10:57 ip-172-31-39-41 sh[26147]: version: 0.6.4
May 24 02:10:57 ip-172-31-39-41 sh[26147]: commit: 085502643ea5281652c6984eed9797872f22698a
May 24 02:10:57 ip-172-31-39-41 sh[26147]: built at: Sat Apr 6 16:43:31 UTC 2024
May 24 02:10:57 ip-172-31-39-41 sh[26147]: built by: Cesar Talledo
May 24 02:10:57 ip-172-31-39-41 sh[26147]: oci-specs: 1.1.0+dev
May 24 02:10:57 ip-172-31-39-41 sh[26154]: sysbox-mgr
May 24 02:10:57 ip-172-31-39-41 sh[26154]: edition: Community Edition (CE)
May 24 02:10:57 ip-172-31-39-41 sh[26154]: version: 0.6.4
May 24 02:10:57 ip-172-31-39-41 sh[26154]: commit: 03f5d7bc584fdcb2319b2c1831bd58581185fc1c
May 24 02:10:57 ip-172-31-39-41 sh[26154]: built at: Sat Apr 6 16:43:43 UTC 2024
May 24 02:10:57 ip-172-31-39-41 sh[26154]: built by: Cesar Talledo
May 24 02:10:57 ip-172-31-39-41 sh[26159]: sysbox-fs
May 24 02:10:57 ip-172-31-39-41 sh[26159]: edition: Community Edition (CE)
May 24 02:10:57 ip-172-31-39-41 sh[26159]: version: 0.6.4
May 24 02:10:57 ip-172-31-39-41 sh[26159]: commit: 1a678b72ac430009739fa6596b824f29b1f7fe2e
May 24 02:10:57 ip-172-31-39-41 sh[26159]: built at: Sat Apr 6 16:43:40 UTC 2024
May 24 02:10:57 ip-172-31-39-41 sh[26159]: built by: Cesar Talledo