sysbox
sysbox copied to clipboard
Published
20 hours ago •
nestybox
nestybox
ID mapping problem inside container
Thanks for sysbox, it's great!
I'm having an issue with files inside sysbox container appearing with nobody:nogroup. Strangely, not all files, but some. I have read several issues here, and i'm looking for culprits.
I have upgraded my host Ubuntu 22.04.3 from kernel 5.15.0-113 to 6.5.0-41 trying to solve the problem to no avail.
I'm looking at LVM now, because my host /var/lib/docker is mounted on a LVM volume (not sure if this still applies, I've seen some old issues)
here are some info I think you should ask, some sensitive or noise info are omitted:
HOST INFO:
# lsb_release -a
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy
# uname -a
Linux 6.5.0-41-generic #41~22.04.2-Ubuntu SMP PREEMPT_DYNAMIC Mon Jun 3 11:32:55 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
lsmod | grep shiftfs
<not loaded>
# lsblk -f
NAME FSTYPE FSVER LABEL MOUNTPOINTS
sda
└─sda1 LVM2_member LVM2 001
└─vld-lvvld ext4 1.0 lvm-varlibdocker /var/lib/docker
sdc
├─sdc1
├─sdc2 ext4 1.0 /boot
└─sdc3 LVM2_member LVM2 001
└─ubuntu--vg-ubuntu--lv ext4 1.0 /
# cat /etc/default/grub
GRUB_CMDLINE_LINUX="ipv6.disable=1"
# systemctl status sysbox-mgr
Jul 01 20:20:58 sysbox-mgr[31471]: level=info msg="Starting ..."
Jul 01 20:20:58 sysbox-mgr[31471]: level=info msg="Sysbox data root: /var/lib/sysbox"
Jul 01 20:20:58 sysbox-mgr[31471]: level=info msg="Shiftfs module found in kernel: no"
Jul 01 20:20:58 sysbox-mgr[31471]: level=info msg="Shiftfs works properly: no"
Jul 01 20:20:58 sysbox-mgr[31471]: level=info msg="Shiftfs-on-overlayfs works properly: no"
Jul 01 20:20:58 sysbox-mgr[31471]: level=info msg="ID-mapped mounts supported by kernel: yes"
Jul 01 20:20:58 sysbox-mgr[31471]: level=info msg="Overlayfs on ID-mapped mounts supported by kernel: yes"
Jul 01 20:20:58 sysbox-mgr[31471]: level=info msg="Operating in system container mode."
Jul 01 20:20:58 sysbox-mgr[31471]: level=info msg="Inner container image preloading enabled."
Jul 01 20:20:58 sysbox-mgr[31471]: level=info msg="Listening on /run/sysbox/sysmgr.sock"
Jul 01 20:20:58 sysbox-mgr[31471]: level=info msg="Ready ..."
# systemctl status sysbox-fs -n 20
Jul 01 20:58:03 sysbox-fs[31492]: time="2024-07-01 20:58:03" level=warning msg="Received seccompNotifMsg generated by unknown container: 6ac160a4b546"
Jul 01 20:58:03 sysbox-fs[31492]: time="2024-07-01 20:58:03" level=warning msg="Received seccompNotifMsg generated by unknown container: 6ac160a4b546"
Jul 01 20:58:03 sysbox-fs[31492]: time="2024-07-01 20:58:03" level=warning msg="Received seccompNotifMsg generated by unknown container: 6ac160a4b546"
Jul 01 20:58:03 sysbox-fs[31492]: time="2024-07-01 20:58:03" level=warning msg="Received seccompNotifMsg generated by unknown container: 6ac160a4b546"
Jul 01 20:58:03 sysbox-fs[31492]: time="2024-07-01 20:58:03" level=warning msg="Received seccompNotifMsg generated by unknown container: 6ac160a4b546"
Jul 01 20:58:03 sysbox-fs[31492]: time="2024-07-01 20:58:03" level=warning msg="Received seccompNotifMsg generated by unknown container: 6ac160a4b546"
Jul 02 08:24:42 sysbox-fs[31492]: time="2024-07-02 08:24:42" level=info msg="Container pre-registration completed: id = 6ac160a4b546"
Jul 02 08:24:42 sysbox-fs[31492]: time="2024-07-02 08:24:42" level=info msg="Container registration completed: id = 6ac160a4b546, initPid = 64477, uid:gid = 165536:165536"
Jul 02 08:33:58 sysbox-fs[31492]: time="2024-07-02 08:33:58" level=info msg="Container pre-registration completed: id = 0757f1a11a71"
Jul 02 08:33:58 sysbox-fs[31492]: time="2024-07-02 08:33:58" level=info msg="Container registration completed: id = 0757f1a11a71, initPid = 70628, uid:gid = 165536:165536"
# systemctl status sysbox -n 20
Jul 01 20:20:58 sh[31502]: sysbox-runc
Jul 01 20:20:58 sh[31502]: edition: Community Edition (CE)
Jul 01 20:20:58 sh[31502]: version: 0.6.2
Jul 01 20:20:58 sh[31502]: oci-specs: 1.0.2-dev
Jul 01 20:20:58 sh[31508]: sysbox-mgr
Jul 01 20:20:58 sh[31508]: edition: Community Edition (CE)
Jul 01 20:20:58 sh[31508]: version: 0.6.2
Jul 01 20:20:58 sh[31513]: sysbox-fs
Jul 01 20:20:58 sh[31513]: edition: Community Edition (CE)
Jul 01 20:20:58 sh[31513]: version: 0.6.2
# cat /etc/docker/daemon.json
{
"bip": "192.168.60.1/27",
"default-address-pools": [
{
"base": "192.168.61.0/24",
"size": 27
}
],
"ip-masq": true,
"ipv6": false,
"default-runtime": "sysbox-runc",
"runtimes": {
"sysbox-runc": {
"path": "/usr/bin/sysbox-runc"
}
}
}
# docker info
Client: Docker Engine - Community
Version: 24.0.6
Context: default
Server:
Storage Driver: overlay2
# alias dps='docker ps -a --format '\''table {{.ID}}\t{{.Names}}\t{{.Status}}\t{{.Ports}}'\'''
# dps
CONTAINER ID NAMES STATUS PORTS
6ac160a4b546 hmapp03 Up 4 hours 5000-5050->5000-5050/tcp, 5051-5100/tcp, 2211->22/tcp
#docker run for container:
docker run \
--runtime=sysbox-runc \
--hostname hmapp03 \
--name hmapp03 \
--restart=unless-stopped \
--mount source=varlibdocker-hmapp03-v1,target=/var/lib/docker \
--ip 192.168.61.5 \
--network br-hmnet \
-p 2211:22 \
-p 5000-5050:5000-5050 \
--detach \
ubuntu-jammy-systemd-docker:v4 <this is a custom image>
INSIDE SYSBOX CONTAINER INFO (HMAPP03):
I've noticed on hmapp03, var/lib/docker is not idmapped!
#mount | grep docker
/dev/mapper/vld-lvvld on /var/lib/docker type ext4 (rw,relatime)
# findmnt -J
# docker exec -it hmapp03 findmnt -J
{
"filesystems": [
{
"target": "/",
"source": "overlay",
"fstype": "overlay",
"options": "rw,relatime,lowerdir=/var/lib/docker/overlay2/l/YKFESOSMHM6Z6CQID7P4P4H3DN:/var/lib/docker/overlay2/l/DAEVDBOZUPJINUQRLNVL2AXQRZ:/var/lib/docker/overlay2/l/2Y25TM5F7W3MDXAFE4RWSXYBYA:/var/lib/docker/overlay2/l/VQJ6BA3564D65XP2YOLXMM2XPO:/var/lib/docker/overlay2/l/AMSLSHZXUEZRUVE6S76C7ITFXK:/var/lib/docker/overlay2/l/MTD6J762Q4K6XBMIDH65CT55Z3:/var/lib/docker/overlay2/l/32OYFFIY5KGNXMKSYJAG636LDT:/var/lib/docker/overlay2/l/D3BINRCBRXO2PNY3WVUIWPYX2U:/var/lib/docker/overlay2/l/5FNLNOK5VPY3C7HI5HGCMO4B2J:/var/lib/docker/overlay2/l/PJX37BIXPOV7QULXJFHIWIEB2E:/var/lib/docker/overlay2/l/FOCPSCVXCLWATQBHZCPOD6H2RN:/var/lib/docker/overlay2/l/5IWJDNKDG64HJFOPLRNRYERE4E,upperdir=/var/lib/docker/overlay2/87d1553acd6027d1b0a47d459dd85f5e52ed71cfa6dea415640acee49872bcbe/diff,workdir=/var/lib/docker/overlay2/87d1553acd6027d1b0a47d459dd85f5e52ed71cfa6dea415640acee49872bcbe/work,nouserxattr",
"children": [
{
"target": "/sys",
"source": "sysfs",
"fstype": "sysfs",
"options": "rw,nosuid,nodev,noexec,relatime",
"children": [
{
"target": "/sys/firmware",
"source": "tmpfs",
"fstype": "tmpfs",
"options": "ro,relatime,uid=165536,gid=165536,inode64"
},{
"target": "/sys/fs/cgroup",
"source": "cgroup",
"fstype": "cgroup2",
"options": "rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot"
},{
"target": "/sys/devices/virtual",
"source": "sysboxfs[/sys/devices/virtual]",
"fstype": "fuse",
"options": "rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other"
},{
"target": "/sys/kernel",
"source": "sysboxfs[/sys/kernel]",
"fstype": "fuse",
"options": "rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other"
},{
"target": "/sys/module/nf_conntrack/parameters",
"source": "sysboxfs[/sys/module/nf_conntrack/parameters]",
"fstype": "fuse",
"options": "rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other"
}
]
},{
"target": "/proc",
"source": "proc",
"fstype": "proc",
"options": "rw,nosuid,nodev,noexec,relatime",
"children": [
{
"target": "/proc/bus",
"source": "proc[/bus]",
"fstype": "proc",
"options": "ro,nosuid,nodev,noexec,relatime"
},{
"target": "/proc/fs",
"source": "proc[/fs]",
"fstype": "proc",
"options": "ro,nosuid,nodev,noexec,relatime"
},{
"target": "/proc/irq",
"source": "proc[/irq]",
"fstype": "proc",
"options": "ro,nosuid,nodev,noexec,relatime"
},{
"target": "/proc/sysrq-trigger",
"source": "proc[/sysrq-trigger]",
"fstype": "proc",
"options": "ro,nosuid,nodev,noexec,relatime"
},{
"target": "/proc/acpi",
"source": "tmpfs",
"fstype": "tmpfs",
"options": "ro,relatime,uid=165536,gid=165536,inode64"
},{
"target": "/proc/keys",
"source": "udev[/null]",
"fstype": "devtmpfs",
"options": "rw,nosuid,relatime,size=3996644k,nr_inodes=999161,mode=755,inode64"
},{
"target": "/proc/timer_list",
"source": "udev[/null]",
"fstype": "devtmpfs",
"options": "rw,nosuid,relatime,size=3996644k,nr_inodes=999161,mode=755,inode64"
},{
"target": "/proc/scsi",
"source": "tmpfs",
"fstype": "tmpfs",
"options": "ro,relatime,uid=165536,gid=165536,inode64"
},{
"target": "/proc/swaps",
"source": "sysboxfs[/proc/swaps]",
"fstype": "fuse",
"options": "rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other"
},{
"target": "/proc/sys",
"source": "sysboxfs[/proc/sys]",
"fstype": "fuse",
"options": "rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other"
},{
"target": "/proc/uptime",
"source": "sysboxfs[/proc/uptime]",
"fstype": "fuse",
"options": "rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other"
}
]
},{
"target": "/dev",
"source": "tmpfs",
"fstype": "tmpfs",
"options": "rw,nosuid,size=65536k,mode=755,uid=165536,gid=165536,inode64",
"children": [
{
"target": "/dev/mqueue",
"source": "mqueue",
"fstype": "mqueue",
"options": "rw,nosuid,nodev,noexec,relatime"
},{
"target": "/dev/pts",
"source": "devpts",
"fstype": "devpts",
"options": "rw,nosuid,noexec,relatime,gid=165541,mode=620,ptmxmode=666"
},{
"target": "/dev/shm",
"source": "shm",
"fstype": "tmpfs",
"options": "rw,nosuid,nodev,noexec,relatime,size=65536k,uid=165536,gid=165536,inode64"
},{
"target": "/dev/null",
"source": "udev[/null]",
"fstype": "devtmpfs",
"options": "rw,nosuid,relatime,size=3996644k,nr_inodes=999161,mode=755,inode64"
},{
"target": "/dev/random",
"source": "udev[/random]",
"fstype": "devtmpfs",
"options": "rw,nosuid,relatime,size=3996644k,nr_inodes=999161,mode=755,inode64"
},{
"target": "/dev/kmsg",
"source": "udev[/null]",
"fstype": "devtmpfs",
"options": "rw,nosuid,relatime,size=3996644k,nr_inodes=999161,mode=755,inode64"
},{
"target": "/dev/full",
"source": "udev[/full]",
"fstype": "devtmpfs",
"options": "rw,nosuid,relatime,size=3996644k,nr_inodes=999161,mode=755,inode64"
},{
"target": "/dev/tty",
"source": "udev[/tty]",
"fstype": "devtmpfs",
"options": "rw,nosuid,relatime,size=3996644k,nr_inodes=999161,mode=755,inode64"
},{
"target": "/dev/zero",
"source": "udev[/zero]",
"fstype": "devtmpfs",
"options": "rw,nosuid,relatime,size=3996644k,nr_inodes=999161,mode=755,inode64"
},{
"target": "/dev/urandom",
"source": "udev[/urandom]",
"fstype": "devtmpfs",
"options": "rw,nosuid,relatime,size=3996644k,nr_inodes=999161,mode=755,inode64"
}
]
},{
"target": "/run",
"source": "tmpfs",
"fstype": "tmpfs",
"options": "rw,nosuid,nodev,relatime,size=65536k,mode=755,uid=165536,gid=165536,inode64",
"children": [
{
"target": "/run/lock",
"source": "tmpfs",
"fstype": "tmpfs",
"options": "rw,nosuid,nodev,noexec,relatime,size=4096k,uid=165536,gid=165536,inode64"
},{
"target": "/run/docker/netns/5aee14bec16d",
"source": "nsfs[net:[4026532823]]",
"fstype": "nsfs",
"options": "rw"
},{
"target": "/run/docker/netns/9b8c45925f2a",
"source": "nsfs[net:[4026532865]]",
"fstype": "nsfs",
"options": "rw"
},{
"target": "/run/docker/netns/0379afe1348b",
"source": "nsfs[net:[4026532982]]",
"fstype": "nsfs",
"options": "rw"
},{
"target": "/run/docker/netns/3d1e73bef29a",
"source": "nsfs[net:[4026533236]]",
"fstype": "nsfs",
"options": "rw"
},{
"target": "/run/docker/netns/7996ac6b4713",
"source": "nsfs[net:[4026532766]]",
"fstype": "nsfs",
"options": "rw"
}
]
},{
"target": "/var/lib/docker",
"source": "/dev/mapper/vld-lvvld[/volumes/varlibdocker-hmapp03-v1/_data]",
"fstype": "ext4",
"options": "rw,relatime",
"children": [
{
"target": "/var/lib/docker/overlay2/794da6b1730f55ec9f529b21bcac141bc5e084873c921cbfd216b65e9c79ef5a/merged",
"source": "overlay",
"fstype": "overlay",
"options": "rw,relatime,lowerdir=/var/lib/docker/overlay2/l/IJGECUQDZZT4B7SAZPN3W5XF57:/var/lib/docker/overlay2/l/242FAZFGRCG5GEL4PI5PSBPPN5:/var/lib/docker/overlay2/l/6E3UMHBHAICZUSIB2K6GPEASGF:/var/lib/docker/overlay2/l/KVIZ37QFA3U32ZOX7NB6664UKF:/var/lib/docker/overlay2/l/SKQZE26LMQUYPOQBRBHZRMVTJY:/var/lib/docker/overlay2/l/3ORJU3I563L3HVMHAODIYSXWV7:/var/lib/docker/overlay2/l/NO5VHYUJX7B4DCOW4MK6XSBZFL:/var/lib/docker/overlay2/l/MDYRSYD6VNSGKIJTKO6OVF3SOL:/var/lib/docker/overlay2/l/O3QRDM6V7YN7TEG37FRFCOIP5D:/var/lib/docker/overlay2/l/DX52OBUHLV5N4QXFLVNPJO5SMV:/var/lib/docker/overlay2/l/3ZZMO7XU6UJESPUIYZI2WDU5W3:/var/lib/docker/overlay2/l/KTAVS67FJALBGNFQTSN4HFKVRJ:/var/lib/docker/overlay2/l/SCDBG2QVH75IQET75ADJJUQ7Y5:/var/lib/docker/overlay2/l/V4VEZI5ANZOEPDR57VM52JEJWA:/var/lib/docker/overlay2/l/ZVFK4KDKTQRMJIZBFJHB3CKZRK,upperdir=/var/lib/docker/overlay2/794da6b1730f55ec9f529b21bcac141bc5e084873c921cbfd216b65e9c79ef5a/diff,workdir=/var/lib/docker/overlay2/794da6b1730f55ec9f529b21bcac141bc5e084873c921cbfd216b65e9c79ef5a/work,redirect_dir=nofollow,userxattr"
},{
"target": "/var/lib/docker/overlay2/2d7613e2f6be28ab4ed42850a587f8c0d70ecc0dd51444e16265bf56ca361739/merged",
"source": "overlay",
"fstype": "overlay",
"options": "rw,relatime,lowerdir=/var/lib/docker/overlay2/l/2IGYB2PAURIM46PDXEMMSOQDTE:/var/lib/docker/overlay2/l/QCEQNQWAJBSFXITCBNPYXHDW4O:/var/lib/docker/overlay2/l/32WN5ZDZWIVELHFYZPP2QJM7Q3:/var/lib/docker/overlay2/l/N2ST2NSFJ7L5SU7ZBJR4HODJIY:/var/lib/docker/overlay2/l/FVARIARVXMQRBJBTSFL56FQ53H:/var/lib/docker/overlay2/l/NFBR7YWBXIPDOGCLSXGOXTJGFM:/var/lib/docker/overlay2/l/H3LH7DM7B32POFRQRVNKWZFFJB:/var/lib/docker/overlay2/l/XJF5XW6JMEHKIJI7NTFFCJYPUO:/var/lib/docker/overlay2/l/VO76EUOKXH5NOQDFHQJPFXXIXN:/var/lib/docker/overlay2/l/GE4P3FMSVD2SSAILR2DCLRORNL:/var/lib/docker/overlay2/l/3ZZMO7XU6UJESPUIYZI2WDU5W3:/var/lib/docker/overlay2/l/KTAVS67FJALBGNFQTSN4HFKVRJ:/var/lib/docker/overlay2/l/SCDBG2QVH75IQET75ADJJUQ7Y5:/var/lib/docker/overlay2/l/V4VEZI5ANZOEPDR57VM52JEJWA:/var/lib/docker/overlay2/l/ZVFK4KDKTQRMJIZBFJHB3CKZRK,upperdir=/var/lib/docker/overlay2/2d7613e2f6be28ab4ed42850a587f8c0d70ecc0dd51444e16265bf56ca361739/diff,workdir=/var/lib/docker/overlay2/2d7613e2f6be28ab4ed42850a587f8c0d70ecc0dd51444e16265bf56ca361739/work,redirect_dir=nofollow,userxattr"
},{
"target": "/var/lib/docker/overlay2/325a7f765a5afbfeaf78fc926eb9bd19dba57d7368f6c500cc7c5a5492be9c17/merged",
"source": "overlay",
"fstype": "overlay",
"options": "rw,relatime,lowerdir=/var/lib/docker/overlay2/l/LKCRDQGZOJVHXXYH67WDNORUEF:/var/lib/docker/overlay2/l/FJSIHV5ATS7P6OW3PF6K6OBVXR:/var/lib/docker/overlay2/l/GCX24ICVZTEC4MHXCOC5RG3JVR:/var/lib/docker/overlay2/l/T5JXXYAXSJYAG324V2HPPWWXGZ:/var/lib/docker/overlay2/l/4H27II67CLWEENJRMQA2C2YANQ:/var/lib/docker/overlay2/l/HWV5TOLZY7EJALI4SZPZSSXCBZ:/var/lib/docker/overlay2/l/JXZBLYGCEZ2VIWJWWIQWABPJ63:/var/lib/docker/overlay2/l/42ZBEEUDNWCHKNB2YCEABZN5BT:/var/lib/docker/overlay2/l/C2ZACQXFHC3Z75FQKEFRLP5TIA:/var/lib/docker/overlay2/l/5HSBWXDIS37PWGRJH7HSXXFVOD:/var/lib/docker/overlay2/l/MADUBJJRSJ6EACBNCEYVXYECRI:/var/lib/docker/overlay2/l/YMSFVFZWO24RX4JFBURYVH2HL6:/var/lib/docker/overlay2/l/BFBYJEBZ5WYDIJ6EB5JAIZISY6:/var/lib/docker/overlay2/l/SEH7KXYAZNLG5DMA3MO55ST2ZT:/var/lib/docker/overlay2/l/ZA7IM4U45WBYDZBXXC3WWHXP4P,upperdir=/var/lib/docker/overlay2/325a7f765a5afbfeaf78fc926eb9bd19dba57d7368f6c500cc7c5a5492be9c17/diff,workdir=/var/lib/docker/overlay2/325a7f765a5afbfeaf78fc926eb9bd19dba57d7368f6c500cc7c5a5492be9c17/work,redirect_dir=nofollow,userxattr"
},{
"target": "/var/lib/docker/overlay2/06ed7e3c8b72c21557fc1b0de430d03ef5f5dec0ddf96baa160fc50fe6dcb08b/merged",
"source": "overlay",
"fstype": "overlay",
"options": "rw,relatime,lowerdir=/var/lib/docker/overlay2/l/SWQBQKDUOHWKBHKZFI53TVXFHF:/var/lib/docker/overlay2/l/JMLUD7ZMFMEC6GB7W2SL6ZGBPM:/var/lib/docker/overlay2/l/QLOX7MTGZVLC3WCVRM2O65WE6T:/var/lib/docker/overlay2/l/QXBMMDMDAXDHPXCR6Q5IAZCACC:/var/lib/docker/overlay2/l/RMEA2WRCJB5BFMUMV65I6FP7D5:/var/lib/docker/overlay2/l/GI65S43RQ7OQ34S4XT3YK6DH5T:/var/lib/docker/overlay2/l/J32RM357H3JNJU3PQATZRILPB3:/var/lib/docker/overlay2/l/F3DZTXATKSVZRDU7TOE25SAOAV:/var/lib/docker/overlay2/l/ITOLGDS3JIP7DDRCLC43W6DLOT:/var/lib/docker/overlay2/l/JMVQZOGQAZYBAAWOYRXIISX4V6:/var/lib/docker/overlay2/l/TXRBEOBOSX2UZHPASF3IPLOCVX,upperdir=/var/lib/docker/overlay2/06ed7e3c8b72c21557fc1b0de430d03ef5f5dec0ddf96baa160fc50fe6dcb08b/diff,workdir=/var/lib/docker/overlay2/06ed7e3c8b72c21557fc1b0de430d03ef5f5dec0ddf96baa160fc50fe6dcb08b/work,redirect_dir=nofollow,userxattr"
},{
"target": "/var/lib/docker/overlay2/c89d9a69088e05f433c3fa272634e0b22ae9ee3adc3d214e65ad3b7f4c165dfe/merged",
"source": "overlay",
"fstype": "overlay",
"options": "rw,relatime,lowerdir=/var/lib/docker/overlay2/l/KROVCC5UCY3GDH7PJZBKYKS2JH:/var/lib/docker/overlay2/l/MFYO2BVZIEZOUJWWTECHGXOVBN:/var/lib/docker/overlay2/l/AFNZU4MF5LDLQOA7QVCZAMUKCA:/var/lib/docker/overlay2/l/FVZ5IRPEYTB4H7EX23LRFJDPAJ:/var/lib/docker/overlay2/l/5O2OYMZUGMUY2VFEOH25MI6JVF:/var/lib/docker/overlay2/l/3EJQUHWAUWLLVCQNIWARMQ6ECT:/var/lib/docker/overlay2/l/TATTM3IMPFQ7V4HEZRI53B3U3W:/var/lib/docker/overlay2/l/XQOD42Y55SS2PWY5QZ357H67LL:/var/lib/docker/overlay2/l/IR3QP77BLA7BIIAUBQZCEFVVE7:/var/lib/docker/overlay2/l/MADUBJJRSJ6EACBNCEYVXYECRI:/var/lib/docker/overlay2/l/YMSFVFZWO24RX4JFBURYVH2HL6:/var/lib/docker/overlay2/l/BFBYJEBZ5WYDIJ6EB5JAIZISY6:/var/lib/docker/overlay2/l/SEH7KXYAZNLG5DMA3MO55ST2ZT:/var/lib/docker/overlay2/l/ZA7IM4U45WBYDZBXXC3WWHXP4P,upperdir=/var/lib/docker/overlay2/c89d9a69088e05f433c3fa272634e0b22ae9ee3adc3d214e65ad3b7f4c165dfe/diff,workdir=/var/lib/docker/overlay2/c89d9a69088e05f433c3fa272634e0b22ae9ee3adc3d214e65ad3b7f4c165dfe/work,redirect_dir=nofollow,userxattr"
}
]
},{
"target": "/etc/resolv.conf",
"source": "/dev/mapper/vld-lvvld[/containers/6ac160a4b546bfcca6d326402adeba578174fe0cb1391575f927bc4c257a15fa/resolv.conf]",
"fstype": "ext4",
"options": "rw,relatime,idmapped"
},{
"target": "/etc/hostname",
"source": "/dev/mapper/vld-lvvld[/containers/6ac160a4b546bfcca6d326402adeba578174fe0cb1391575f927bc4c257a15fa/hostname]",
"fstype": "ext4",
"options": "rw,relatime,idmapped"
},{
"target": "/etc/hosts",
"source": "/dev/mapper/vld-lvvld[/containers/6ac160a4b546bfcca6d326402adeba578174fe0cb1391575f927bc4c257a15fa/hosts]",
"fstype": "ext4",
"options": "rw,relatime,idmapped"
},{
"target": "/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs",
"source": "/dev/mapper/ubuntu--vg-ubuntu--lv[/var/lib/sysbox/containerd/6ac160a4b546bfcca6d326402adeba578174fe0cb1391575f927bc4c257a15fa]",
"fstype": "ext4",
"options": "rw,relatime,idmapped"
},{
"target": "/var/lib/rancher/k3s",
"source": "/dev/mapper/ubuntu--vg-ubuntu--lv[/var/lib/sysbox/rancher-k3s/6ac160a4b546bfcca6d326402adeba578174fe0cb1391575f927bc4c257a15fa]",
"fstype": "ext4",
"options": "rw,relatime,idmapped"
},{
"target": "/var/lib/rancher/rke2",
"source": "/dev/mapper/ubuntu--vg-ubuntu--lv[/var/lib/sysbox/rancher-rke2/6ac160a4b546bfcca6d326402adeba578174fe0cb1391575f927bc4c257a15fa]",
"fstype": "ext4",
"options": "rw,relatime,idmapped"
},{
"target": "/var/lib/kubelet",
"source": "/dev/mapper/ubuntu--vg-ubuntu--lv[/var/lib/sysbox/kubelet/6ac160a4b546bfcca6d326402adeba578174fe0cb1391575f927bc4c257a15fa]",
"fstype": "ext4",
"options": "rw,relatime,idmapped"
},{
"target": "/var/lib/k0s",
"source": "/dev/mapper/ubuntu--vg-ubuntu--lv[/var/lib/sysbox/k0s/6ac160a4b546bfcca6d326402adeba578174fe0cb1391575f927bc4c257a15fa]",
"fstype": "ext4",
"options": "rw,relatime,idmapped"
},{
"target": "/var/lib/buildkit",
"source": "/dev/mapper/ubuntu--vg-ubuntu--lv[/var/lib/sysbox/buildkit/6ac160a4b546bfcca6d326402adeba578174fe0cb1391575f927bc4c257a15fa]",
"fstype": "ext4",
"options": "rw,relatime,idmapped"
},{
"target": "/usr/src",
"source": "/dev/mapper/ubuntu--vg-ubuntu--lv[/usr/src]",
"fstype": "ext4",
"options": "ro,relatime,idmapped",
"children": [
{
"target": "/usr/src/linux-headers-6.5.0-41-generic",
"source": "/dev/mapper/ubuntu--vg-ubuntu--lv[/usr/src/linux-headers-6.5.0-41-generic]",
"fstype": "ext4",
"options": "ro,relatime,idmapped"
}
]
},{
"target": "/usr/lib/modules/6.5.0-41-generic",
"source": "/dev/mapper/ubuntu--vg-ubuntu--lv[/usr/lib/modules/6.5.0-41-generic]",
"fstype": "ext4",
"options": "ro,relatime,idmapped"
}
]
}
]
}
Here is the nobody:nogroup problem
# docker exec -it hmapp03 ls -la /etc
total 852
drwxr-xr-x 1 root root 4096 Jun 12 23:44 .
drwxr-xr-x 1 root root 4096 Jun 25 21:00 ..
-rw------- 1 root root 0 Oct 4 2023 .pwd.lock
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 17:32 X11
-rw-r--r-- 1 root root 3028 Oct 4 2023 adduser.conf
drwxr-xr-x 1 root root 4096 Jun 13 00:13 alternatives
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 17:33 apparmor
drwxr-xr-x 1 root root 4096 May 28 18:47 apparmor.d
drwxr-xr-x 1 root root 4096 Oct 4 2023 apt
-rw-r--r-- 1 root root 2319 Jan 6 2022 bash.bashrc
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 18:21 bash_completion.d
-rw-r--r-- 1 root root 367 Dec 16 2020 bindresvport.blacklist
drwxr-xr-x 1 nobody nogroup 4096 Sep 19 2023 binfmt.d
drwxr-xr-x 1 nobody nogroup 4096 Nov 30 2023 ca-certificates
-rw-r--r-- 1 root root 5892 Nov 30 2023 ca-certificates.conf
drwxr-xr-x 1 nobody nogroup 4096 Oct 4 2023 cloud
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 17:33 containerd
drwxr-xr-x 1 root root 4096 Jan 7 17:33 cron.d
drwxr-xr-x 1 root root 4096 Jan 7 17:33 cron.daily
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 17:33 cron.hourly
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 17:33 cron.monthly
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 17:33 cron.weekly
-rw-r--r-- 1 root root 1136 Mar 23 2022 crontab
drwxr-xr-x 1 nobody nogroup 4096 Nov 30 2023 dbus-1
-rw-r--r-- 1 root root 2969 Feb 20 2022 debconf.conf
-rw-r--r-- 1 root root 13 Aug 22 2021 debian_version
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 18:21 default
-rw-r--r-- 1 root root 604 Sep 15 2018 deluser.conf
drwxr-xr-x 1 nobody nogroup 4096 Nov 30 2023 depmod.d
drwxr-xr-x 1 root root 4096 Jan 7 18:21 docker
drwxr-xr-x 1 nobody nogroup 4096 Oct 4 2023 dpkg
-rw-r--r-- 1 root root 685 Jan 8 2022 e2scrub.conf
-rw-r--r-- 1 root root 106 Oct 4 2023 environment
-rw-r--r-- 1 root root 1816 Dec 27 2019 ethertypes
-rw-r--r-- 1 root root 37 Oct 4 2023 fstab
-rw-r--r-- 1 root root 2584 Feb 3 2022 gai.conf
-rw-r--r-- 1 root root 903 Jun 11 13:33 group
-rw-r--r-- 1 root root 889 Jun 11 13:33 group-
-rw-r----- 1 root shadow 756 Jun 11 13:33 gshadow
-rw-r----- 1 root shadow 742 Jun 11 13:33 gshadow-
drwxr-xr-x 1 nobody nogroup 4096 Feb 21 2022 gss
-rw-r--r-- 1 root root 92 Oct 15 2021 host.conf
-rw-r--r-- 1 root root 8 Jul 2 11:24 hostname
-rw-r--r-- 1 root root 171 Jul 2 11:24 hosts
-rw-r--r-- 1 nobody nogroup 411 Jan 7 18:21 hosts.allow
-rw-r--r-- 1 nobody nogroup 711 Jan 7 18:21 hosts.deny
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 17:33 init
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 18:21 init.d
-rw-r--r-- 1 root root 1748 Jan 6 2022 inputrc
drwxr-xr-x 1 nobody nogroup 4096 Nov 30 2023 iproute2
-rw-r--r-- 1 root root 26 Aug 2 2023 issue
-rw-r--r-- 1 root root 19 Aug 2 2023 issue.net
drwxr-xr-x 1 root root 4096 Nov 30 2023 kernel
-rw-r--r-- 1 root root 10907 May 28 18:47 ld.so.cache
-rw-r--r-- 1 root root 34 Dec 16 2020 ld.so.conf
drwxr-xr-x 1 nobody nogroup 4096 Oct 4 2023 ld.so.conf.d
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 17:32 ldap
-rw-r--r-- 1 root root 267 Oct 15 2021 legal
-rw-r--r-- 1 root root 191 Mar 17 2022 libaudit.conf
-rw-r--r-- 1 root root 2996 Sep 25 2023 locale.alias
-rw-r--r-- 1 root root 9458 Nov 30 2023 locale.gen
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 17:32 logcheck
-rw-r--r-- 1 root root 10734 Nov 11 2021 login.defs
-rw-r--r-- 1 root root 592 May 25 2022 logrotate.conf
drwxr-xr-x 1 root root 4096 May 17 20:43 logrotate.d
-rw-r--r-- 1 root root 104 Aug 2 2023 lsb-release
-rw-r--r-- 1 root root 33 Nov 30 2023 machine-id
-rw-r--r-- 1 root root 72029 Mar 21 2022 mime.types
-rw-r--r-- 1 root root 744 Jan 8 2022 mke2fs.conf
drwxr-xr-x 1 nobody nogroup 4096 Nov 30 2023 modprobe.d
-rw-r--r-- 1 root root 195 Nov 30 2023 modules
drwxr-xr-x 1 nobody nogroup 4096 Nov 30 2023 modules-load.d
lrwxrwxrwx 1 root root 19 Jun 12 23:44 mtab -> ../proc/self/mounts
-rw-r--r-- 1 root root 767 Mar 24 2022 netconfig
-rw-r--r-- 1 root root 91 Oct 15 2021 networks
-rw-r--r-- 1 root root 494 Dec 16 2020 nsswitch.conf
drwxr-xr-x 1 nobody nogroup 4096 Oct 4 2023 opt
lrwxrwxrwx 1 root root 21 Aug 2 2023 os-release -> ../usr/lib/os-release
-rw-r--r-- 1 root root 552 Aug 12 2020 pam.conf
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 18:21 pam.d
-rw-r--r-- 1 root root 1738 Jun 11 13:33 passwd
-rw-r--r-- 1 root root 1735 Jun 11 13:33 passwd-
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 17:33 perl
-rw-r--r-- 1 root root 582 Oct 15 2021 profile
drwxr-xr-x 1 nobody nogroup 4096 Oct 4 2023 profile.d
-rw-r--r-- 1 root root 2932 Apr 1 2013 protocols
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 17:33 python3.10
drwxr-xr-x 1 root root 4096 Jan 7 17:33 rc0.d
drwxr-xr-x 1 root root 4096 Jan 7 17:33 rc1.d
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 18:21 rc2.d
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 18:21 rc3.d
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 18:21 rc4.d
drwxr-xr-x 1 nobody nogroup 4096 Jan 7 18:21 rc5.d
drwxr-xr-x 1 root root 4096 Jan 7 17:33 rc6.d
drwxr-xr-x 1 root root 4096 Jan 7 17:33 rcS.d
-rw-r--r-- 1 root root 55 Jul 2 11:24 resolv.conf
Trying to change ownership:
# docker exec -it hmapp03 chown -v root:root /etc/gss
chown: changing ownership of '/etc/gss': Operation not permitted
