ceph
run ceph-csi as non-root and configurable UID/GID
Describe the feature you'd like to have
Run ceph-csi as non-root and with configurable UID/GID.
What is the value to the end user? (why is it a priority?)
In an enterprise environment, running containers as root is a security concern. For example, Ceph daemons support running as non-root with configurable UID/GID https://docs.ceph.com/en/pacific/man/8/ceph/?highlight=setuser#cmdoption-ceph-setuser
--setuser user
will apply the appropriate user ownership to the file specified by the option ‘-o’.
--setgroup group
will apply the appropriate group ownership to the file specified by the option ‘-o’.
If ceph csi would support running as non-root with configurable UID/GID, the solution would be complete.
How will we know we have a good solution? (acceptance criteria)
Check OS process owner. Example for ceph-mgr
$ ps aux
167 244582 1.3 2.3 1547924 387256 ? Ssl Sep15 23:30 ceph-mgr --fsid=4b8c33bf-eaa1-45fa-a350-c40b17c04e2a --keyring=/etc/ceph/keyring-store/keyring --log-to-stderr=true --err-to-stderr=true --mon-cluster-log-to-stderr=true --log-stderr-prefix=debug --default-log-to-file=false --default-mon-cluster-log-to-file=false --mon-host=[v2:10.10.10.7:3300,v1:10.10.10.7:6789],[v2:10.10.10.172:3300,v1:10.10.10.172:6789],[v2:10.10.10.112:3300,v1:10.10.10.112:6789] --mon-initial-members=a,b,c --id=a --setuser=ceph --setgroup=ceph --client-mount-uid=0 --client-mount-gid=0 --foreground
Additional context
This feature was mentioned in https://github.com/ceph/ceph-csi/issues/714, but the ticket was closed due to inactivity.
@ceph/ceph-csi-contributors any volunteers for analysis/PR to check is it possible especially for daemonset pods?
Mounting will still require CAP_SYSADMIN, so the DaemonSet Pods can not drop a lot of privileges. The provisioner and other components should not need to run as root.
To add to above statement we cannot run the daemonset plugin pod with normal users as we need to create/delete the csi socket on the hostPath and the hostPath is created by kubelet with root user and if we try to run the daemonset plugin pod with the normal user the plugin pod will never come up
$ ls -lrt
total 36
drwxr-xr-x 2 root root 4096 Sep 28 10:34 pki
-rw------- 1 root root 62 Sep 28 10:34 cpu_manager_state
-rw-r--r-- 1 root root 189 Sep 28 10:35 kubeadm-flags.env
-rw-r--r-- 1 root root 944 Sep 28 10:35 config.yaml
drwxr-x--- 2 root root 4096 Sep 28 10:35 pod-resources
drwxr-x--- 2 root root 4096 Sep 29 07:56 plugins_registry
drwxr-x--- 5 root root 4096 Sep 29 12:34 plugins
drwxr-xr-x 2 root root 4096 Sep 29 12:51 device-plugins
drwxr-x--- 24 root root 4096 Sep 29 12:51 pods
drwxr-xr-x 2 root root 4096 Sep 29 12:34 rook-ceph.rbd.csi.ceph.com
drwxr-xr-x 2 root root 4096 Sep 29 12:34 rook-ceph.cephfs.csi.ceph.com
E0929 12:57:17.589273 1 node_register.go:42] failed to stat the socket /registration/rook-ceph.cephfs.csi.ceph.com-reg.sock with error: stat /registration/rook-ceph.cephfs.csi.ceph.com-reg.sock: permission denied
F0929 13:02:07.734019 1 server.go:130] Failed to listen: listen unix //csi/csi.sock: bind: permission denied
the user &{Uid:0 Gid:0 Username:root Name:root HomeDir:/root} and error <nil>
the group &{Gid:0 Name:root} and error <nil>
F0929 13:03:23.706695 1 server.go:124] Failed to remove //csi/csi.sock, error: remove //csi/csi.sock: permission denied
To add to above statement we cannot run the daemonset plugin pod with normal users as we need to create/delete the csi socket on the hostPath and the hostPath is created by kubelet with root user and if we try to run the daemonset plugin pod with the normal user the plugin pod will never come up
$ ls -lrt total 36 drwxr-xr-x 2 root root 4096 Sep 28 10:34 pki -rw------- 1 root root 62 Sep 28 10:34 cpu_manager_state -rw-r--r-- 1 root root 189 Sep 28 10:35 kubeadm-flags.env -rw-r--r-- 1 root root 944 Sep 28 10:35 config.yaml drwxr-x--- 2 root root 4096 Sep 28 10:35 pod-resources drwxr-x--- 2 root root 4096 Sep 29 07:56 plugins_registry drwxr-x--- 5 root root 4096 Sep 29 12:34 plugins drwxr-xr-x 2 root root 4096 Sep 29 12:51 device-plugins drwxr-x--- 24 root root 4096 Sep 29 12:51 pods drwxr-xr-x 2 root root 4096 Sep 29 12:34 rook-ceph.rbd.csi.ceph.com drwxr-xr-x 2 root root 4096 Sep 29 12:34 rook-ceph.cephfs.csi.ceph.com E0929 12:57:17.589273 1 node_register.go:42] failed to stat the socket /registration/rook-ceph.cephfs.csi.ceph.com-reg.sock with error: stat /registration/rook-ceph.cephfs.csi.ceph.com-reg.sock: permission denied F0929 13:02:07.734019 1 server.go:130] Failed to listen: listen unix //csi/csi.sock: bind: permission denied the user &{Uid:0 Gid:0 Username:root Name:root HomeDir:/root} and error <nil> the group &{Gid:0 Name:root} and error <nil> F0929 13:03:23.706695 1 server.go:124] Failed to remove //csi/csi.sock, error: remove //csi/csi.sock: permission denied
How about using an init container to chown the directory first and then run under the 167 UID?
