k8s-infra
k8s-infra copied to clipboard
Published
20 hours ago •
snowdrop
snowdrop
Implement podman rootless and rootful
Implement podman as a provider both rootless as well as rootful. For rootless the installation fails for now as connecting the cluster to the container registry using the podman network fails with the following error.
Error: "slirp4netns" is not supported: invalid network mode
Also added a filter so that these validations are only triggered when something changes under the kind/ folder.
The following jobs are disabled, for now, because they don't work:
kind-test-podman-rootless-linux: due to errorError validating CNI config file /home/runner/.config/cni/net.d/kind.conflistkind-test-podman-rootless-macos: because the job freezeskind-test-podman-rootful-macos: because the job freezes
don't forget to also change the registry.sh script as sudo is also asked ?
./kind/registry.sh install --provider podman --registry-name kind-registry.local
Welcome to our
_____ _
/ ____| | |
| (___ _ __ ___ __ __ __| | _ __ ___ _ __
\___ \ | '_ \ / _ \ \ \ /\ / / / _ | | __| / _ \ | \ _ \
____) | | | | | | (_) | \ V V / | (_| | | | | (_) | | |_) |
|_____/ |_| |_| \___/ \_/\_/ \__,_| |_| \___/ | __/
| |
|_|
Script to create/delete a container registry (secure or insecure)
✔ Pre requisites check passed!
Password:
don't forget to also change the registry.sh script as sudo is also asked ?
./kind/registry.sh install --provider podman --registry-name kind-registry.local Welcome to our _____ _ / ____| | | | (___ _ __ ___ __ __ __| | _ __ ___ _ __ \___ \ | '_ \ / _ \ \ \ /\ / / / _ | | __| / _ \ | \ _ \ ____) | | | | | | (_) | \ V V / | (_| | | | | (_) | | |_) | |_____/ |_| |_| \___/ \_/\_/ \__,_| |_| \___/ | __/ | | |_| Script to create/delete a container registry (secure or insecure) ✔ Pre requisites check passed! Password:
The same --rootless option is available on the registry deployment.
Both scenario work on Macos BUT I still don't know if the pod of the kind container is running as "root" or"rootless" !!
## Rootfull
podman machine stop
podman machine rm -f
podman machine init
podman machine set --rootful=true
podman machine start
podman ps
./kind/kind.sh install --provider podman --rootless
cat <<EOF | k apply -f -
apiVersion: v1
kind: Pod
metadata:
name: security-context-demo
spec:
securityContext:
runAsUser: 0
volumes:
- name: sec-ctx-vol
emptyDir: {}
containers:
- name: sec-ctx-demo
image: busybox:1.28
command: [ "sh", "-c", "sleep 1h" ]
volumeMounts:
- name: sec-ctx-vol
mountPath: /data/demo
securityContext:
allowPrivilegeEscalation: false
EOF
## Rootless
podman machine stop
podman machine rm -f
podman machine init
podman machine set --rootful=false
podman machine start
podman ps
./kind/kind.sh install --provider podman --rootless
cat <<EOF | k apply -f -
apiVersion: v1
kind: Pod
metadata:
name: security-context-demo
spec:
securityContext:
runAsUser: 0
volumes:
- name: sec-ctx-vol
emptyDir: {}
containers:
- name: sec-ctx-demo
image: busybox:1.28
command: [ "sh", "-c", "sleep 1h" ]
volumeMounts:
- name: sec-ctx-vol
mountPath: /data/demo
securityContext:
allowPrivilegeEscalation: false
EOF
I deployed another pod on the kind cluster running on podman "rootless" but pod has been started as root =>
// see: https://www.golinuxcloud.com/kubernetes-privileged-pod-examples/#Example-1_Create_Kubernetes_Privileged_Pod_With_all_Capabilities
kubectl exec -it privileged-pod -- bash
[root@privileged-pod /]# capsh --print
Current: = cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,cap_audit_read,38,39,40+ep
Bounding set =cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,cap_audit_read,38,39,40
Ambient set =
Securebits: 00/0x0/1'b0
secure-noroot: no (unlocked)
secure-no-suid-fixup: no (unlocked)
secure-keep-caps: no (unlocked)
secure-no-ambient-raise: no (unlocked)
uid=0(root)
gid=0(root)
groups=0(root)
[root@privileged-pod /]#
Why ?
Both scenario work on Macos BUT I still don't know if the pod of the kind container is running as "root" or"rootless" !!
## Rootfull podman machine stop podman machine rm -f podman machine init podman machine set --rootful=true podman machine start podman ps ./kind/kind.sh install --provider podman --rootless cat <<EOF | k apply -f - apiVersion: v1 kind: Pod metadata: name: security-context-demo spec: securityContext: runAsUser: 0 volumes: - name: sec-ctx-vol emptyDir: {} containers: - name: sec-ctx-demo image: busybox:1.28 command: [ "sh", "-c", "sleep 1h" ] volumeMounts: - name: sec-ctx-vol mountPath: /data/demo securityContext: allowPrivilegeEscalation: false EOF ## Rootless podman machine stop podman machine rm -f podman machine init podman machine set --rootful=false podman machine start podman ps ./kind/kind.sh install --provider podman --rootless cat <<EOF | k apply -f - apiVersion: v1 kind: Pod metadata: name: security-context-demo spec: securityContext: runAsUser: 0 volumes: - name: sec-ctx-vol emptyDir: {} containers: - name: sec-ctx-demo image: busybox:1.28 command: [ "sh", "-c", "sleep 1h" ] volumeMounts: - name: sec-ctx-vol mountPath: /data/demo securityContext: allowPrivilegeEscalation: false EOFI deployed another pod on the kind cluster running on podman "rootless" but pod has been started as root =>
// see: https://www.golinuxcloud.com/kubernetes-privileged-pod-examples/#Example-1_Create_Kubernetes_Privileged_Pod_With_all_Capabilities kubectl exec -it privileged-pod -- bash [root@privileged-pod /]# capsh --print Current: = cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,cap_audit_read,38,39,40+ep Bounding set =cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,cap_audit_read,38,39,40 Ambient set = Securebits: 00/0x0/1'b0 secure-noroot: no (unlocked) secure-no-suid-fixup: no (unlocked) secure-keep-caps: no (unlocked) secure-no-ambient-raise: no (unlocked) uid=0(root) gid=0(root) groups=0(root) [root@privileged-pod /]#Why ?
I don't know. That's must be related to the way the podman machine works and I don't have a MAC to test or dig into it.