buildkitd pod is failing on kubernetes cluster having PSP even after allowing unmasked ProcMounTypes

[asmi10]

Hello!

I tried to deploy buildkitd in kubernetes cluster having below PSP applied, but it failed with error: [rootlesskit:parent] error: failed to setup UID/GID map: newuidmap 10 [0 1000 1 1 100000 65536] failed: newuidmap: Could not set caps : exit status 1 If I allow privilege escalation in PSP then it works fine.

spec: allowPrivilegeEscalation: false allowedCapabilities:

• '*' allowedProcMountTypes:
• Unmasked fsGroup: ranges:
  • max: 65535 min: 1 rule: MustRunAs hostIPC: true hostPID: true hostPorts:
• max: 65535 min: 0 runAsUser: rule: RunAsAny seLinux: rule: RunAsAny supplementalGroups: rule: RunAsAny volumes:
• '*'

[nickhs]

I'm having this issue as well - a few notes:

• newuidmap has the setuid bit but allowPrivilegeEscalation: false prevents that from happening. Even if filecaps were used the same issue would happen
• It's not clear to me why newuidmap needs setuid? I can do unshare -r (which writes to /proc/pid/uid_map) without needing CAP_SETUID.
• newuidmap calls capset here but again its not totally clear to me why it needs to do so?
• By allowing priv escalation doesn't that imply the container can become root if run from an untrusted image?

[AkihiroSuda]

It's not clear to me why newuidmap needs setuid? I can do unshare -r (which writes to /proc/pid/uid_map) without needing CAP_SETUID.

For writing multiple entries to uid_map, newuidmap needs setuid

[candrews]

Is there any way to eliminate the use of newuidmap/newgidmap?

I need to be able to run buildkit in a context with allowPrivilegeEscalation: false.

I came across https://unix.stackexchange.com/questions/759533/what-causes-newuidmap-to-be-dissallowed-in-new-user-namespace - perhaps it provides a way forward?

[AkihiroSuda]

Is there any way to eliminate the use of newuidmap/newgidmap?

I need to be able to run buildkit in a context with allowPrivilegeEscalation: false.

Conceptually yes, by using UML as in https://github.com/weber-software/diuid

Or just use seccomp_unotify to hook UID-relevant syscalls (POC https://github.com/rootless-containers/subuidless)

[candrews]

Those projects are unmaintained - is there a maintained, actively used solution available?