Tighten permissions (security best practices)

[akostic-kostile]

I've been going through security review of our Kubernetes infrastructure, using kubescape specifically, and there are couple of valid findings that I think can be fixed. It's clear why we need hostPID: true and privileged: true, and why we're running the container as root, but a few others are left as Kubernetes defaults and are too lax.

I'm not 100% sure how exactly Kured operates, but to me it looks like all it does is call /bin/systemctl reboot by accessing host's mount namespace directly. Thus I suspect that this DaemonSet would work equally as well, while being much tighter on security.

apiVersion: apps/v1
kind: DaemonSet
spec:
  template:
    spec:
      containers:
      - name: kured
        securityContext:
          allowPrivilegeEscalation: false # not 100% sure how privilege escalation works, and if we need it or not in this case
          capabilities:
            add:
            - SYS_BOOT
            drop:
            - ALL
          privileged: true
          readOnlyRootFilesystem: true
      hostPID: true
      securityContext:
        seccompProfile:
          type: RuntimeDefault

Thoughts?

[ckotzbauer]

If I'm not completely wrong (maybe someone can confirm) it is not possible to run a container with allowPrivilegeEscalation: false and privileged: true. Not sure. The other question is, which actions and calls would be really restricted with this, when on the other hand the container is still running as privileged...

[akostic-kostile]

You are correct on point 1, you can not have allowPrivilegeEscalation: false with privileged: true.

On the other question I was actually struggling to find out what exactly privileged: true does, other than "it's akin to running as root on the host". So I did some tests myself and it seems that privileged mode bypasses almost everything except readOnlyRootFilesystem: true.

Did you think about using a base image without a shell? Since this is a go application distroless image would fit perfectly, and it would lessen the attack surface.

[ckotzbauer]

With the privileged flag set, the container-runtime grants all root-privileges to the container (that's more than running only as a root-user). A distroless-image would be something we can discuss about, I think. Maybe in our next monthly meeting on 27th. You can attend as well, if you want.

/cc @dholbach

[akostic-kostile]

What time, and time zone, are we talking about for that meeting?

Should I close this issue and open a new one with correct description? Edit: or just rename this one?

[ckotzbauer]

September 27th, 16:00 UTC (via Zoom). You can also join our slack #kured channel at https://slack.weave.works/

Just leave the this issue as it is and discuss about it.

[akostic-kostile]

I'll try to be there, but if I'm unable please discuss about this issue. Security is twice as important when you have an app that needs to run privileged and no shell inside the container really cuts down all the attack vectors other than supply chain poisoning (kured image itself being compromised).

[akostic-kostile]

@ckotzbauer Sadly I won't be joining you tonight as I have different commitments, but please discuss this issue.

[jackfrancis]

TODO: identify the right upstream SIG to consult with on this

[rvanmaanen]

Hi! We're also in the process of reviewing our security and are running into an issue related to this. Azure Advisor is recommending to not run containers as root and while looking into this we noticed that it doesn't seem possible to change the securitycontext settings and from what I read here it also would break Kured. Is there any update on this?

[github-actions[bot]]

This issue was automatically considered stale due to lack of activity. Please update it and/or join our slack channels to promote it, before it automatically closes (in 7 days).

[rptaylor]

See also https://github.com/weaveworks/kured/issues/416 which proposes some ways to reboot without using a privileged pod.