k0s --strict mode implementation

[chasemp]

trafficstars

Is your feature request related to a problem? Please describe.

There are some behaviors for k0s currently that I believe are oriented to fulfill the "zero friction" tag line -- which is great. Some of these "just get a me a thing to try" hurdle leaps are dangerous or not ideal for production usage. In addition, the packaging of k0s lends itself to contextually valuable pre-flight (or in-flight) checks across components that is difficult in other models (kube-bench for example). A tradeoff for "zero friction" and robust operation may be the implementation of a --strict mode that exits/warns/take strong action on practices that are not safe in at-risk environments. This would allow both use cases to exist side by side.

Examples of zero friction but concerning behavior:

• If a config file is explicitly referenced that does not exist k0s marches on

A consequence of this is that the running process lists the non-existent config file as contextually significant:

root 20426 20381 0 18:04 pts/0 00:00:00 /usr/bin/k0s api --config=/home/ubuntu/k0s.yaml --data-dir=/var/lib/k0s

In this case if I were an adversary I would attempt to create the referenced but non-existent config file with the intent that settings are pulled in on service restart.

• An explicitly referenced config file should be ensured as not world writable
• The default PSP is privileged. I believe there are 2 PSP's pre-populated, and having workloads start with these capabilities by default is not ideal.

podSecurityPolicy:
defaultPolicy: 00-k0s-privileged

• Child processes for k0s (including etcd) attempt to run as service accounts but fallback to root with a WARN. This is dangerous as it would be unexpected behavior for the majority of production use cases.

WARN[2020-12-11 21:15:01] Running etcd as root: user: unknown user etcd
INFO[2020-12-11 21:15:01] Staging /var/lib/k0s/bin/etcd
WARN[2020-12-11 21:15:01] Running kube-apiserver as root: user: unknown user kube-apiserver
INFO[2020-12-11 21:15:01] Staging /var/lib/k0s/bin/kube-apiserver
WARN[2020-12-11 21:15:01] Running konnectivity as root: user: unknown user konnectivity-server
INFO[2020-12-11 21:15:01] Staging /var/lib/k0s/bin/konnectivity-server
WARN[2020-12-11 21:15:01] Running kube-scheduler as root: user: unknown user kube-scheduler
INFO[2020-12-11 21:15:01] Staging /var/lib/k0s/bin/kube-scheduler
WARN[2020-12-11 21:15:01] Running kube-controller-manager as root: user: unknown user kube-apiserver

• config not found reported as info, and starts with defaults. This is unintended behavior in all but the most PoC of use cases.

ERRO[2020-12-11 21:25:49] THINGS MIGHT NOT WORK PROPERLY AS WE'RE GONNA USE DEFAULTS

Beyond a flag that allows for sanity checking this opens the door for equivalent functionality to something like kube-bench where we can vet the interaction and configuration of internal components with minimum risk of false positives. I believe this could be a really nice differentiator for k0s.

Describe the solution you'd like

Examples of behavior under --strict (or whatever)

• If a config file is explicitly referenced that does not exist k0s should exit with a code of 1
• An explicitly referenced config file should be ensured as not world writable
• Non-privileged PSP as default
• Child processes for k0s fail if their limited service account is not present

--strict --validate-cis ? Possible next evolution or really turning up the zero friction on CIS/whatever compliant k8s.

Describe alternatives you've considered

• We could put the really concerning stuff in regular mode and not go for a --strict thing etc.
• We could forget this all together (but I fear the contention between "zero friction" and "sane default" will remain)

Additional context mysql, js, etc all have some semantic for "strict" mode. It's a fairly well understand idiom in the industry.

[chasemp]

Note: Config file handling behavior changed in https://github.com/k0sproject/k0s/issues/705 moves that bit to the 'saner handling all around' bucket rather than as a candidate for separate permissive and strict approaches.

[chasemp]

https://github.com/kubernetes/enhancements/pull/2569

#2569 KEP-2568: Run control-plane as non-root in kubeadm. This KEP proposes that the control-plane in kubeadm be run as non-root. If containers are running as root an escape from a container may result in the escalation to root in host. CVE-2019-5736 is an example of a container escape vulnerabilty that can be mitigated by running containers/pods as non-root.

[jnummelin]

The issue is marked as stale since no activity has been recorded in 30 days

[github-actions[bot]]

The issue is marked as stale since no activity has been recorded in 30 days