Investigate performance impact and overhead of enabling Wireguard

[davidumea]

Is your feature request related to a problem? Please describe.

WireGuard is not enabled in ck8s by default, but why not? It is very simple to enable and offers an extra layer of security by encrypting node-to-node communication. I believe it does not have any notable performance impact but I have not compared this.

Describe the solution you'd like

Add calico_wireguard_enabled: true to config/common/group_vars/k8s_cluster/ck8s-k8s-cluster.yaml

migration step:

./bin/ck8s-kubespray run-playbook wc|sc upgrade-cluster.yml -b --tags=network,calico

Definition of done:

WireGuard is enabled by default

[davidumea]

Maybe @cristiklein has some thoughts on this, I couldn't find your investigation document.

[cristiklein]

Hi @davidumea. Thanks for bringing this up.

I would recommend against enabling WireGuard by default. Why?

1. The security benefits are unclear. We do regular provider audits on the providers we work with, so we know the network can be trusted. If we cannot trust the providers, they can easily circumvent WireGuard encryption by dumping the VM memory.
2. It adds complexity and overhead.
3. It creates a sort of "blindness", where the network is opaque for the data-center.

The NIST Application Container Security Guide, Section 3.3.3 seems to recommend against network encryption by default:

Although an encrypted overlay network provides many operational and security benefits, it can also create a security ‘blindness’ scenario in which organizations are unable to effectively monitor traffic within their own networks.

Makes sense?

@lucianvlad Whether we decide for or against network encryption by default, feels like this is ADR worthy. :smile: Can you draft one?

[llarsson]

On the contrary, I think we ought to do it, as per the NSA's Kubernetes Hardening Guide, Page 20. I value their judgement of security higher than that of NIST. And we actually also say that NSA's guide has influenced our architectural choices right on the overview picture (the one with the ISO controls and such).

[cristiklein]

@llarsson Thanks for pointing that out!

I get the feeling that NSA's Kubernetes Hardening Guide, Page 20 insists on control plane traffic encryption, which is on by default (can one disable it at all nowadays? :smile:).

From what I understood, the NSA guide is rather vague on whether data plane traffic encryption is a must have:

While service meshes are not necessary, they are an option that is highly suitable to the
Kubernetes environment. [...]

Another major benefit of modern service meshes is encryption of service-to-service
communications. [...] Some service meshes even perform this service-to-service encryption by default.

I see no wording suggesting that this is a MUST HAVE. More like a NICE TO HAVE.

Let's bring it up at the next arch meeting.

[llarsson]

Administrators should configure all traffic in the Kubernetes cluster—including between components, nodes, and the control plane—to use TLS 1.2 or 1.3 encryption. Encryption can be set up during installation or afterward using TLS bootstrapping, detailed in the Kubernetes documentation, to create and distribute certificates to nodes. For all methods, certificates must be distributed among nodes to communicate securely.

I don't read words such as "all traffic" and "between nodes" as being limited to the control plane.

[llarsson]

If anything, "between components" ought to mean that even Pods on the same machine should have encrypted traffic. So it would go beyond WireGuard.

[cristiklein]

We just bumped into this with a project. Seems like WireGuard needs a smaller MTU to make room for the extra headers. This definitely has some performance implications:

Source: https://docs.tigera.io/calico/latest/networking/configuring/mtu