xcp
xcp copied to clipboard
xcp-ng
WireGuard support
It would be great to have WireGuard support in Dom0.
Protocol Information
WireGuard is a simple and performant encrypted tunneling protocol that could be of use in XCP-ng as an easier to maintain alternative to IPsec-based tunnels.
Challenge
Because WireGuard is not implemented in Linux kernels > 5.6 as CentOS uses, WireGuard has to be installed manually. Although there are packages available for CentOS, XCP-ng would require the kernel module to be compiled for XCP-ng's kernel.
I have tried this with wireguard-dkms. However it required a lot of workarounds and is not really recommendable.
Hacky solution to try it out
Patching kernel-alt quickly and dirtily I was able to make the WireGuard module loadable:
patch
diff --git a/SOURCES/kernel-x86_64.config b/SOURCES/kernel-x86_64.config
index 341ae3e..f2a9ce5 100644
--- a/SOURCES/kernel-x86_64.config
+++ b/SOURCES/kernel-x86_64.config
@@ -1063,6 +1063,8 @@ CONFIG_NET_PTP_CLASSIFY=y
CONFIG_NETFILTER=y
CONFIG_NETFILTER_ADVANCED=y
CONFIG_BRIDGE_NETFILTER=m
+CONFIG_WIREGUARD=y
+CONFIG_WIREGUARD_DEBUG=y
#
# Core Netfilter Configuration
diff --git a/SPECS/kernel-alt.spec b/SPECS/kernel-alt.spec
index ddfb473..f21e7dd 100644
--- a/SPECS/kernel-alt.spec
+++ b/SPECS/kernel-alt.spec
@@ -606,6 +606,7 @@ Conflicts: python2-perf
%prep
%autosetup -p1 -n kernel-%{base_version}
+(uuid=$(uuidgen); git clone -b v1.0.20201221 https://git.zx2c4.com/wireguard-linux-compat.git "/tmp/${uuid}"; /tmp/${uuid}/kernel-tree-scripts/jury-rig.sh .)
%build
dmesg
[ 70.875906] wireguard: chacha20 self-tests: pass
[ 70.884016] wireguard: poly1305 self-tests: pass
[ 70.884400] wireguard: chacha20poly1305 self-tests: pass
[ 70.885224] wireguard: blake2s self-tests: pass
[ 70.897190] wireguard: curve25519 self-tests: pass
[ 70.897237] wireguard: allowedips self-tests: pass
[ 70.898816] wireguard: nonce counter self-tests: pass
[ 71.154215] wireguard: ratelimiter self-tests: pass
[ 71.154251] wireguard: WireGuard 1.0.20201221 loaded. See www.wireguard.com for information.
[ 71.154252] wireguard: Copyright (C) 2015-2019 Jason A. Donenfeld <[email protected]>. All Rights Reserved.
Further questions
Is anyone else interested in WireGuard support? Any comments on how WireGuard should be implemented? Some arguments against WireGuard support?
I really like Wireguard and here at Vates, we use it everyday.
However, for a dom0, it's another extra component, and adding new components should require careful balance between what it brings and the cost to maintain it.
Can you tell more about the use cases you have in mind?
A use case I am looking into - connecting hosted bare metal XCP-ng nodes using wireguard.
I suppose you mean: "accessing publicly exposed hosts without leaving SSH active for public IPs or exposing mgmt interface"?
Thanks for replying!
What it brings
what it brings
I tried to keep my first message short, but here are some advantages that we might want to look at:
- Higher throughput and lower latency than IPsec, which XCP-ng currently uses for GRE/ VxLAN security,
- The protocol is much easier to set up correctly, compared to the currently used IPsec-based security,
- The protocol was verified to have: correctness, message secrecy, forward secrecy, mutual authentication, key-compromise impersonation resistance, unknown key-share attack resistance and replay resistance of the first protocol message,
- The WireGuard network interface allows simpler mesh routing than currently used,
- Because WireGuard implements it's own simple protocol and not a ton of different cipher suites, compatibility across different hosts may be better to achieve over a longer period of time,
- Talking about the Future: The Pre-Shared-Key mode promises post-quantum resistance,
- WireGuard has gotten adopted very fast and even lot of Unix-like distributions/ projects patched their older kernel branches to implement WireGuard; where necessary XCP-ng's Dom0 could communicate with those through a fairly secure, simple and fast channel,
- et cetera
Cost to maintain
the cost to maintain it
The cost of maintenance mostly depends on how easy it is and how often those changes have to be applied.
How to maintain
The wireguard-compat repository offers some very useful scripts that make it fairly simple to add WireGuard to the kernel tree.
There is either create-patch.sh which automatically generates a patch file for a given source tree or jury-rig.sh – which I used in my test – that writes all changes directly into the kernel source tree.
Of course relying on automated scripts like jury-rig.sh or create-patch.sh can make the process fairly simple.
Of course we may look into how this process can be integrated into the kernel's repository in the least troublesome manner.
Maintenance frequency
wireguard-compat is released approximately once a month containing mostly minor changes. This almost fits in the release frequency of XCP-ng's kernel.
In case there are no critical vulnerabilities it looks totally sufficient to me to apply changes whenever a new kernel is about to released.
Troubleshooting
Of course Troubleshooting could add cost. Having built the kernel-alt just once for the most recent version, I am not sure how many obstacles are coming with new wireguard-compat releases.
Looking at an other Xen Dom0 kernel (QubesOS), that used to implement wireguard-compat (until 5.6), it should not cause any other trouble than occasionally changing version numbers.
Use cases
Can you tell more about the use cases you have in mind?
- As pointed out above, it could be a simpler and more modern alternative to XCP-ng's IPsec-based tunnels.
accessing publicly exposed hosts without leaving SSH active for public IPs or exposing mgmt interface
- Yes, because WireGuard is not easily detectable by port scanners, it allows for more trustless network layouts and reduces attack surface.
- Yes, it would allow XCP-ng pools to communicate securely over the public internet.
- Yes, it would be a secure second layer for administrative access across untrusted networks.
Of course, there may be more use cases imaginable. Looking at how steadily global interest in WireGuard has grown and how it is compared to IPsec today, there may be a lot of people with use cases that I did not even think of, yet.
That's where I'm at at the moment. Please let me know if you need help with anything.
As I said, I already use Wireguard and it's an excellent VPN software.
The problem then is not to compile and make it inside dom0. The main huge task is to integrate it. Right now, almost all network is handled by OpenvSwitch. How would you plan to integrate that in XAPI and the rest of the toolstack? WG is layer 3, OVS layer 2.
Having the package is one thing, integration is another. But PRs are welcome obviously!
I'm interested in this. I tried compiling WireGuard but it fails - probably due to the limitations outlined in the original post. I did not want to tamper with my host OS further, but would very much appreciate if this was implemented in a future release, in order to allow for secure access to internet-connected xcp-ng hosts through WireGuard tunnels. Favorably, it would be nice if the WireGuard interface was selectable as the management interface.
IMHO: Don't add that to dom0, as dom0 should be as slim as possible and not bloating up too much. Instead: Integrate it into XO. If something "explodes" there, the host would still be running stable. Loosing VPN can be annyoing but it's (usually) not necessary to keep the host running.
Wireguard alone isn't the trick, you'd also need some easy-to-use management. As XO already has a webgui, it's way more easy to implement such a management there.
I agree - but if the host is directly attached to the internet, it's the only way to attach it to a LAN on a host level. Of course it's not essential, however it eases manageability it the management VM is not reachable, as distributing IPMI/ILO is not always as easy.
One thing I'd never, ever, ever, do: Attach XCP-ng or other alike solutions management interface directly to the internet. I mean you can do that, but that's... nah. Security vs. RAS.
Not everyone can spare the expense of paying the hosting provider big money for an ASA.
@olivierlambert: Isn't it possible to create an internal network, connect XOA (or whatever) to it and make that one XCP-NGs PMI? Would be a simple solution, nothing fancy, nothing expensive.
You can create "internal" networks on XCP-ng. Those not phyical ones. The idea is to create one, make that one the primary management interface and bring it as 2nd interface to XOA. So you have XOA "public" as admin interface but won't fully expose the host to the world.
Internal networks between hosts are relying on OVS tunnels. But yes, you can already do that: having XAPI only using an "non public" network, and getting your XOA with 2 VIFs, one public and one in that private network.
But there's no wireguard anywhere in that equation.
It relies on my opinion: "wireguard would be best implemented in XO(A) instead on hostlevel". It keeps dom0/hypervisor as small as possible and also makes a comfortable management of wireguard more easy. Otherwise it would need documentation and scripts or even a bunch of shell commands to maintain the VPN clients. XO(A) could just supply a section on the, already existing, web GUI for that.
Let me try to rephrase then :) So you'd like to have wireguard in XOA to be able to access it remotely without exposing it to any public facing network, right?
That at least would my concept of "RAS" while keeping attention of security and stability of the host, yes.
So you could maintain your server, possibly even through VPN, without having XAPI etc. exposed to the internet.
You can already do that without exposing XAPI: XO can act as a proxy to only expose XO.
But yeah it works only if your XOA is inside the network in question, and not if your XOA is outside. In that case, the best solution would be to rely on "XO proxies" also doing proxy for XAPI not just for backup :)
We now have XO proxies that could be used to use a central XO and manage any remote sites securely. See https://xen-orchestra.com/blog/xo-proxy-a-concrete-guide/
We now have XO proxies that could be used to use a central XO and manage any remote sites securely. See https://xen-orchestra.com/blog/xo-proxy-a-concrete-guide/
Thanks for the quick replying. Does the XO proxy support IPv6? I have public and private IPv6 assigned to my network.
@olivierlambert it seems like it does not: xcp-ng shows me permission denied error when I try to assign ipv6 to xcp-ng interface.
XO proxy supports IPv6, but you are doing a command in your dom0. IPv6 support in the dom0 is coming for XCP-ng 8.3.
We now have XO proxies that could be used to use a central XO and manage any remote sites securely. See https://xen-orchestra.com/blog/xo-proxy-a-concrete-guide/
I didn’t see this article when it was written — it’s a fabulous explanation of exactly the use case I’m about to tackle. Thanks, @olivierlambert!
Question: is there more documentation about for proxy? I think this all but negates my desire for Wireguard on dom0, so long as I don’t need to open a port to the proxy. I can have my main XOA exposed, but I can’t rely on remote sites having a) static IP or b) access to NAT on the router. Would love for proxy to initiate the connection to XOA, rather than other way round. This and other questions I have for the documentation.
XO proxy supports IPv6, but you are doing a command in your dom0. IPv6 support in the dom0 is coming for XCP-ng 8.3.
There is other ways how to unite xcp-ngs. https://github.com/RiV-chain/RiV-mesh is an IPv6 based mesh network which allows to assign a fd00::/8 address to xcp-ng from subnetwork being routed from XOA. No proxies, no NAT issues. Proven that worked for ipv6 xcp-ng.
@andrewreid no extra/specific docs so far. But this will land in the official XO docs as soon we release a new one (we are transitioning from VuePress to Docusaurus).
Also, regarding your other request, I thought about it already, it's not an easy task. I need to discuss this with my team. Might worth a dedicated issue though.
+1 for Wireguard with XCP-NG. Maybe we can incorporate this in XO SDN ? basically replace the existing SDN with Wireguard ?
The star topology in current SDN would get bandwidth throttled very soon if you continue adding hosts to the pool.
