netbirdio
Feature Request: raw Wireguard/dumbed-down client implementation for devices with limited capabilities
It would be cool to have a way to benefit from automatic mesh on systems where it's impossible to run native client or where client is not available yet.
For example Mikrotik routers supports Wireguard. But they don't have way to run custom binaries. At the same time they have own scripting language that is able to do HTTP requests, configure network interface, add routes, etc.
Similar Wireguard-based mesh networks are solving this by using 'gateway' servers (one machine in network that has public IP is assigned a 'gateway' role and such 'static' clients have config with just one peer to that 'gateway' ). So basically 'unsupported' platforms are more like traditional VPN servers. They accessing network through fixed gateway that routes traffic to other peers without mesh.
But I think that better solution should be possible:
Generally it should looks like:
- generate some sort of authentication 'token' to access server, plus Wireguard private key
- Provide certain HTTP endpoint to get actual wireguard config with all known peers and their addresses. It may be whole wireguard config file (maybe just without private key) or just some JSON (or even both depending on parameter)
- Client may send list of ip address/ports that it listens via same or additional HTTP request. And servers should update it's internal DB with such information and notify other clients if it's changed.
- Client will poll server periodically to sync it's wireguard config
Surely such client will be unable to do NAT traverse, but it'll be able to reach others with public IP or others who runs native client
There are a few related issues, like:
- #115
- #187
- #213
- #461
PS. Feel free to rename issue to something more easy to understand.
Yes, I'm aware of this feature. It requires RouterOS 7.5 to create TUN device.
- It's only arm/x86. While most of low/mid end routers are MIPS based.
- On most routers It requires extra USB storage for container image (and not every router has USB port or it may be used by LTE modem)
- It can't create native Wireguard interface on router itself. Basically the only way to communicate with such container is over virtual 'bridge' network. So we'll have more complex network topology that pretty looks like 'Netmaker is running on Raspberry Pi that is plugged to one of ethernet ports of Mikrotik router'.
- It'll for sure use
wireguard-goand Wireguard over TUN device because there is no access to Wireguard kernel module from container. And all of this will run on pretty low power router.
Right now the most difficult part that makes it almost impossible is 'grpc'. The only way to solve this for netbird is to use some sort of proxy/adapter...
This feature is an excellent addition. I finally have a decent firewall with a Mikrotik Router. My board has Native Wireguard support and container support, but the container feature is a HIGH RISK feature covered in documentation with no guarantee of security I will probably never use it in a router. If we can use vanilla Wireguard as a client even with restrictions it would be nice. In the meantime, I can create a site-to-site connection with clients using the native wireguard in a static manner and use routes to expose the net bird network if necessary.
This feature is an excellent addition. I finally have a decent firewall with a Mikrotik Router. My board has Native Wireguard support and container support, but the container feature is a HIGH RISK feature covered in documentation with no guarantee of security I will probably never use it in a router. If we can use vanilla Wireguard as a client even with restrictions it would be nice. In the meantime, I can create a site-to-site connection with clients using the native wireguard in a static manner and use routes to expose the net bird network if necessary.
I seccond this need limited wiregaurd client support
In my opinion, this would be the most important missing feature in Netbird. Having this ability would add a lot more flexibility and open Netbird for many more use cases.
a short (possibly already outdated/incomplete by the time you are reading) list of feature requests related to implementing a raw wireguard and/or very dumbed down script-style client utilizing it:
- https://github.com/netbirdio/netbird/issues/496 - let's aggregate the discussion there
- https://github.com/netbirdio/netbird/issues/2627
- https://github.com/netbirdio/netbird/issues/1000
- https://github.com/netbirdio/netbird/issues/2504
It would be nice if the actual WireGuard configuration could be abstracted in a way that allows different devs to implement the interface for their wished setup. This would allow it to separate the control plane from the data plane. e.g. I could write a plugin that configures a WireGuard on Mikrotik router via the RouterOS Rest API. I could do that from another system or a container which only runs the control plan, so I get full speed. Someone other could write something for Ubiquity or pfSense/OpenSense.
The default module would be the integrated Kernel Wireguard you currently have and the backup for the user space Wireguard via tun.
I rethink about the same ideas until I have this ticket.
My proposal the very straightforward and direct:
- let's add some special option to netbird client "external device" and single configuration option - HTTP URL
- let's define HTTP API for all necessary actions - create WireGuard interface, list them, manage DNS, manage firewall, etcetera
- let's delegate to this HTTP API the actual way how to manage device.
As the result - Mikrotik RB4011iGS+RM - I will launch two containers
- Small container, which implement this abstract HTTP API to work with RouterOS command. I will manage the question on how to connect to Mikrotik by SSH and what kind of command I need to execute on Mikrotik side.
- NetBird container with NetBird client, configured to use HTTP API of the first container
On Mikrotik I could isolate & manage container in the way, where they could access each other, but not outside world Realiable, clean, straightforward solution
@nazarewk
- is it feasible solution?
- if I implement this solution, could I expect that Netbird will include it to NetBird upstream, or I will need to maintain my own fork?
Also
- Mikrotik has Golang library - https://pkg.go.dev/github.com/go-routeros/routeros/v3
- Maybe the best possible support should be inside https://github.com/WireGuard/wireguard-go
The only question is how to delivery this extra configuration (like endpoint-user-password or endpoint-user-private-ssh-key) from netbird client inside this library, but this purely technical question :)
I would just abstract the WireGuard/NAT calls in Netbird, so instead of calling the local WireGuard/Firewall stuff, anyone can write custom code for their router. Your idea with a webhook and running a second container is one way. Another way is just to call a script in the container. With docker images, it is easy to take one docker image as base image and extend it. This allows the Netbird code to stay small and manageable, but everyone can take the base image and extend it with their script/binary and just set and environment variable so it is called by Netbird.
No need for 2 Container and HTTP API. As I'm not a go programmer I just need someone to but the code into Netbird that external scripts can be called, the REST API for the Mikrotiks I can then do :-) and someone else can then write a code for pfSense/openSense or something like this
I would just abstract the WireGuard/NAT calls in Netbird, so instead of calling the local WireGuard/Firewall stuff, anyone can write custom code for their router. Your idea with a webhook and running a second container is one way. Another way is just to call a script in the container. With docker images, it is easy to take one docker image as base image and extend it. This allows the Netbird code to stay small and manageable, but everyone can take the base image and extend it with their script/binary and just set and environment variable so it is called by Netbird.
No need for 2 Container and HTTP API. As I'm not a go programmer I just need someone to but the code into Netbird that external scripts can be called, the REST API for the Mikrotiks I can then do :-) and someone else can then write a code for pfSense/openSense or something like this
Or add ability to use golang plugin with so files One plugin could call scripts, other could work with RouterOS :)
goland plugin so files are also ok ... with an example plugin which calls external scripts - I would be happy :-)
goland plugin so files are also ok ... with an example plugin which calls external scripts - I would be happy :-)
I do not have any problem to quickly implement it. I just need confirmation from maintainers, that this is okay and will be eventually merged and supported
I do not like waste time to idea which will not be approved by maintainers
ok, the golang plugins have the additional benefit if a plugin is in mature and in high demand it could go into the base docker image at some point and others can stay separate.
ok, the golang plugins have the additional benefit if a plugin is in mature and in high demand it could go into the base docker image at some point and others can stay separate.
Also
- independent dependency management
- ability to maintain them outside of this repository with netbird agent
- even you have bundle of plugins in image, you will connect only required for you (could be done by config)
I could implement all this, I just need confirmation from maintainers, for instance, from @nazarewk that this is okay for project and will be merged/supported (in case of proper implementation, of course) After that I will implement it.
I just want to avoid time wasting to dead-end, which never will be accepted by netbird's team
For what it is worth, I'd just copy Netmaker's approach -- given the same Wireguard base, I'm guessing maybe even some of the code could be reused? -- which allows one to use any standard Wireguard client. While I've found Netbird more reliable than Netmaker, this is a spot they have a clear edge since it doesn't require implementing new clients or anything like that. https://www.netmaker.io/features/client-types
@trbutler I don't believe that is possible with the mesh traffic and therefore tunnels .. netbirds needs to create a wireguard peer on every endpoint so they talk with each other, that configurations needs to be done somehow. If you only connect to a central spoke with a "standard wireguard" that's ok, no need to create peers ... but that's not how netbird works, which is also the main differentiator to others.
@robertpenz I admittedly didn't explore too much how exactly Netmaker's system handled Wireguard clients in relation to other peers (other than that it worked), but the overall architecture is very close to Netbird's or Talescale's with a central controller and peer-to-peer communication beyond coordination. It too is a "mesh" system. I find that Netbird's management system has been more reliable (or was when I decided to go with Netbird instead of Netmaker back in 2022), but the basic design is very similar.
Mikrotik has native Wireguard support - so why not let Netbird use the Mikrotik API to manage Wireguard interfaces/peers?
@scr4tchy: that's the idea, but to be able to manage the Mikrotik API we need to add that functionality in Netbird, hence this discussion. Adding this is fine, but the maintainers of Netbird also need to stated that they will to accept such a pull request - therefor, we are waiting on @nazarewk. And do not be vendor specific, the idea is a plugin system which allows writing to different router APIs.
@trbutler I don't believe that is possible with the mesh traffic and therefore tunnels .. netbirds needs to create a wireguard peer on every endpoint so they talk with each other, that configurations needs to be done somehow. If you only connect to a central spoke with a "standard wireguard" that's ok, no need to create peers ... but that's not how netbird works, which is also the main differentiator to others.
Hello folks. NetBird won't work with vanilla Wireguard without major changes on the system since it defines the best endpoint for the p2p connections for every peer. For instance, we launch local proxies when the connection should be using relay, and that's something that would require more than the plain Wireguard client.
If anyone has experience building a client for Mikrotik, we would be happy to discuss how to build and integrate NetBird directly.
We do have some users that used docker in Mikrotik and generated a nice doc on that: https://docs.netbird.io/how-to/client-on-mikrotik-router
@trbutler I don't believe that is possible with the mesh traffic and therefore tunnels .. netbirds needs to create a wireguard peer on every endpoint so they talk with each other, that configurations needs to be done somehow. If you only connect to a central spoke with a "standard wireguard" that's ok, no need to create peers ... but that's not how netbird works, which is also the main differentiator to others.
Hello folks. NetBird won't work with vanilla Wireguard without major changes on the system since it defines the best endpoint for the p2p connections for every peer. For instance, we launch local proxies when the connection should be using relay, and that's something that would require more than the plain Wireguard client.
If anyone has experience building a client for Mikrotik, we would be happy to discuss how to build and integrate NetBird directly.
We do have some users that used docker in Mikrotik and generated a nice doc on that: https://docs.netbird.io/how-to/client-on-mikrotik-router
Hello!
Thank you so much for your reply!
I take a look to netbird source code, and it's clear how you build up abstraction on WireGuard/DNS/Firewall configuration I have understanding how to implement MikroTik API client to perform the same actions on Mikrotik side Pre-requisite - should be container on MikriTik in host network mode (to share NAT traversal port directly)
So, @mlsmaycon , my questiion the following: Option A. would you like to have support of RouterOS API directly in netbird client source code? (my be with some golang build tag) OR Option B. would you like to have some "external" kind of configuration, where netbird receive in configuration path to custom scripts? OR OptionC. would you like to have some "plugin" kind of configuration, where netbird load golang plugin of certain interface?
The difference the followings Option A. Simpler for user, for it will force to add all other devices, different from RouterOS, directly to the source code base Option B. Ideal for non-developers, system administrators. They will able to DIY integration with own router or very custom devices Option C. Ideal for developer, implement own plugin, load it, no injection to codebase
@mlsmaycon I could implement any approach, my question to you - what NetBird team prefer?
Let's keep this issue in the spirit of supporting low-capability devices, either through raw Wireguard or a separate resource-conscious implementation/operation mode for NetBird.
I would be very wary of saying that running a full client in compatibility mode would be sufficiently resource-efficient to run on most of the networking devices, let's move this part to https://github.com/netbirdio/netbird/issues/3921
To summarize the on-topic discussion https://github.com/netbirdio/netbird/issues/496#issuecomment-2933728123 .
We are not discarding the possibility of supporting this in the distant future, but we are not planning to support a plain WireGuard client or low-resources operation mode in the foreseeable future. In short, this is too large a codebase rework effort for too little perceived benefit. It seems like a relatively niche usage, considering most of the use cases could be achieved by running a client/routing peer externally to the routing devices.
We would definitely put more emphasis on the feature if a large enough number of customers required this, funded or took the bulk of the development effort on themselves (which again, is expected to be large).
PS: feel free to keep the discussion going and ideas flowing, it won't go to waste :)
@excavador would you be available for a chat? we are on Slack and perhaps we can arrange a call to discuss how the Mikrotik integration would look like
@excavador would you be available for a chat? we are on Slack and perhaps we can arrange a call to discuss how the Mikrotik integration would look like
Slack or meeting is fine. Send link or calendar event to [email protected]
interesting discussion. i am managing a bunch of low-end teltonika routers remotely, which runs on a dinosaur fork of openwrt. as for now we are only able to configure raw wireguard client in the devices. there is only 1-2MB flash disk, so installing eg https://downloads.openwrt.org/snapshots/packages/mips_24kc/packages/netbird-0.45.3-r1.apk will not work since the binary is 10MB. having the ability to use "raw" wireguard config files would be very much appreciated!
@tobias-carlbom - What if there was a shell script that could run on OpenWRT that simply makes regular calls to Netbird to fetch the WireGuard configs to update locally?
@haneef95 sure that would work. But I don't know - I actually got the binary working on my test device, running in ram memory. It feels more "safe" to run the binaries.
Although it feels hacky running from ram, it's pretty easy to automate this to even get it working if the device restarts.