subotai

subotai copied to clipboard

Does this currently work with nats between the computers, or do they need to have to have public ips also, ipv6-firewalls, that only let you recieve trafic when you sent trafic on the same port.

Hello @iqualfragile,

At the moment, NAT traversal is not supported, although that's a priority for me when I'm done ironing out some kinks :). So far I've been using a single node for my entire LAN in cases where NAT traversal would be desirable. I will update the docs to reflect this; thanks for the heads up.

I'm not too familiar with the ipv6 firewall problem. Is that a problem for UDP too? Each node uses a pair of in-out UDP ports that you can specify on construction, so I imagine you could configure your firewall to work well with that, but I may be missing something.

@PabloMansanet Yes, that is especially a problem with udp, as the firewall in case its inspecting state can't determine if a connection is over. Even though its recommended against, many nat-producers will pre-configure their ipv6-firefwall to not let any packets go to machines in their network, some even go as far as to only allow response-trafic, which makes things even harder, but actually handleable in a dht.

T = natted node, N = neighbor of T, X = node trying to contact T T -> N | Hey, i need to sort myself into the dht N -> T | response traffic: ok, you are smaller then me, move on (...) | time goes by, keepalives/pings are being sent X -> N | Where is T N -> T | "response traffic": X is trying to find you T -> X | Hi i am T X -> T | Query

It sucks, makes attacks a bit easier as you can trigger two messages (N->T, T-> X) by just sending one and probably breaks half the time, but thats just the world we live in. As DHTs are often used for peer discovery, thats an important use case.

Alternative options would be to use a less commonly used protocoll, i.e. sctp and hope no nat-boxes have firewall-rules for that, but they might just outright block everything that is not udp or tcp.

There are some crates for NAT traversal, like this one: https://github.com/maidsafe-archive/nat_traversal But it shouldn't be hard to implement, as long as you have a mediating server that's not behind a NAT. The only case where it fails is when both clients are behind a symmetric NAT that only allows traffic back in from the target of a request. But P2P applications should try to open ports via UPnP first, there are some Rust crates for that too, like https://github.com/SBSTP/rust-igd but AFAIK that's only one possible protocol that the router may or may not respond to, you still have to cover the other ones, so I suggest writing bindings for libminiupnpc and libnatpmp, they are good libraries and also work with MSVC. (But it would be useful to have port mapping in a separate crate).

@Boscop sadly all the maidsafe stuff is full gpl3 so you can't use it as a dependency of a more liberally licensed application/library. having a mediating server kind of defeats the purpose of a dht, unless by server you refer to the entity i call "N" in my previous post.

@iqualfragile Every subotai node can easily find out if it's behind a NAT (e.g. through a list of public STUN servers or by asking the router for the public IP using libminiupnpc). In the DHT you could store a list of nodes that aren't behind a NAT, and use them as mediators. This would require being able to access this info already before the NAT hole-punching. Otherwise you could store the list of non-NATed nodes on a tracker server. Even if a tracker server is necessary, it doesn't defeat the whole purpose of a DHT. Content distribution is still decentralized.. But yeah, it would be best if we don't need a tracker server.

Last time I looked at this I was wondering how feasible it would be to implement something similar to pwnat for UDP hole punching without a mediating server.

Simplifying the explanation horribly (and admitting this is not my domain of expertise) it would involve abusing ICMP by allowing any node to open a direct UDP communication, by pretending to answer an echo request that has timed out. Every node would in turn be constantly sending echo requests to a previously agreed upon "black hole" ip.

pwnat is actually genius, but i don't think we need that for a dht. assuming that there are some well-known/bootstrap nodes in the dht, everybody always knows all addresses and ports involved, see my post from 12-09-2016. depending on the kind of nat involved the last two lines would need to be slightly changed if its a restricted/port restricted nat. i am not sure about symetric nats yet. i fear that pwnat might "spam the internet" by sending many unroutable packets, but they should be sorted out at the next full table router.

I see what you mean. In any case, the volume of traffic generated by a pwnat-like system would be insignificant compared to the background traffic required to keep the DHT itself running. After all, all that's needed is an ICMP packet once in a while (dictated by the flush time on your router translation table I guess) where subotai nodes are constantly exchanging much bulkier stuff.

I'm not sure we can always count with a bootstrap node; the personal use case that made me start to work on Subotai involves being able to create smaller ad-hoc networks where no node is dependable, so I'm not sure that would cut it. It's a matter of testing it :)

i am not quite sure how you would bootstrap a dht without knowing at least one parties ip address (well besides ip6 multicast maybe). At least on the first join the new peer needs to know what to connect to. on subsequent ones they can just use cached information about their neighbours from earlier. you do have a point though, what if all the neighbours are behind a nat? is that a likely scenario?

So, about bootstrapping, I'd like to leave it as a choice for the application layer. With my proposed system, it would be trivial to use a dependable, static node for bootstrapping if your network can be modeled like that. It could also be possible to do something silly like a temporary invitation link based on your current dynamic IP, which is something I might have an use for :).

Regardless of your bootstrap method, I think there's value in doing things that way because it really reduces the workload on the mediating server. Even in nodes that are already bootstrapped, if you want A to talk to B, you will need to do so through the mediating server(s) as well (or find a convoluted way to relay that initial message). However, with the pwnat-like approach, only the bootstrap requires a known IP, then every communication is point-to-point.

I didn't know about pwnat, it seems awesome, it would be so useful to have a cross-platform implementation of it in Rust!

Yup. So, about that, @Boscop, do you think it would be valuable to have a separate crate that implements the pwnat mechanism with libpnet as an only dependency, and then bring that into Subotai?

Yes, I think a lot of other applications can benefit from it as well. (E.g. right now I'm working on a multiplayer game where every client also comes with a server, so that anyone can host a game. But most players will be behind a NAT, so this would be very useful. And I plan to use subotai for content distribution of user-made game maps / assets, on top of a distributed package manager.)

Cool, you just gave me a lot of motivation ;)

having a rust impl (lgpl or freer) of proper nat-traversal techniques would generally be nice, including stuff like the port mapping protocol and similar.

Any update on this? How difficult would it be to port pwnat to Rust?