ZeroNet
ZeroNet copied to clipboard
HelloZeroNet
Use torrents over I2P for anonymity
I2P has bittorrent support, and it works very well (but slow). So suggestion is to use it. There should be an option to use only I2P, only clearnet, or both.
More info: https://geti2p.net/en/ https://geti2p.net/en/docs/applications/bittorrent
Relevant thread: http://zzz.i2p/topics/1613-azi2phelper-vuze-plugin (requires I2P to view). Vuze has implemented both I2P-only and clearnet+I2P modes for regular torrenting, and their code may help provide insight about how to add I2P support to ZeroNet.
Disclaimer: I'm an I2P developer (and happy to answer questions!)
Some more points:
- I2P has a well-used in-net torrent DHT. See the BitTorrent docs page (also linked above) for details (there are some slight differences from the clearnet DHT protocol to support I2P addresses).
- AFAIR Vuze's azi2phelper plugin manages the I2P and clearnet DHTs separately.
- I2P supports UDP, if it helps (via SAMv3 API).
Buit-in Tor (with hidden services) just landed 1 week ago, the core modifications was made in mind of future i2p/ipv6 support. I was looking for python libraries for i2p, but only found some outdated ones (updated 12 years or 3 years ago).
Need more research to see if i2p support many (per site so 100+) services from same host and have to find out the cryptography of i2p address to be able to create proof-of-ownership signature to avoid some network attacks. The other main problem is to make it any usefull we have to pack it with ZeroNet, i'm not sure if its possible with java applications.
Thank you for your quick response.
Maybe your first set of issues is something @str4d can help with.
Re bundling i2p: i think that making the i2p transport optional and then require the user who wants to use it, to install i2p himself is ok. I don't think everything should be bundled.
The problem with optional installation: I2p does not allow socket connections outside the network, so to be able to connect to a site using i2p it must have at least one peer with i2p address which is unlikely if they have to start/configure by hand.
@HelloZeroNet if you were using Twisted I'd point you at txi2p, but instead I'll point at the more general Python library. Both use the SAM API, which is supported by all I2P implementations.
Re: bundling, I agree bundling Java would be tricky, because it would require the JVM. Bundling Python in Java is easier because of things like Jython - I don't know if there is maybe some reverse equivalent? Otherwise you could look at the C++ I2P routers being developed (i2pd and kovri), which I assume could be bundled the same way you are bundling Tor. I would certainly recommend at least having a UI option to use an external I2P router, even if it is not the default; many I2P users run a router 24/7 on a VM and run their transient apps through it remotely. This would be the same setup as other I2P-supporting apps like Vuze or Bote.
Need more research to see if i2p support many (per site so 100+) services from same host
I need some more clarification on this. Are you wanting one Destination (I2P equivalent of Tor Onion Service) per site, or multiple listening sockets at different ports on the same Destination? The former is possible (though YMMV regarding how many hundreds you can run at once :P ); the latter will be possible soon via the above Python libraries (we only just exposed via the SAM API our support for multiple ports).
and have to find out the cryptography of i2p address to be able to create proof-of-ownership signature to avoid some network attacks.
The binary format of the B64 private key that SAM returns to you is documented here; you could instantiate the private signing key yourself and use it to sign anything you wanted. That may be something that gets added to the Python libraries in future if enough people require it.
@str4d Thanks for the responses. As for the "many (per site so 100+) services from same host":
In full Tor mode we create a new .onion address for every site you seeding to improve privacy by make it harder to find out what other sites you using. So if someone has 100 sites in his/her client then on the startup we need to create 100 i2p address on startup.
that will not scale in i2p. because i2p router must build those tunnels and rebuild them as they change every few minutes for increased privacy. Maybe sorting them into different pools could be a viable solution? though this would require much work. Who decides which pool a site would be in? would have to be the user and would require one additional step for him before seeing the site. So not really good.
OTOH, vuze does not create a different destination for each torrent either. But I perfectly understand the point of uncorrelating the sites.
I wouldn't go as far as saying it isn't possible - I'm running ~30 services for a combined ~110 tunnels on my router. But you would definitely want to do some kind of tunnel management if you were spinning up one Destination per site. I would suggest dropping to one or two inbound and outbound tunnels each for low load (probably dropping down to only one for sites that have no peers downloading for some time), and adding more as load increases. Vuze uses a similar mechanism to great effect (I've had as much as 200kB/s combined download speed for torrents over I2P, at which point Vuze will have IIRC six inbound and outbound tunnels).
One downside to this mechanism would be that it becomes easy to determine which sites are popular, by observing which ZeroNet Destinations have more tunnels. But maybe this isn't part of your threat model.
The setup Vuze uses is to have only two Destinations: one for torrents that it seeds inside I2P only, and the other for torrents that it simultaneously seeds on the public internet. This minimizes correlation between their public and anonymous activity.
@HelloZeroNet Any news? I'm really looking forward for this to be implemented, as I prefer I2P support much more than Tor support... And many other users want to have I2P support, instead of Tor support because it is distributed and peer-to-peer friendly
more info:
Just FYI, I2P has launched a development program aimed at helping developers integrate I2P into their apps. If you would like some help with this in July during our Apps month, let me know! We are currently in APIs month, so I'll have a poke around the ZeroNet source to make sure the libraries you would need are being improved :smile:
@str4d : ZN is running around 75 hidden services in my case. (To make sites served by same peer less linkable) Is i2p able to handle such an amount of different destinations, besides maybe other tunnels like i2psnark etc?
@aaannndddyyy in theory, yes it can. In practice, you'd want to set up tunnel management such that each Destination only had one tunnel unless it was being actively used and needed the additional bandwidth. And of course you'd want to have your I2P router sharing a lot of bandwidth, because even with every hidden service Destination using only one tunnel, you'd be using a total of 225 hops by default (for 3-hop tunnels), so you'd need to be sharing appropriately to avoid Zeronet causing network congestion. (We had that problem initially with Vuze, until they changed the default sharing percentage of their internal I2P router to 80%. Now Vuze nodes contribute a sizeable fraction of our routers.)
First structural question: I2P has the concept of a Destination, analogous to an IP address. We have Python datatypes for these Destinations as part of the module I'm using for the integration. Would the Zeronet devs prefer to use that datatype throughout (ie. wherever Peer.ip is used when generating a string, check if it is a Destination and behave accordingly)? Or would it be better to just keep Peer.ip as a string, and then parse from that only inside the I2P-specific code?
- Note that there are many places that do
if ip.endswith(".onion")that currently need to becomeip.endswith(".onion") or ip.endswith(".i2p"), but using the Destination datatype they would instead beisinstance(ip, Destination)orip.endswith(".onion"). The ordering would be necessary to ensure thatipis only parsed as a string after we know it isn't a Destination.
Follow-up question: if using strings and not Destinations, would the Zeronet devs prefer to use the full 387+ byte (ie. variable-length depending on crypto used) base 64 strings (which are just serialised Destinations), or the B32s (52chars.b32.i2p)? The latter is only 32 bytes (and therefore easier in logs and less data to transfer between peers), but requires an additional lookup before it can be used anywhere (ie. B32 -> B64, then connect to B64) which will add some latency to making connections.
- Note that if using the Destination datatype, it can easily be converted to either B32 or B64 exactly where required (so e.g. in log messages the B32 could be used, while in network messages the full Destination could be used).
- I2P's BitTorrent spec specifies that non-compact tracker responses include the full B64, while compact responses and PEX use the B32 hash. That may or may not be helpful, because AIUI Zeronet only uses a BitTorrent-like protocol.
WRT the first question, I thought it was possible, but I've found that some IPs have a cert_pin appended, the origin of which I cannot trace but it indicates that the ip field is overloaded with meaning. Therefore I'll try to stick with strings.
On 2016-07-23 13:46, str4d wrote:
I'm doing the grunt work for this issue this weekend 😄
That's great news. Thanks
On 2016-07-23 10:25, str4d wrote:
@aaannndddyyy [1] in theory, yes it can. In practice, you'd want to set up tunnel management such that each Destination only had one tunnel unless it was being actively used and needed the additional bandwidth. And of course you'd want to have your I2P router sharing a lot of bandwidth, because even with every hidden service Destination using only one tunnel, you'd be using a total of 225 hops by default (for 3-hop tunnels), so you'd need to be sharing appropriately to avoid Zeronet causing network congestion. (We had that problem initially with Vuze, until they changed the default sharing percentage of their internal I2P router to 80%. Now Vuze nodes contribute a sizeable fraction of our routers.)
I guess the reason why tor can do so without any proble is that for tor it makes almost no difference if you access 200 sites through one own hidden service (just like visiting 200 eepsites using your eepProxy) or you access the same 200 sites through/from 200 different hiddenserver addresses. The per service overhead should be as small as possibly, so that the overwhelming factor is only the actual data being transferred, which in both cases is the same. Then it would also not matter if you use a total of 6 hops or 300 or 1200 hops (200 sites * 3 hops * 2 directions), because all the 1200 hops would consume only little more than the 6 hops would do if the same sites were visited/served through them. Like if a user share X kB/s, it should not matter if he routes two tunnels with X/2 kB/s each, or 100 tunnels with X/100, also amounting to exactly the same ressource usage. Only floodfills would have more work.
I have the impression, thugh, that i2p in that respect is much more ressource hungry and needs significant traffic overhead per-tunnel. Is the reason bhind that, that i2p is more decentralized than tor or that tunnels are more short-lived? Is there anything that could be done to reduce? - apart from tunnel management that reduces the per-dest tunnel count to one, which would still be a huge amount if you are serving 200 site only with zeronet and are still having other aplications use i2p too (torrent, …)
I have this mostly-done. Outgoing communication is working fine (ZeroNet can announce to I2P open trackers), but incoming communication is currently semi-to-non-functional because i2p.socket's accept() doesn't quite work yet with gevent's cooperative sockets. I'll finish preparing the commits on ZeroNet today and make a PR for discussion.
@up4 thanks for the notification, I'll have a look at it.
@str4d my patch is the beginning of something to abstract the transport and create a configuration to allow the user to have the control over which transports he wants to use. Maybe I should PR to your repo those abstractions and then you can PR to the main repo. Anyways I2P support would be great!
@HelloZeroNet Maybe the future torrent plugin should have i2p support? The zites themselves run fast enough through tor.
It will use https://github.com/arvidn/libtorrent which has no i2p support atm.
@HelloZeroNet Are you sure about no i2p support? https://github.com/arvidn/libtorrent/blob/9fd83aaa10c1bca0d36b783004af89897bf3e618/src/i2p_stream.cpp
Oh! Sorry for the lack of updates here. I finished implementing I2P support in ZeroNet back in July, but didn't make a PR because it didn't work (due to i2p.socket not working fully with gevent). Then I decided I'd just make a PR anyway in case someone else could get i2p.socket working, but then work became busy :sweat_smile: I'll push the PR now.
patched i2p.socket to like gevent more but need some i2p testers, once it's confirmed working I'll tag a new release for i2p.socket.
Any news here? what's the status of this, now that i2p.socket was patched?
I assume that i2p.socket fully works with gevent. The remaining problem is that ZeroNet creates too much I2P hidden services and i2p will be overloaded. I am thinking about randomly dividing the list of sites in to at most 50 subgroups of variable lengths in order to reduce the number of hidden services used.
I don't think that overload will be a big enough issue for zeronet to worry about.
In my experience this was an issue with i2p. It was some time ago, but it happened. Because of all the tunnel maintenance that has to take place. But only actual test will tell, if it is still the case. And if it is, the pooling @MuxZeroNet has talked about could be a way to mitigate that. Maybe the i2p side itself could also work on improving that, as it is a non-issue for tor.
shouldn't just 1 hidden service be used with zeronet? that makes a lot more sense.
in tor it uses different circuits per site for inter-site unlinkability - which is a pretty neat feature.
i see, such an option makes little sense with i2p you have a reply to destination so if you churned through that you'd be creating new unreliable peers with short lifespans.
it makes sense, it just adds a lot of load. so instead of on a per-site basis, this could be done on a per-pool basis, joining the different sites into pools. This works fine on i2p and is a compromise between anon/privacy and performance. The churn come from i2p rebuilding tunnels every some minutes.
Making more than 1 destination could spread the attack surface out AND increase network load. I see very little reason for it but I digress. I will release the next version of i2p.socket if someone can point me to a demo branch with i2p support in it.
Thank you very much. Do you think a even simpler SAM enabled socket module is worth integrating into ZeroNet? Just like what TorManager does: it has a simple control socket implementation plus Stem Control Library support.
| TorManager | I2PManager | |
|---|---|---|
| Simple Implementation | ✓ | ? |
| Control Library | ✓ stem | ✓ i2p.socket |
i2p.socket is a simple implementation, the only changes required (ideally) is
import socket
to
from i2p import socket
The other main problem is to make it any usefull we have to pack it with ZeroNet, i'm not sure if its possible with java applications.
You can use i2pd, it has a much smaller footprint than java and works cross-platform. https://github.com/PurpleI2P/i2pd
i2pd's sam code is still a bit crashy, idk how well it'll work, otherwise i2pd should be stable by now.
another alternative is i2pcontrol and socks5 but that may require too much configuration, unless you are planning on bundling a binary.
ZeroNet have about 58,570 USD, more than enough to hire developer for this. https://www.blockchain.com/btc/address/1QDhxQ6PraUZa21ET5fYUCPgdrwBomnFgX
