libtorrent icon indicating copy to clipboard operation
libtorrent copied to clipboard

Bug: DHT doesn't work under any proxy

Open master255 opened this issue 3 years ago • 34 comments

@arvidn libtorrent version (or branch): any

platform/architecture: any

compiler and compiler version: any

DHT doesn't work under any proxy. A continuation of this problem: https://github.com/arvidn/libtorrent/issues/4078

master255 avatar Mar 11 '21 00:03 master255

please be more specific

arvidn avatar Mar 11 '21 08:03 arvidn

@arvidn I used the WinGate proxy server with the latest, working version of the library 1.2.13. I've tried all combinations of proxies. DHT doesn't work. I checked DHT with get and put commands.

master255 avatar Mar 11 '21 08:03 master255

@arvidn I just checked. In Utorrent DHT works through socks4 proxy. Libtorrent does not. I used similar settings and the same proxy.

master255 avatar Mar 11 '21 21:03 master255

@arvidn I checked how Libretorrent works. There proxy never starts at startup and if it is applied after startup, DHT doesn't work. Proxy in Libtorrent does not work for DHT. Arvin, fix it. This is important.

master255 avatar Mar 12 '21 00:03 master255

@arvidn There are a lot of problems:

  1. It is necessary setting to disable proxy for DHT. Because, for me, without a proxy DHT works, but peers do not. With a proxy, peers work, but DHT does not. Through the trackers the download goes, but this is not enough.
  2. DHT does not work through any proxies. Fixing this is more difficult, it seems to me.
  3. DHT should not be proxied through HTTP proxies. Right now, with an HTTP proxy, DHT stops working.
  4. i2p proxies don't work either. But this is the lowest priority.

master255 avatar Mar 12 '21 01:03 master255

can you provide a wireshark dump of the packets sent to and from the SOCKS5 proxy?

arvidn avatar Mar 20 '21 22:03 arvidn

@arvidn Adding torrents does not cause new traffic. I used this filter: ((ip.dst == 192.168.1.101) && (ip.src == 192.168.1.100)) || ((ip.dst == 192.168.1.100) && (ip.src == 192.168.1.101)) proxy port 4449 123.zip

master255 avatar Mar 20 '21 23:03 master255

This message is the SOCKS5 handshake. I don't see the proxy respond.

05 01 00

I also don't see any UDP traffic going to the proxy, presumably because it got stuck handshaking.

The server is definitely supposed to respond, but it isn't. The protocol is specified here.

Are you sure your proxy is supposed to support SOCKS5?

arvidn avatar Mar 21 '21 01:03 arvidn

@arvidn This is the first proxy server to be found in a Google search. Wingate - you can install it yourself and check it out. It's free. 321313123

I made a mistake last time. The proxy service was disabled. I just recorded a log with the service turned on. UTorrent doesn't work with Socks 5 either. But with Socks 4 it works for sure. 123.zip

master255 avatar Mar 21 '21 09:03 master255

@arvidn image

master255 avatar Mar 21 '21 09:03 master255

as you can see in your wireshark dump, the exact same behavior is exhibited. The proxy server does not respond to the SOCKS5 handshake. Perhaps you could try to use SOCKS4 instead.

arvidn avatar Jun 07 '21 14:06 arvidn

@arvidn Similarly, it doesn't work with SOCKS4. I checked all kinds of proxies.

master255 avatar Jun 07 '21 19:06 master255

Do any of them respond to the SOCKS handshake?

arvidn avatar Jun 07 '21 19:06 arvidn

@arvidn I don't understand and I don't know. How can I check it? Wouldn't it be better if you checked it yourself? I spent all evening trying to set up a proxy for DHT. And it didn't work.

master255 avatar Jun 07 '21 20:06 master255

it works in my test: https://github.com/arvidn/libtorrent/blob/RC_2_0/simulation/test_transfer.cpp#L325

arvidn avatar Jun 07 '21 20:06 arvidn

I made a mistake last time. The proxy service was disabled. I just recorded a log with the service turned on. UTorrent doesn't work with Socks 5 either. But with Socks 4 it works for sure.

It's possible that uTorrent is not using the Socks 4 proxy for its DHT, just bypassing it and using the regular IPv4 internet connection.

Seeker2 avatar Jun 09 '21 12:06 Seeker2

this may be of help finding DHT download history for an ip. https://iknowwhatyoudownload.com/en/contacts/

zero77 avatar Jun 09 '21 13:06 zero77

@arvidn Okay. Give me the name of a proxy server (proxy server for Windows 10) which I can use with libtorrent? I tried WinGate. It doesn't work.

master255 avatar Jun 09 '21 20:06 master255

@arvidn This is a very big and important problem that needs to be dealt with more as a priority. It greatly affects propagation. Especially in Russia and similar countries. Don't tell me we have to invent a proxy server specifically for your library.

master255 avatar Jun 21 '21 21:06 master255

please feel free to trouble shoot. You can collect a wireshark dump of libtorrent talking to a proxy and some other program talking to it (using UDP). That would be a good start to see if libtorrent speaks socks5 incorrectly.

arvidn avatar Jun 21 '21 21:06 arvidn

I confirm that DHT connections do not work through the SOCKS5 proxy. Tried client_test, does not work.

While investigating DNS leak in Tribler, my Wireshark dumps of client_test are showed that Libtorrent requests the DHT bootstrap servers directly (w/o proxy), no matter the proxy settings.

Here is the test line I used (using ssh to a VPS as the proxy): client_test --enable_upnp=0 --proxy_type=2 --proxy_port=12666 --proxy_hostnames=1 --proxy_peer_connections=1 --proxy_tracker_connections=1 --proxy_hostname=127.0.0.1 --force_proxy=1 ./<some.torrent>

Also, even with --enable_dht=0, the client still issues DNS lookups for DHT bootstrap servers.

ichorid avatar Jun 22 '21 13:06 ichorid

@ichorid would you mind creating a separate ticket for the DNS leak?

Have you seen any other application successfully tunnel UDP via an ssh proxy? (if so, I would be interested in seeing wireshark dumps)

arvidn avatar Jun 22 '21 13:06 arvidn

@ichorid would you mind creating a separate ticket for the DNS leak?

You're right! OpenSSH SOCKS5 implementation does not support UDP forwarding!

(I've checked their source code, not signs of UDP support)

I'll try to check it with Dante or something.

ichorid avatar Jun 24 '21 15:06 ichorid

Please just make an option to bypass the proxy for DHT. Because the most time proxy is needed just to reach trackers (blocked by ISP), not for the DHT nodes.

HanabishiRecca avatar Jul 12 '21 14:07 HanabishiRecca

FWIW: you can consider using docker to do complex networking management, if it fits your use case. It's a nice way to do app-specific networking that isn't normally possible with e.g. iptables.

I end up running torrent clients with docker run --net <vpn container>, which helps guarantee that all traffic from the client (and only that traffic) goes over the vpn.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Oct 11 '21 05:10 stale[bot]

Still not working

master255 avatar Oct 11 '21 11:10 master255

Could be actually related to #6512

ichorid avatar Oct 11 '21 21:10 ichorid

double

master255 avatar Nov 12 '21 16:11 master255

@arvidn Are there plans to fix it?

proninyaroslav avatar Feb 16 '22 07:02 proninyaroslav

@proninyaroslav I have not observed the problem. Do you have a wireshark dump demonstrating the issue? Ideally I would have two wireshark dumps, the second one demonstrating the proxy successfully forwarding UDP packets (because it's so common among socks5 proxies not to support that)

arvidn avatar Feb 16 '22 07:02 arvidn

I make some dumps here https://github.com/arvidn/libtorrent/issues/4078

proninyaroslav avatar Feb 16 '22 07:02 proninyaroslav

I second the suggestion for adding option to use direct connection for DHT even if proxy is used for connection to tracker/announcer.

Rollinnn avatar Apr 18 '22 16:04 Rollinnn

I confirm that DHT nodes are always 0 when using Shadowsocks SOCKS5 proxy(in UDP mode) for tracker connections.

Alexey104 avatar Jun 07 '22 00:06 Alexey104