httpx icon indicating copy to clipboard operation
httpx copied to clipboard

add socks5h proxy support

Open liubin opened this issue 1 year ago • 5 comments

In some case socks5h is needed to resolve the hostname.

Summary

add socks5h proxy support.

Checklist

  • [x] I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
  • [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • [x] I've updated the documentation accordingly.

liubin avatar Apr 25 '24 01:04 liubin

socks5 are resolving hostnames in remote side by default (socks5h). Also, see httpx-socks where you can explicitly set rdns=False to locally resolving.

T-256 avatar Apr 25 '24 09:04 T-256

@T-256 Indeed I run into a complexity case that socks5 not work:

12:39:33-bin~/tmppp\ -> export https_proxy=socks5h://127.0.0.1:15666
12:39:49-bin~/tmppp\ -> curl -I https://twitter.com
HTTP/2 403 
date: Fri, 26 Apr 2024 04:39:51 GMT
server: tsa_p
x-powered-by: Express
... ...

12:39:51-bin~/tmppp\ -> export https_proxy=socks5://127.0.0.1:15666
12:39:56-bin~/tmppp\ -> curl -I https://twitter.com
curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to twitter.com:443 

And my code with requests only work with socks5h too, if set to socks5, it will get an error:

  File "/Users/bin/Library/Python/3.9/lib/python/site-packages/requests/adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: (MaxRetryError("SOCKSHTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /BAAI/bge-small-en-v1.5/resolve/main/config.json (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1129)')))"), '(Request ID: a0fdd8ed-dd99-4b38-bf53-47f133004e22)')

My code using both requests and httpx, it is not controlled by me, but be the packages that I'm using directly.

So let httpx to support socks5h is the simplest way for my case.

liubin avatar Apr 26 '24 04:04 liubin

Thanks, yep we probably should have socks5h proxy support.

Have you tested the above? I don't think it'll work, although you'll be able to pass socks5h:// through, support for it isn't actually implemented in httpcore yet.

Here's where you'll actually need to start on this...

lovelydinosaur avatar Apr 30 '24 09:04 lovelydinosaur

Hello @tomchristie! I am also running into this issue, i.e. lack of socks5h:// support. I cannot use socks5:// because it is interpreted differently by other tools (e.g. curl). I did test this PR + 1 line change in https://github.com/encode/httpcore/blob/master/httpcore/_models.py#L294 to add default port for sock5h and it works as expected - traffic is sent to the proxy without resolving hostname.

mateuszlitwin avatar Sep 13 '24 00:09 mateuszlitwin