px icon indicating copy to clipboard operation
px copied to clipboard

Published 20 hours ago • verified publisher icon genotrance

px 0.9.0 responds with "URL malformed" on macOS

Open JamieEdge opened this issue 1 year ago • 8 comments

When running px 0.9.0 using Python 3.11.7 on macOS 14.3.1, requests are responded to with a "URL malformed" error.

px starts successfully, as shown below.

$ px --proxy=192.0.2.1:3128
Serving at 127.0.0.1:3128 proc MainProcess

Upon requesting a HTTP URL, the following behaviour is observed.

$ curl -v --proxy http://127.0.0.1:3128 http://example.com
*   Trying 127.0.0.1:3128...
* Connected to 127.0.0.1 (127.0.0.1) port 3128
> GET http://example.com/ HTTP/1.1
> Host: example.com
> User-Agent: curl/8.4.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 400 3; URL malformed
< Server: BaseHTTP/0.6 Python/3.11.7
< Date: Wed, 14 Feb 2024 11:38:34 GMT
< Connection: close
< Content-Type: text/html;charset=utf-8
< Content-Length: 348
<
<!DOCTYPE HTML>
<html lang="en">
    <head>
        <meta charset="utf-8">
        <title>Error response</title>
    </head>
    <body>
        <h1>Error response</h1>
        <p>Error code: 400</p>
        <p>Message: 3; URL malformed.</p>
        <p>Error code explanation: 400 - Bad request syntax or unsupported method.</p>
    </body>
</html>

Upon requesting a HTTPS URL, the following behaviour is observed.

$ curl -v --proxy http://127.0.0.1:3128 https://example.com
*   Trying 127.0.0.1:3128...
* Connected to 127.0.0.1 (127.0.0.1) port 3128
* CONNECT tunnel: HTTP/1.1 negotiated
* allocate connect buffer
* Establish HTTP proxy tunnel to example.com:443
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> User-Agent: curl/8.4.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 503 3; URL malformedFailed to get active socket: 48, 0;
< Server: BaseHTTP/0.6 Python/3.11.7
< Date: Wed, 14 Feb 2024 11:38:26 GMT
< Connection: close
< Content-Type: text/html;charset=utf-8
< Content-Length: 400
<
* CONNECT tunnel failed, response 503
* Closing connection
curl: (56) CONNECT tunnel failed, response 503

No errors are written to stderr/stdout by px, however the following debug logs appear to be relevant.

MainProcess: Thread_0: 1707911158: /do_curl/bridge/dprint: 4fdc9181de539d7ebbb46a2942a8fac305c0a558: Setting up bridge
MainProcess: Thread_0: 1707911158: /do_curl/set_headers/dprint: 4fdc9181de539d7ebbb46a2942a8fac305c0a558: Adding header => Host: example.com
MainProcess: Thread_0: 1707911158: /set_headers/set_useragent/dprint: 4fdc9181de539d7ebbb46a2942a8fac305c0a558: Setting user agent to curl/8.4.0
MainProcess: Thread_0: 1707911158: /do_curl/set_headers/dprint: 4fdc9181de539d7ebbb46a2942a8fac305c0a558: Adding header => Accept: */*
MainProcess: Thread_0: 1707911158: /do_curl/set_headers/dprint: 4fdc9181de539d7ebbb46a2942a8fac305c0a558: Adding header => Proxy-Connection: Keep-Alive
MainProcess: Thread_0: 1707911158: /do_curl/set_headers/dprint: 4fdc9181de539d7ebbb46a2942a8fac305c0a558: Setting headers
MainProcess: Thread_0: 1707911158: /do/add/dprint: 4fdc9181de539d7ebbb46a2942a8fac305c0a558: Handles = 0
MainProcess: Thread_0: 1707911158: /add/_add_handle/dprint: 4fdc9181de539d7ebbb46a2942a8fac305c0a558: Add handle
MainProcess: Thread_0: 1707911158: /add/_add_handle/dprint: 4fdc9181de539d7ebbb46a2942a8fac305c0a558: Added handle
MainProcess: Thread_0: 1707911158: /do_GET/do_curl/dprint: 4fdc9181de539d7ebbb46a2942a8fac305c0a558: Connection failed: 3; URL malformed
MainProcess: Thread_0: 1707911158: /log_error/log_message/dprint: code 400, message 3; URL malformed
MainProcess: Thread_0: 1707911158: /log_request/log_message/dprint: "GET http://example.com/ HTTP/1.1" 400 -
MainProcess: Thread_0: 1707911158: /remove/_remove_handle/dprint: 4fdc9181de539d7ebbb46a2942a8fac305c0a558: Remove handle: 3; URL malformed

JamieEdge avatar Feb 14 '24 11:02 JamieEdge

Can you please share the full log for both the http and https requests, including the libcurl version info at the top? Also, is this an x86_64 Mac or the new M1/2? I don't have a machine to test the newer models.

genotrance avatar Feb 14 '24 16:02 genotrance

Thanks for looking into this.

Sure, please find the full logs below.

MainProcess: MainThread: 1708272826: /__init__/print_curl_version/dprint: libcurl/8.4.0 SecureTransport (LibreSSL/3.3.6) zlib/1.2.12 nghttp2/1.58.0
MainProcess: MainThread: 1708272826: /__init__/print_curl_version/dprint: CURL_VERSION_SSL: True
MainProcess: MainThread: 1708272826: /__init__/print_curl_version/dprint: CURL_VERSION_SSPI: False
MainProcess: MainThread: 1708272826: /__init__/print_curl_version/dprint: CURL_VERSION_SPNEGO: True
MainProcess: MainThread: 1708272826: /__init__/print_curl_version/dprint: CURL_VERSION_GSSAPI: True
MainProcess: MainThread: 1708272826: /__init__/print_curl_version/dprint: CURL_VERSION_GSSNEGOTIATE: False
MainProcess: MainThread: 1708272826: /__init__/print_curl_version/dprint: CURL_VERSION_KERBEROS5: True
MainProcess: MainThread: 1708272826: /__init__/print_curl_version/dprint: CURL_VERSION_NTLM: True
MainProcess: MainThread: 1708272826: /__init__/print_curl_version/dprint: CURL_VERSION_NTLM_WB: True
MainProcess: MainThread: 1708272826: /__init__/print_curl_version/dprint: Host: x86_64-apple-darwin23.0
Serving at 127.0.0.1:3128 proc MainProcess
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: proxy:server = 192.0.2.1:3128
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: proxy:pac =
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: proxy:pac_encoding = utf-8
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: proxy:port = 3128
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: proxy:listen = 127.0.0.1
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: proxy:gateway = 0
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: proxy:hostonly = 0
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: proxy:allow = *.*.*.*
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: proxy:noproxy =
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: proxy:useragent =
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: proxy:username =
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: proxy:auth =
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: settings:log = 1
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: settings:workers = 2
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: settings:threads = 32
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: settings:idle = 30
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: settings:socktimeout = 20.0
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: settings:proxyreload = 60
MainProcess: MainThread: 1708272826: /run_pool/print_banner/dprint: settings:foreground = 0
MainProcess: MainThread: 1708272850: /_handle_request_noblock/verify_request/dprint: Client address: 127.0.0.1
MainProcess: Thread_0: 1708272850: /do_curl/do_client_auth/dprint: No client authentication required
MainProcess: Thread_0: 1708272850: /do_curl/__init__/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: New curl instance
MainProcess: Thread_0: 1708272850: /__init__/_setup/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: GET http://example.com/ using HTTP/1.1
MainProcess: Thread_0: 1708272850: /do_GET/do_curl/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Path = http://example.com/
MainProcess: Thread_0: 1708272850: /find_proxy_for_url/get_netloc/dprint: netloc = ('example.com', 80), path = /
MainProcess: Thread_0: 1708272850: /do_curl/get_destination/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Proxy = [('192.0.2.1', 3128)]
MainProcess: Thread_0: 1708272850: /do_GET/do_curl/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Configuring proxy settings
MainProcess: Thread_0: 1708272850: /do_curl/set_curl_auth/dprint: SSPI not available and no username configured - no auth
MainProcess: Thread_0: 1708272850: /do_curl/bridge/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Setting up bridge
MainProcess: Thread_0: 1708272850: /do_curl/set_headers/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Adding header => Host: example.com
MainProcess: Thread_0: 1708272850: /set_headers/set_useragent/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Setting user agent to curl/8.4.0
MainProcess: Thread_0: 1708272850: /do_curl/set_headers/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Adding header => Accept: */*
MainProcess: Thread_0: 1708272850: /do_curl/set_headers/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Adding header => Proxy-Connection: Keep-Alive
MainProcess: Thread_0: 1708272850: /do_curl/set_headers/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Setting headers
MainProcess: Thread_0: 1708272850: /do/add/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Handles = 0
MainProcess: Thread_0: 1708272850: /add/_add_handle/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Add handle
MainProcess: Thread_0: 1708272850: /add/_add_handle/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Added handle
MainProcess: Thread_0: 1708272850: /do_GET/do_curl/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Connection failed: 3; URL malformed
MainProcess: Thread_0: 1708272850: /log_error/log_message/dprint: code 400, message 3; URL malformed
MainProcess: Thread_0: 1708272850: /log_request/log_message/dprint: "GET http://example.com/ HTTP/1.1" 400 -
MainProcess: Thread_0: 1708272850: /remove/_remove_handle/dprint: e8f8429f88128bce399ea783553bec4c2b9f6e70: Remove handle: 3; URL malformed
MainProcess: MainThread: 1708272855: /_handle_request_noblock/verify_request/dprint: Client address: 127.0.0.1
MainProcess: Thread_0: 1708272855: /do_curl/do_client_auth/dprint: No client authentication required
MainProcess: Thread_0: 1708272855: /do_curl/__init__/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: New curl instance
MainProcess: Thread_0: 1708272855: /__init__/_setup/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: CONNECT example.com:443 using HTTP/1.1
MainProcess: Thread_0: 1708272855: /_setup/set_tunnel/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: HTTP proxy tunneling = True
MainProcess: Thread_0: 1708272855: /do_CONNECT/do_curl/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Path = example.com:443
MainProcess: Thread_0: 1708272855: /find_proxy_for_url/get_netloc/dprint: netloc = ('example.com', 443), path = /
MainProcess: Thread_0: 1708272855: /do_curl/get_destination/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Proxy = [('192.0.2.1', 3128)]
MainProcess: Thread_0: 1708272855: /do_CONNECT/do_curl/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Configuring proxy settings
MainProcess: Thread_0: 1708272855: /set_proxy/set_tunnel/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: HTTP proxy tunneling = False
MainProcess: Thread_0: 1708272855: /do_curl/set_curl_auth/dprint: SSPI not available and no username configured - no auth
MainProcess: Thread_0: 1708272855: /do_curl/set_headers/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Adding header => Host: example.com:443
MainProcess: Thread_0: 1708272855: /set_headers/set_useragent/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Setting user agent to curl/8.4.0
MainProcess: Thread_0: 1708272855: /do_curl/set_headers/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Adding header => Proxy-Connection: Keep-Alive
MainProcess: Thread_0: 1708272855: /do_curl/set_headers/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Delaying headers
MainProcess: Thread_0: 1708272855: /do/add/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Handles = 0
MainProcess: Thread_0: 1708272855: /add/_add_handle/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Add handle
MainProcess: Thread_0: 1708272855: /add/_add_handle/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Added handle
MainProcess: Thread_0: 1708272855: /do_curl/do/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Getting active socket
MainProcess: Thread_0: 1708272855: /do_curl/do/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Failed to get active socket: 48, 0
MainProcess: Thread_0: 1708272855: /do_CONNECT/do_curl/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Connection failed: 3; URL malformedFailed to get active socket: 48, 0;
MainProcess: Thread_0: 1708272855: /log_error/log_message/dprint: code 503, message 3; URL malformedFailed to get active socket: 48, 0;
MainProcess: Thread_0: 1708272855: /log_request/log_message/dprint: "CONNECT example.com:443 HTTP/1.1" 503 -
MainProcess: Thread_0: 1708272855: /remove/_remove_handle/dprint: 5675f785ad4f8ad301ce1ef9f0979eaec3fedbe2: Remove handle: 3; URL malformedFailed to get active socket: 48, 0;

I'm using an Apple Silicon device, rather than an Intel-based one. The version of curl included with macOS appears to be built for x86_64 instead of arm64 though.

$ curl --version
curl 8.4.0 (x86_64-apple-darwin23.0) libcurl/8.4.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.58.0
Release-Date: 2023-10-11
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets

I'd be happy to contribute a fix however I'm not sure where to start with this given that I can't see any malformed URLs.

JamieEdge avatar Feb 18 '24 16:02 JamieEdge

Adding a data point about possible libcurl issue? When I use px with another client like httpie, it works fine:

http --version
3.2.2

http -I --proxy http://localhost:3130 https://google.com --follow -h
HTTP/1.1 200 OK

I am on M1 Sonoma 14.3 with below details:

curl --version
curl 8.4.0 (x86_64-apple-darwin23.0) libcurl/8.4.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.58.0
Release-Date: 2023-10-11
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets

Px info:

config

[proxy]
port = 3130
hostonly = 1

Startup

px --proxy <corportate-proxy-ip>:8080 --verbose
MainProcess: MainThread: 1711471397: /main/parse_config/dprint: Host-only mode - overriding 'listen' and binding to all interfaces
MainProcess: MainThread: 1711471397: /main/parse_config/dprint: Px will automatically restrict access to host interfaces
MainProcess: MainThread: 1711471397: /main/parse_config/dprint: Removing default 'allow' everyone rule
MainProcess: MainThread: 1711471397: /__init__/print_curl_version/dprint: libcurl/8.4.0 SecureTransport (LibreSSL/3.3.6) zlib/1.2.12 nghttp2/1.58.0
MainProcess: MainThread: 1711471397: /__init__/print_curl_version/dprint: CURL_VERSION_SSL: True
MainProcess: MainThread: 1711471397: /__init__/print_curl_version/dprint: CURL_VERSION_SSPI: False
MainProcess: MainThread: 1711471397: /__init__/print_curl_version/dprint: CURL_VERSION_SPNEGO: True
MainProcess: MainThread: 1711471397: /__init__/print_curl_version/dprint: CURL_VERSION_GSSAPI: True
MainProcess: MainThread: 1711471397: /__init__/print_curl_version/dprint: CURL_VERSION_GSSNEGOTIATE: False
MainProcess: MainThread: 1711471397: /__init__/print_curl_version/dprint: CURL_VERSION_KERBEROS5: True
MainProcess: MainThread: 1711471397: /__init__/print_curl_version/dprint: CURL_VERSION_NTLM: True
MainProcess: MainThread: 1711471397: /__init__/print_curl_version/dprint: CURL_VERSION_NTLM_WB: True
MainProcess: MainThread: 1711471397: /__init__/print_curl_version/dprint: Host: x86_64-apple-darwin23.0
Serving at :3130 proc MainProcess
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: proxy:port = 3130
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: proxy:hostonly = 1
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: proxy:server = 1
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: proxy:pac =
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: proxy:pac_encoding = utf-8
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: proxy:listen =
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: proxy:gateway = 0
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: proxy:allow =
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: proxy:noproxy =
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: proxy:useragent =
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: proxy:username =
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: proxy:auth =
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: settings:workers = 2
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: settings:threads = 32
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: settings:idle = 30
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: settings:socktimeout = 20.0
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: settings:proxyreload = 60
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: settings:foreground = 1
MainProcess: MainThread: 1711471397: /run_pool/print_banner/dprint: settings:log = 4

HTTPS logs:

curl --proxy 'http://localhost:3130' https://google.com
curl: (56) CONNECT tunnel failed, response 503

MainProcess: MainThread: 1711471618: /_handle_request_noblock/verify_request/dprint: Client address: 127.0.0.1
MainProcess: MainThread: 1711471618: /_handle_request_noblock/verify_request/dprint: Host-only IP allowed
MainProcess: Thread_0: 1711471618: /do_curl/do_client_auth/dprint: No client authentication required
MainProcess: Thread_0: 1711471618: /do_curl/__init__/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: New curl instance
MainProcess: Thread_0: 1711471618: /__init__/_setup/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: CONNECT google.com:443 using HTTP/1.1
MainProcess: Thread_0: 1711471618: /_setup/set_tunnel/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: HTTP proxy tunneling = True
MainProcess: Thread_0: 1711471618: /do_CONNECT/do_curl/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Path = google.com:443
MainProcess: Thread_0: 1711471618: /find_proxy_for_url/get_netloc/dprint: netloc = ('google.com', 443), path = /
MainProcess: Thread_0: 1711471618: /do_curl/get_destination/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Proxy = [('1', 80)]
MainProcess: Thread_0: 1711471618: /do_CONNECT/do_curl/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Configuring proxy settings
MainProcess: Thread_0: 1711471618: /set_proxy/set_tunnel/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: HTTP proxy tunneling = False
MainProcess: Thread_0: 1711471618: /do_curl/set_curl_auth/dprint: SSPI not available and no username configured - no auth
MainProcess: Thread_0: 1711471618: /do_curl/set_headers/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Adding header => Host: google.com:443
MainProcess: Thread_0: 1711471618: /set_headers/set_useragent/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Setting user agent to curl/8.4.0
MainProcess: Thread_0: 1711471618: /do_curl/set_headers/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Adding header => Proxy-Connection: Keep-Alive
MainProcess: Thread_0: 1711471618: /do_curl/set_headers/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Delaying headers
MainProcess: Thread_0: 1711471618: /do/add/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Handles = 0
MainProcess: Thread_0: 1711471618: /add/_add_handle/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Add handle
MainProcess: Thread_0: 1711471618: /add/_add_handle/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Added handle
MainProcess: Thread_0: 1711471618: /do_curl/do/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Getting active socket
MainProcess: Thread_0: 1711471618: /do_curl/do/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Failed to get active socket: 48, 0
MainProcess: Thread_0: 1711471618: /do_CONNECT/do_curl/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Connection failed: 3; URL malformedFailed to get active socket: 48, 0;
MainProcess: Thread_0: 1711471618: /log_error/log_message/dprint: code 503, message 3; URL malformedFailed to get active socket: 48, 0;
MainProcess: Thread_0: 1711471618: /log_request/log_message/dprint: "CONNECT google.com:443 HTTP/1.1" 503 -
MainProcess: Thread_0: 1711471618: /remove/_remove_handle/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Remove handle: 3; URL malformedFailed to get active socket: 48, 0;
MainProcess: MainThread: 1711471624: /_handle_request_noblock/verify_request/dprint: Client address: 127.0.0.1
MainProcess: MainThread: 1711471624: /_handle_request_noblock/verify_request/dprint: Host-only IP allowed
MainProcess: Thread_0: 1711471624: /do_curl/do_client_auth/dprint: No client authentication required
MainProcess: Thread_0: 1711471624: /do_curl/__init__/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: New curl instance
MainProcess: Thread_0: 1711471624: /__init__/_setup/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: CONNECT google.com:443 using HTTP/1.1
MainProcess: Thread_0: 1711471624: /_setup/set_tunnel/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: HTTP proxy tunneling = True
MainProcess: Thread_0: 1711471624: /do_CONNECT/do_curl/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Path = google.com:443
MainProcess: Thread_0: 1711471624: /find_proxy_for_url/get_netloc/dprint: netloc = ('google.com', 443), path = /
MainProcess: Thread_0: 1711471624: /do_curl/get_destination/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Proxy = [('1', 80)]
MainProcess: Thread_0: 1711471624: /do_CONNECT/do_curl/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Configuring proxy settings
MainProcess: Thread_0: 1711471624: /set_proxy/set_tunnel/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: HTTP proxy tunneling = False
MainProcess: Thread_0: 1711471624: /do_curl/set_curl_auth/dprint: SSPI not available and no username configured - no auth
MainProcess: Thread_0: 1711471624: /do_curl/set_headers/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Adding header => Host: google.com:443
MainProcess: Thread_0: 1711471624: /set_headers/set_useragent/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Setting user agent to curl/8.4.0
MainProcess: Thread_0: 1711471624: /do_curl/set_headers/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Adding header => Proxy-Connection: Keep-Alive
MainProcess: Thread_0: 1711471624: /do_curl/set_headers/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Delaying headers
MainProcess: Thread_0: 1711471624: /do/add/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Handles = 0
MainProcess: Thread_0: 1711471624: /add/_add_handle/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Add handle
MainProcess: Thread_0: 1711471624: /add/_add_handle/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Added handle
MainProcess: Thread_0: 1711471624: /do_curl/do/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Getting active socket
MainProcess: Thread_0: 1711471624: /do_curl/do/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Failed to get active socket: 48, 0
MainProcess: Thread_0: 1711471624: /do_CONNECT/do_curl/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Connection failed: 3; URL malformedFailed to get active socket: 48, 0;
MainProcess: Thread_0: 1711471624: /log_error/log_message/dprint: code 503, message 3; URL malformedFailed to get active socket: 48, 0;
MainProcess: Thread_0: 1711471624: /log_request/log_message/dprint: "CONNECT google.com:443 HTTP/1.1" 503 -
MainProcess: Thread_0: 1711471624: /remove/_remove_handle/dprint: 2cb1b69ede760e79fbf71613f360cee7e2670cc4: Remove handle: 3; URL malformedFailed to get active socket: 48, 0;

HTTP request

 curl --proxy 'http://localhost:3130' http://google.com

<!DOCTYPE HTML>
<html lang="en">
    <head>
        <meta charset="utf-8">
        <title>Error response</title>
    </head>
    <body>
        <h1>Error response</h1>
        <p>Error code: 400</p>
        <p>Message: 3; URL malformed.</p>
        <p>Error code explanation: 400 - Bad request syntax or unsupported method.</p>
    </body>
</html>

MainProcess: MainThread: 1711471785: /_handle_request_noblock/verify_request/dprint: Client address: 127.0.0.1
MainProcess: MainThread: 1711471785: /_handle_request_noblock/verify_request/dprint: Host-only IP allowed
MainProcess: Thread_0: 1711471785: /do_curl/do_client_auth/dprint: No client authentication required
MainProcess: Thread_0: 1711471785: /do_curl/__init__/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: New curl instance
MainProcess: Thread_0: 1711471785: /__init__/_setup/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: GET http://google.com/ using HTTP/1.1
MainProcess: Thread_0: 1711471785: /do_GET/do_curl/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Path = http://google.com/
MainProcess: Thread_0: 1711471785: /find_proxy_for_url/get_netloc/dprint: netloc = ('google.com', 80), path = /
MainProcess: Thread_0: 1711471785: /do_curl/get_destination/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Proxy = [('1', 80)]
MainProcess: Thread_0: 1711471785: /do_GET/do_curl/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Configuring proxy settings
MainProcess: Thread_0: 1711471785: /do_curl/set_curl_auth/dprint: SSPI not available and no username configured - no auth
MainProcess: Thread_0: 1711471785: /do_curl/bridge/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Setting up bridge
MainProcess: Thread_0: 1711471785: /do_curl/set_headers/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Adding header => Host: google.com
MainProcess: Thread_0: 1711471785: /set_headers/set_useragent/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Setting user agent to curl/8.4.0
MainProcess: Thread_0: 1711471785: /do_curl/set_headers/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Adding header => Accept: */*
MainProcess: Thread_0: 1711471785: /do_curl/set_headers/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Adding header => Proxy-Connection: Keep-Alive
MainProcess: Thread_0: 1711471785: /do_curl/set_headers/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Setting headers
MainProcess: Thread_0: 1711471785: /do/add/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Handles = 0
MainProcess: Thread_0: 1711471785: /add/_add_handle/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Add handle
MainProcess: Thread_0: 1711471785: /add/_add_handle/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Added handle
MainProcess: Thread_0: 1711471785: /do_GET/do_curl/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Connection failed: 3; URL malformed
MainProcess: Thread_0: 1711471785: /log_error/log_message/dprint: code 400, message 3; URL malformed
MainProcess: Thread_0: 1711471785: /log_request/log_message/dprint: "GET http://google.com/ HTTP/1.1" 400 -
MainProcess: Thread_0: 1711471785: /remove/_remove_handle/dprint: 053dd22528d83912b0b9f8156ced668967e3de66: Remove handle: 3; URL malformed

I can confirm from Network capture there's no traffic going out so it seems to fail within px itself

SohamJ avatar Mar 26 '24 16:03 SohamJ

To try to figure out the cause, I attached a debugger to a working install on Linux and a non-working install on macOS.

The arguments passed to functions in the libcurl package that I inspected (particularly easy_setopt) appear to match. As a result, I think that the issue lies somewhere between px/libcurl/_easy.py, px/libcurl/_platform/_osx/__init__.py and libcurl.4.dylib.

The following minimal program, as executed from the px directory, encounters the same error.

import libcurl

easy = libcurl.easy_init()

libcurl.easy_setopt(easy, libcurl.CURLOPT_VERBOSE, True)
libcurl.easy_setopt(easy, libcurl.CURLOPT_URL, 'http://example.com/'.encode('utf-8'))

err = libcurl.easy_perform(easy)

if err == libcurl.CURLE_OK:
    print('Success')
else:
    print('Error: ' + str(err))

libcurl.easy_cleanup(easy)

To verify that libcurl.4.dylib is working correctly, I compiled and successfully ran a simple C program that uses the same functions.

I think that the issue is not just related to strings. For example, the _debug_callback function does not appear to be called, as demonstrated by the lack of libcurl messages in the logs shared.

Curiously, I found no difference between curl and httpie. In my tests, both resulted in the same error when handling a GET request for http://example.com/.

JamieEdge avatar Apr 07 '24 15:04 JamieEdge

Probably related : px --test fails

$ px --debug --proxy=10.161.163.100:8080 --test 
MainProcess: MainThread: 1716560926: /__init__/print_curl_version/dprint: libcurl/8.6.0 SecureTransport (LibreSSL/3.3.6) zlib/1.2.12 nghttp2/1.61.0
MainProcess: MainThread: 1716560926: /__init__/print_curl_version/dprint: CURL_VERSION_SSL: True
MainProcess: MainThread: 1716560926: /__init__/print_curl_version/dprint: CURL_VERSION_SSPI: False
MainProcess: MainThread: 1716560926: /__init__/print_curl_version/dprint: CURL_VERSION_SPNEGO: True
MainProcess: MainThread: 1716560926: /__init__/print_curl_version/dprint: CURL_VERSION_GSSAPI: True
MainProcess: MainThread: 1716560926: /__init__/print_curl_version/dprint: CURL_VERSION_GSSNEGOTIATE: False
MainProcess: MainThread: 1716560926: /__init__/print_curl_version/dprint: CURL_VERSION_KERBEROS5: True
MainProcess: MainThread: 1716560926: /__init__/print_curl_version/dprint: CURL_VERSION_NTLM: True
MainProcess: MainThread: 1716560926: /__init__/print_curl_version/dprint: CURL_VERSION_NTLM_WB: True
MainProcess: MainThread: 1716560926: /__init__/print_curl_version/dprint: Host: x86_64-apple-darwin23.0
Serving at 127.0.0.1:3128 proc MainProcess
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: proxy:server = 10.161.163.100:8080
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: proxy:pac = 
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: proxy:pac_encoding = utf-8
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: proxy:port = 3128
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: proxy:listen = 127.0.0.1
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: proxy:gateway = 0
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: proxy:hostonly = 0
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: proxy:allow = *.*.*.*
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: proxy:noproxy = 
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: proxy:useragent = 
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: proxy:username = 
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: proxy:auth = 
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: settings:workers = 1
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: settings:threads = 32
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: settings:idle = 30
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: settings:socktimeout = 20.0
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: settings:proxyreload = 60
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: settings:foreground = 0
MainProcess: MainThread: 1716560926: /run_pool/print_banner/dprint: settings:log = 1
MainProcess: Thread-1 (queryall): 1716560926: /query/__init__/dprint: 63540ceef23166b8767df65d2e4f4bcc5937522b: New curl instance
MainProcess: MainThread: 1716560926: /_handle_request_noblock/verify_request/dprint: Client address: 127.0.0.1
MainProcess: Thread-1 (queryall): 1716560926: /__init__/_setup/dprint: 63540ceef23166b8767df65d2e4f4bcc5937522b: GET http://httpbin.org/get using HTTP/1.1
MainProcess: Thread-1 (queryall): 1716560926: /query/set_curl_auth/dprint: 63540ceef23166b8767df65d2e4f4bcc5937522b: Skipping proxy authentication
MainProcess: Thread-1 (queryall): 1716560926: /query/buffer/dprint: 63540ceef23166b8767df65d2e4f4bcc5937522b: Setting up buffers for bridge
MainProcess: Thread-1 (queryall): 1716560926: /buffer/bridge/dprint: 63540ceef23166b8767df65d2e4f4bcc5937522b: Setting up bridge
MainProcess: Thread-1 (queryall): 1716560926: /query/set_useragent/dprint: 63540ceef23166b8767df65d2e4f4bcc5937522b: Setting user agent to mcurl v0.9.2
MainProcess: Thread-1 (queryall): 1716560926: /query/perform/dprint: 63540ceef23166b8767df65d2e4f4bcc5937522b: Connection failed: 3; 

Testing GET http://httpbin.org/get
Failed with error 3

barijaona avatar May 24 '24 14:05 barijaona

I got same issue.

here's curl response.

Q604934@LSCN1062776 ~ % curl -v https://www.google.com
* Uses proxy env variable no_proxy == '<company_internal_domains>,localhost,127.0.0.1'
* Uses proxy env variable https_proxy == '127.0.0.1:3128'
*   Trying 127.0.0.1:3128...
* Connected to 127.0.0.1 (127.0.0.1) port 3128
* CONNECT tunnel: HTTP/1.1 negotiated
* allocate connect buffer
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: curl/8.6.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 503 3; URL malformedFailed to get active socket: 48, 0; 
< Server: BaseHTTP/0.6 Python/3.12.5
< Date: Thu, 15 Aug 2024 03:14:39 GMT
< Connection: close
< Content-Type: text/html;charset=utf-8
< Content-Length: 400
< 
* CONNECT tunnel failed, response 503
* Closing connection
curl: (56) CONNECT tunnel failed, response 503

here's px log

Q604934@LSCN1062776 bin % px --config=./px.ini
MainProcess: MainThread: 1723691668: /__init__/parse_noproxy/dprint: {'localhost', '.<company-domain>.net'}
MainProcess: MainThread: 1723691668: /__init__/print_curl_version/dprint: libcurl/8.6.0 SecureTransport (LibreSSL/3.3.6) zlib/1.2.12 nghttp2/1.61.0
MainProcess: MainThread: 1723691668: /__init__/print_curl_version/dprint: CURL_VERSION_SSL: True
MainProcess: MainThread: 1723691668: /__init__/print_curl_version/dprint: CURL_VERSION_SSPI: False
MainProcess: MainThread: 1723691668: /__init__/print_curl_version/dprint: CURL_VERSION_SPNEGO: True
MainProcess: MainThread: 1723691668: /__init__/print_curl_version/dprint: CURL_VERSION_GSSAPI: True
MainProcess: MainThread: 1723691668: /__init__/print_curl_version/dprint: CURL_VERSION_GSSNEGOTIATE: False
MainProcess: MainThread: 1723691668: /__init__/print_curl_version/dprint: CURL_VERSION_KERBEROS5: True
MainProcess: MainThread: 1723691668: /__init__/print_curl_version/dprint: CURL_VERSION_NTLM: True
MainProcess: MainThread: 1723691668: /__init__/print_curl_version/dprint: CURL_VERSION_NTLM_WB: True
MainProcess: MainThread: 1723691668: /__init__/print_curl_version/dprint: Host: x86_64-apple-darwin23.0
Serving at 127.0.0.1:3128 proc MainProcess
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: proxy:server = <company_proxy>:<port>
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: proxy:pac = https://<pac_domain>/proxy.pac
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: proxy:pac_encoding = utf-8
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: proxy:port = 3128
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: proxy:listen = 127.0.0.1
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: proxy:gateway = 0
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: proxy:hostonly = 0
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: proxy:allow = *.*.*.*
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: proxy:noproxy = .bmwgroup.net,.muc,localhost,127.0.0.1
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: proxy:useragent = 
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: proxy:username = <my-domain-account>
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: proxy:password = <mypassword>
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: proxy:auth = 
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: settings:workers = 2
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: settings:threads = 32
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: settings:idle = 30
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: settings:socktimeout = 20.0
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: settings:proxyreload = 60
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: settings:foreground = 0
MainProcess: MainThread: 1723691668: /run_pool/print_banner/dprint: settings:log = 4
MainProcess: MainThread: 1723691679: /_handle_request_noblock/verify_request/dprint: Client address: 127.0.0.1
MainProcess: Thread_0: 1723691679: /do_curl/do_client_auth/dprint: No client authentication required
MainProcess: Thread_0: 1723691679: /do_curl/__init__/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: New curl instance
MainProcess: Thread_0: 1723691679: /__init__/_setup/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: CONNECT www.google.com:443 using HTTP/1.1
MainProcess: Thread_0: 1723691679: /_setup/set_tunnel/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: HTTP proxy tunneling = True
MainProcess: Thread_0: 1723691679: /do_CONNECT/do_curl/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Path = www.google.com:443
MainProcess: Thread_0: 1723691679: /find_proxy_for_url/get_netloc/dprint: netloc = ('www.google.com', 443), path = /
MainProcess: Thread_0: 1723691679: /find_proxy_for_url/check_noproxy_for_netloc/dprint: Couldn't resolve host
MainProcess: Thread_0: 1723691679: /do_curl/get_destination/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Proxy = [('<company_proxy>', 443)]
MainProcess: Thread_0: 1723691679: /do_CONNECT/do_curl/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Configuring proxy settings
MainProcess: Thread_0: 1723691679: /do_curl/set_proxy/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Set noproxy to <company_internal_domains>
MainProcess: Thread_0: 1723691679: /set_proxy/set_tunnel/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: HTTP proxy tunneling = False
MainProcess: Thread_0: 1723691679: /set_curl_auth/set_auth/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Blank password for user
MainProcess: Thread_0: 1723691679: /set_curl_auth/set_auth/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Setting proxy auth mechanism to ANY
MainProcess: Thread_0: 1723691679: /set_auth/set_tunnel/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: HTTP proxy tunneling = True
MainProcess: Thread_0: 1723691679: /do_curl/set_headers/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Adding header => Host: www.google.com:443
MainProcess: Thread_0: 1723691679: /set_headers/set_useragent/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Setting user agent to curl/8.6.0
MainProcess: Thread_0: 1723691679: /do_curl/set_headers/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Skipping header =!> Proxy-Connection: Keep-Alive
MainProcess: Thread_0: 1723691679: /do_curl/set_headers/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Setting headers
MainProcess: Thread_0: 1723691679: /do/add/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Handles = 0
MainProcess: Thread_0: 1723691679: /add/_add_handle/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Add handle
MainProcess: Thread_0: 1723691679: /add/_add_handle/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Added handle
MainProcess: Thread_0: 1723691679: /do_curl/do/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Getting active socket
MainProcess: Thread_0: 1723691679: /do_curl/do/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Failed to get active socket: 48, 0
MainProcess: Thread_0: 1723691679: /do_CONNECT/do_curl/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Connection failed: 3; URL malformedFailed to get active socket: 48, 0; 
MainProcess: Thread_0: 1723691679: /log_error/log_message/dprint: code 503, message 3; URL malformedFailed to get active socket: 48, 0; 
MainProcess: Thread_0: 1723691679: /log_request/log_message/dprint: "CONNECT www.google.com:443 HTTP/1.1" 503 -
MainProcess: Thread_0: 1723691679: /remove/_remove_handle/dprint: 6d1dd9727669e07ae90451f1f136031b9d88bac0: Remove handle: 3; URL malformedFailed to get active socket: 48, 0;

another wired things is I set password in the ini file, log showes it loaded. But I can see 'Blank password for user' in the output log, looks like the password I set isn't working.

chengwhynot avatar Aug 15 '24 03:08 chengwhynot

Looks like we are not getting an active socket:

https://github.com/genotrance/px/blob/d40f03f51e17a6256aec861f78f2d62d92bde4b3/px/mcurl.py#L998-L1005

Error 48 = CURLE_UNKNOWN_OPTION but this is libcurl v8.6.0 and the feature was added in 7.45. I don't see why this should fail so will need additional debugging on libcurl side. It's possible that the new Macs get a wrong value for CURLINFO_ACTIVESOCKET.

https://github.com/genotrance/px/blob/d40f03f51e17a6256aec861f78f2d62d92bde4b3/px/libcurl/_curl.py#L2976

Meanwhile, Px is moving to pymcurl as the backend going forward. It will be great if that can be tested on the new Macs to see if this issue can be recreated. Definitely will try to test this case myself when porting is completed but any help will be appreciated.

@chengwhynot - password has to be stored in keyring or env var, not in the configuration file. Please see the documentation on that.

genotrance avatar Sep 11 '24 18:09 genotrance

Just to let you guys know: I have the same issue on my Mac (MacBook Air, M2, macOS 14.5):

MainProcess: Thread_0: 1726124894: /do_curl/set_headers/dprint: 5531a55ed452e842f5caf1eb07dfb1a19b1a5dab: Delaying headers
MainProcess: Thread_0: 1726124894: /do/add/dprint: 5531a55ed452e842f5caf1eb07dfb1a19b1a5dab: Handles = 0
MainProcess: Thread_0: 1726124894: /add/_add_handle/dprint: 5531a55ed452e842f5caf1eb07dfb1a19b1a5dab: Add handle
MainProcess: Thread_0: 1726124894: /add/_add_handle/dprint: 5531a55ed452e842f5caf1eb07dfb1a19b1a5dab: Added handle
MainProcess: Thread_0: 1726124894: /do_curl/do/dprint: 5531a55ed452e842f5caf1eb07dfb1a19b1a5dab: Getting active socket
MainProcess: Thread_0: 1726124894: /do_curl/do/dprint: 5531a55ed452e842f5caf1eb07dfb1a19b1a5dab: Failed to get active socket: 48, 0
MainProcess: Thread_0: 1726124894: /do_CONNECT/do_curl/dprint: 5531a55ed452e842f5caf1eb07dfb1a19b1a5dab: Connection failed: 3; URL malformedFailed to get active socket: 48, 0;

zisuu avatar Sep 12 '24 07:09 zisuu