Protocol inference issue in -p flag + Enhancement suggestion for automatic TLS detection

[haprodev]

Hi Team, When using the -p flag in httpx to specify ports without explicitly defining protocols (e.g., -p 80,443), httpx defaults all ports to HTTPS. This results in incorrect assumptions — such as probing https://host:80 or http://host:443 — which are often invalid. More importantly, it leads to missed targets, especially in automated reconnaissance workflows where protocol accuracy is crucial. This report includes: A detailed description of the current behavior The impact on recon workflows Suggestions for protocol inference improvements An advanced enhancement proposal to make httpx more intelligent in real-world scanning scenarios.

Reproduction: httpx -u preprod-support-conversations.stripe.com -p 80,443 Output: https://preprod-support-conversations.stripe.com:80 https://preprod-support-conversations.stripe.com

Now try: httpx -u preprod-support-conversations.stripe.com -p http:80,https:443 Correct Output: http://preprod-support-conversations.stripe.com https://preprod-support-conversations.stripe.com

Real-World Impact This silent fallback can cause missed results during recon when scanning large scopes with ports like 8080, 8000, 4433, etc. Many targets expose SSL on non-standard ports. Without automatic protocol detection, httpx may fail to identify them properly unless the user provides specific scheme:port pairs manually. It also misleads automation scripts and causes false assumptions about protocol/port combinations.

Environment httpx version: 1.6.9 OS: Linux (Ubuntu) Target: Public test domain with known behavior Reproducible: Yes (on all ports when scheme is omitted)

Severity I wasn’t able to confidently assign a specific severity level to this issue, as its impact may vary depending on how httpx is integrated into different reconnaissance workflows. I’d greatly appreciate it if your team could evaluate and determine the appropriate severity based on your internal criteria. Final Note This improvement could significantly boost accuracy and reduce configuration friction for httpx in real-world recon scenarios, especially in automated pipelines and with complex infrastructure setups (e.g., wildcard subdomains on CDN/custom ports). Add-on Note: Over the past three years of using httpx extensively in production environments, this exact protocol-port misclassification has silently led to thousands of missed assets during reconnaissance. This issue becomes even more critical in large-scale asset discovery systems where the engine scans tens or hundreds of thousands of cyber assets — especially when many of them expose web interfaces on non-standard ports or use SSL on uncommon ports. I'm attaching a sample from our internal database showing a pattern of misidentified assets due to incorrect protocol assumptions, which directly impacted the accuracy of our scans. Part of the thousand incorrect results stored in the database:

Wrong and right results obtained while using -x all feature

As you can see, repeating the HTTPX process yields different results. However, if the protocol for the included ports is not explicitly specified, port 80 may be missed during the first run—even though the service is actually available. Service not recognized on first run: http://preprod-support-conversations.stripe.com

Please describe your feature request:

Recommendation 1: Smarter Port-to-Scheme Inference If the user provides a port without a scheme, httpx could smartly infer the protocol based on known defaults: Port | Inferred Scheme 80 | http 443 | https 8080 | http 8443 | https 4433 | https This mapping could be embedded in the tool as a lookup table. A warning could also be shown when the ports are ambiguous: [WRN] Port 80 was used without protocol; defaulting to HTTPS. Consider using 'http:80' or --auto-protocol for accuracy. Recommendation 2: Introduce --auto-protocol / --tls-detect Add a flag like --auto-protocol or --tls-detect that enables the following behavior: For each target+port: Attempt a lightweight TLS handshake first (like tls.Dial() with short timeout). If the handshake succeeds → treat the port as HTTPS. If it fails → fallback to HTTP probing. This would allow httpx to automatically identify HTTPS-enabled services on arbitrary ports, such as: 8081 9443 8888 10443 This approach can be implemented efficiently without affecting speed significantly, especially when used with goroutines and short timeouts. Optional UX Enhancements: Suggest http:80,https:443 syntax if users use common ports without scheme. Add verbose output for protocol decision-making (e.g., Inferred https://target:8443 from TLS handshake). Display clearer output when probes are skipped due to invalid protocol/port assumptions.

[Mzack9999]

@haprodev Thanks for opening the issue. By default httpx attempts already https protocol on the specified port, and then http. The particular target that you provided supports TLS on port 80 via H2 upgrade, in fact also curl works correctly:

% curl https://preprod-support-conversations.stripe.com:80 -v

Maybe what you are looking for is rather a flag that always try both http and https protocols and report both in case the port supports them?

[haprodev]

@haprodev Thanks for opening the issue. By default httpx attempts already https protocol on the specified port, and then http. The particular target that you provided supports TLS on port 80 via H2 upgrade, in fact also curl works correctly:

% curl https://preprod-support-conversations.stripe.com:80 -v Maybe what you are looking for is rather a flag that always try both http and https protocols and report both in case the port supports them?

Hello, @Mzack9999 Thank you for your detailed response. Yes, my goal is to simultaneously check both HTTP and HTTPS protocols across cyber assets to prevent issues such as the failure to detect active services running on port 80 and similar ports. As shown in the images above, there are several active web services that were not detected by httpx when using the HTTP protocol.

In this case, in httpx, there is no need to directly select the corresponding protocol, and we only need to select the ports for which we want both protocols to be checked.