"malformed HTTP response" with `-tlsi` enabled

[anykno]

httpx version:

1.3.9

Current Behavior:

httpx -duc -u https://www.cloudflare.com -debug -tlsi

randomly got output:

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   /
 / / / / /_/ /_/ /_/ /   |
/_/ /_/\__/\__/ .___/_/|_|
             /_/

		projectdiscovery.io

[INF] Dumped HTTP request for https://www.cloudflare.com

GET / HTTP/1.1
Host: www.cloudflare.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.9 (KHTML, like Gecko) Chrome/5.0.310.0 Safari/532.9
Accept-Charset: utf-8
Accept-Encoding: gzip

[DBG] Failed 'https://www.cloudflare.com': GET https://www.cloudflare.com giving up after 1 attempts: Get "https://www.cloudflare.com": net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x12\x04\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00d\x00\x04\x00\x01\x00\x00\x00\x05\x00\xff\xff\xff\x00\x00\x04\b\x00\x00\x00\x00\x00\x7f\xff\x00\x00\x00\x00\b\a\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01"

Expected Behavior:

get right probe output

Steps To Reproduce:

httpx -duc -u https://www.cloudflare.com -debug -tlsi

Anything else:

httpx -duc -u https://www.cloudflare.com -debug -tlsi -unsafe

can avoid this bug

[olearycrew]

Thanks for this report @anykno - our team will take a look. I'm able to reproduce this problem as well

[olearycrew]

It appears it could have something to do with the way Cloudflare is handling those redirections? My website which is hosted on Cloudflare exhibits the same behavior

httpx -duc -u https://www.boleary.dev -debug -tlsi

[Mzack9999]

I think this might be expected as -tlsi uses randomic cipher settings and this might interfere with ANLP and HTTP/2 narrower cipher suites selection, or simply might imitate some known JA3S already blocked. In fact if you retry multiple times, most of the requests pass through. While using -tlsi you can mitigate this by increasing the number of retries (for example-retries 5), so that different impersonations are tried before giving up.

[Mzack9999]

Closing as it seems working as expected - It will be reopened if needed