curl-impersonate icon indicating copy to clipboard operation
curl-impersonate copied to clipboard

User agent client hints also sent in http

Open momala454 opened this issue 3 years ago • 0 comments

https://wicg.github.io/ua-client-hints/#security-privacy Client Hints will not be delivered to non-secure endpoints (see the secure transport requirements in Section 2.2.1 of [[RFC8942]](https://wicg.github.io/ua-client-hints/#biblio-rfc8942)).

The headers sec-ch-xxx must not be sent when the url is http://, only https:// but if i set CURL_IMPERSONATE=chrome98 env variable, it will always set those use agent headers even on http

	putenv('CURL_IMPERSONATE=chrome98');
	$ch = curl_init();
	curl_setopt($ch, CURLOPT_URL, 'http://headers.cf');
	curl_setopt($ch, CURLINFO_HEADER_OUT, 1);
	curl_setopt( $ch, CURLOPT_ENCODING, "" );
	curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, false );
	curl_setopt( $ch, CURLOPT_ENCODING, "" );
	curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
	//curl_setopt( $ch, CURLOPT_HTTPHEADER, ['Host: abc.com']);
	curl_setopt( $ch, CURLOPT_AUTOREFERER, true );
	

	
	
	echo curl_exec($ch);
	print_r(curl_getinfo($ch));

(take note that the website redirect to https version, but we are not following the redirect) Headers sent

GET / HTTP/1.1
Host: headers.cf
Connection: Upgrade, HTTP2-Settings
Upgrade: h2c
HTTP2-Settings: AAEAAQAAAAMAAAPoAAQAYAAAAAYABAAAjau_38Px
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="98", "Google Chrome";v="98"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Sec-Fetch-Site: none
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9

real headers sent by chrome :

GET / HTTP/1.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate
Accept-Language: fr
Connection: keep-alive
Host: headers.cf
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.88 Safari/537.36

you can also see some differences, like, only on curl I see "HTTP2-Settings" headers.

Also, there are lot of user agent hints headers. Once the website tells you that they want that you send more headers, chrome will send them : https://headers.cf/ Go to the website, a few headers are sent. Refresh the page, a lot of headers are sent. Curl-impersonate only send the minimal of the first request. The browser keep in cache the list of headers that the domain wants.

I don't know also if the website send an header of "Accept-CH" empty, if chrome doesn't send the 3 base headers sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="98", "Google Chrome";v="98" sec-ch-ua-mobile: ?0 sec-ch-ua-platform: "Windows" or if chrome still send them. If chrome doesn't send them, that's another way to detect a spoofed chrome. But this only works on the second request as the browser must know which headers the domain support

momala454 avatar Apr 19 '22 07:04 momala454