bbs
bbs copied to clipboard
net4people
Google Chrome TLS extension permutation
I am reposting some information from @ValdikSS on NTC.
Since 2023-01-20, Google Chrome has started randomizing the order of extensions in the TLS Client Hello. Fastly has a post about it:
A first look at Chrome's TLS ClientHello permutation in the wild
A recently implemented feature in the Google Chrome browser permutes the set of TLS extensions sent in the ClientHello message, resulting in a different JA3 fingerprint with every new connection from the browser.
This change was made to prevent servers from expecting a certain fixed order of extensions, which could limit Chrome from making future modifications to its TLS implementation, with a goal of making the TLS ecosystem more robust. If you’ve heard of TLS GREASE, this might sound familiar. It’s noteworthy to mention the TLS Version 1.3 RFC specifies that extensions may be sent in any order, except for the pre_shared_key which, if present, MUST be sent as the last of the extensions.
This feature was initially to be released in Chrome version 110, but seemed to have taken effect in versions 108 and 109 as well, beginning soon after the feature was enabled by default in Chromium source code.
Contemporary versions of Chrome were 108, 109, and 110, though apparently the feature had been trialed back to 106.
Ready for Trial: TLS ClientHello extension permutation
Chrome Platform Status: Feature: TLS ClientHello extension permutation
The TLS RFC, Section 4.2, specifies that extensions MAY appear in any order. When multiple extensions of different types are present, the extensions MAY appear in any order, with the exception of "pre_shared_key" (Section 4.2.11) which MUST be the last extension in the ClientHello
This randomizes the order of extensions, subject to the pre_shared_key constraint in the RFC. This will reduce the risk of server and middleboxes fixating on the details of the current structure of ClientHello. This should make the TLS ecosystem more robust to changes.
Bug #1789436 is for doing the same thing in Firefox. It is closed with a commit since a couple of weeks ago, so I guess it will soon be happening in Firefox as well.
I see there is an issue in uTLS and a merge request (already merged) that adds a Chrome_106_Shuffle fingerprint.
@gaukas, how might randomized extension order be taken into account at https://tlsfingerprint.io/? With extension ordering being significant, I suppose Chrome fingerprints are going to be close to unique from now on. Maybe a secondary fingerprint that has all the extensions in a canonical order? Something like that could be useful for projects like naiveproxy that use a Chrome fingerprint.
adds a Chrome_106_Shuffle fingerprint.
We have observed the behavior that Chrome shuffling the order of their extensions (skipping GREASE and Padding/PSK) when it was still in A/B test, thanks to one of my personal devices being selected for test. So as a result we implemented the Extension Shuffling in Chrome_106_Shuffle of uTLS back then.
taken into account at https://tlsfingerprint.io/
Recently we have also updated tlsfingerprint.io to reflect this change and we call it Normalized Fingerprint, which integrates the website with the latest change to our database/tap (db/tap change by @sippejw). And it is computed by sorting the Extensions List before hashing (exactly as you have suggested here)
For those of you interested in the latest change, please refer to the screenshot down below. We prefix all Normalized Fingerprints with N/ to differentiate them from the collected, "Real" fingerprints.
For Google Chrome, it is very likely that the "real fp" (be018011859fa141 in this screenshot) wouldn't have a corresponding entry in the database. Instead, the normalized version could be used to inspect any related specs and user-agents.
This change to TLSFingerprint.io is not yet final since we are still updating the page template to include more details, therefore you won't be able to find the corresponding code change in refraction-networking/tls-fingerprint yet. But we would welcome any input on what information/statistics people would like us to include, given if we could record them on tap without violating our privacy policy or any regulations.
suppose Chrome fingerprints are going to be close to unique from now on
All the fingerprinting shops would also implement their own order normalization so that their feature vector in the extensions would not degenerate into combinatorial noise. For the purpose of fingerprinting the Chrome fingerprints would remain more or less as distinct as before.
But for the purpose of comparing fingerprints, the ways to normalize the order are not publicly specified and could well be slightly different across fingerprint databases, and then the fingerprint results cannot be used across databases.
Here's an issue in JA3, no responses to it yet.
https://github.com/salesforce/ja3/issues/88
https://chromestatus.com/feature/5124606246518784 discusses a new change, designed to prevent ossification, in which client hellos will be randomized, subject to the limits in the RFC.
It was originally scheduled to go into Chrome 110, but was actually merged in Chrome 109.
Other fingerprinting implementations such as https://tlsfingerprint.io/ have started to sort these headers to restore some consistency; see https://tlsfingerprint.io/norm_fp .
Does JA3 plan to address this?
My question is that what is the purpose of shuffling while the privacy that random fingerprints bring can't be satisfied?
the privacy
Might not be the only reason to shuffle the extensions. Most likely, it is an attempt to defeat middleboxes which hardcoded all well-known ~fingerprints~ ClientHello messages. As stated in Ready for Trial: TLS ClientHello extension permutation:
Randomize the order of TLS ClientHello extensions, to reduce ossification.
what is the point of this issue? I read the full issue text, but I am not understanding the concern? what is the problem you are hoping to address? to me this seems like a good change. if Chrome is sending random fingerprints, then servers can no longer whitelist Chrome only, or other dumb restrictions, at least not without some extra effort.
For utls which try to simulate chrome tls, this will increase extra work.
It means utls library need to generate a new fingerprint inside chrome possible permutations for each connection.
And Firefox will follow the way.
So a false simulated utls fingerprint (say a fix fingerprint) may cause fingerprint of the fingerprint.
not sure what that nonsense was, but if Chrome is randomizing extensions, then other clients like uTLS can do that as well. In fact they already have:
https://github.com/refraction-networking/utls/issues/132
so I will ask again, what is the issue here?
@4cq2 there is no problem to address. It's just a discussion topic. TLS fingerprinting is an important topic in censorship circumvention. This was a note to keep people updated with what's happening in that topic.
My question is that what is the purpose of shuffling while the privacy that random fingerprints bring can't be satisfied?
By preventing servers and middleboxes from relying on a static order of TLS extensions, Chrome makes its TLS implementation more adaptable to future changes. This reduces the risk of issues or restrictions arising if Chrome updates how it arranges TLS extensions internally.
While not guaranteeing privacy, it does disrupt server-side fingerprinting techniques built solely on TLS extension order. For a service solely focusing on this specific aspect, the Chrome change adds uncertainty to their data.
fingerprinting services can easily implement normalization to reduce the uniqueness of individual connections for better comparison.
Every additional fingerprinting vector that becomes dynamic (rather than static) introduces hurdles for trackers. This forces them to either gather more data points or invest in research to maintain the same level of identification.
