Yotter icon indicating copy to clipboard operation
Yotter copied to clipboard

Use QUIC when connecting to YouTube

Open FireMasterK opened this issue 3 years ago • 14 comments

Today, I discovered something interesting about bypassing Google's Recaptcha. https://github.com/iv-org/invidious/issues/957#issuecomment-576424042

According to Omar's comment, the type of captcha /das_captcha can be bypassed completely when using QUIC. This would be huge and would fix #103 if true.

FireMasterK avatar Oct 17 '20 18:10 FireMasterK

Awesome! This is really the answer to what we were doing!

pluja avatar Oct 18 '20 07:10 pluja

It won't be easy to implement because the most famous Python library for using QUIC, aioquic, doesn't support the Google implementation version: Q046 and Q050. Instead, it only supports draft-28. Why is this important? Well you won't look like a real Google Chrome browser if you aren't using Google implementation and thus the /das_captcha won't probably be bypassed.

You can find a detailed explanation from omarroth himself here: https://github.com/searx/searx/issues/729#issuecomment-576431634

That's why omarroth made Crystal bindings to lsquic library: https://github.com/iv-org/lsquic.cr

One possible way to introduce a support for QUIC Q046 and Q050 version in this project would be to use a proxy in Golang with a Golang library that support the versions. Like http3-proxy but with another library because the current one only support draft-29 and draft-32.

unixfox avatar Dec 04 '20 08:12 unixfox

Thank you for this information! I will be investigating when I have time. If anyone want to take this (or any) issue don't doubt about it.

pluja avatar Dec 04 '20 10:12 pluja

It won't be easy to implement because the most famous Python library for using QUIC, aioquic, doesn't support the Google implementation version: Q046 and Q050. Instead, it only supports draft-28.

Q0xx versions are the application protocol of QUIC, the HTTP/3 versions are different. Take a look at this, even lsquic doesn't support these protocols for HTTP/3.

Why is this important? Well you won't look like a real Google Chrome browser if you aren't using Google implementation and thus the /das_captcha won't probably be bypassed.

We currently look like a Firefox browser with privacy.resistFingerprinting enabled, I'm not sure version of QUIC Firefox uses.

One possible way to introduce a support for QUIC Q046 and Q050 version in this project would be to use a proxy in Golang with a Golang library that support the versions.

The Golang proxy uses http/2 due the quic-go library being extremely unstable when used with Google's servers. Here's why: https://github.com/lucas-clemente/quic-go/issues/2836

Like http3-proxy but with another library because the current one only support draft-29 and draft-32.

Like I said above, this would need some research as we use Firefox's UA and Firefox does not report the QUIC version in its devtools.

FireMasterK avatar Dec 04 '20 10:12 FireMasterK

It won't be easy to implement because the most famous Python library for using QUIC, aioquic, doesn't support the Google implementation version: Q046 and Q050. Instead, it only supports draft-28.

Q0xx versions are the application protocol of QUIC, the HTTP/3 versions are different. Take a look at this, even lsquic doesn't support these protocols for HTTP/3.

You are incorrect, lsquic does support, take a look at the description of its GitHub repository: https://github.com/litespeedtech/lsquic#description Q0xx are internal versions developed at Google and only for Google services. Whereas draft-xx is for the official implementation of HTTP3 by the IETF.

Like I included in my previous comment, omarroth explained that here: https://github.com/searx/searx/issues/729#issuecomment-576431634

Why is this important? Well you won't look like a real Google Chrome browser if you aren't using Google implementation and thus the /das_captcha won't probably be bypassed.

We currently look like a Firefox browser with privacy.resistFingerprinting enabled, I'm not sure version of QUIC Firefox uses.

Firefox doesn't support the QUIC implementation of Google. It does matter to send HTTP requests using the QUIC implementation of Google because you make it even better to look like a Google Chrome browser, that's why Invidious even use a Chromium user agent: https://github.com/iv-org/invidious/blob/master/src/invidious/helpers/utils.cr#L5

unixfox avatar Dec 04 '20 15:12 unixfox

I can't be find the supported version of https://github.com/cloudflare/quiche pyo3 or wasmer-python can make the binding between Rust and Python.

dalf avatar Dec 04 '20 15:12 dalf

You are incorrect, lsquic does support, take a look at the description of its GitHub repository: https://github.com/litespeedtech/lsquic#description

Turns out you are right indeed.

Like I included in my previous comment, omarroth explained that here: searx/searx#729 (comment)

This is no longer true, they have support for draft-29 now.

alt-svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"

Firefox doesn't support the QUIC implementation of Google. It does matter to send HTTP requests using the QUIC implementation of Google because you make it even better to look like a Google Chrome browser, that's why Invidious even use a Chromium user agent: https://github.com/iv-org/invidious/blob/master/src/invidious/helpers/utils.cr#L5

I don't see why this matters, is there any reason to look like chrome? Has anyone tested/proved that the IETF draft doesn't bypass the captcha?

FireMasterK avatar Dec 04 '20 16:12 FireMasterK

I can't be find the supported version of https://github.com/cloudflare/quiche

This supports draft-27, draft-28 and draft-29.

FireMasterK avatar Dec 04 '20 16:12 FireMasterK

Firefox doesn't support the QUIC implementation of Google. It does matter to send HTTP requests using the QUIC implementation of Google because you make it even better to look like a Google Chrome browser, that's why Invidious even use a Chromium user agent: https://github.com/iv-org/invidious/blob/master/src/invidious/helpers/utils.cr#L5

I don't see why this matters, is there any reason to look like chrome? Has anyone tested/proved that the IETF draft doesn't bypass the captcha?

Let's imagine it does in fact bypass it, if they found out about it then they could start giving captcha to clients that support only draft-29 but not for Q050 because up to this day only browsers based on Chromium like Google Chrome support it. They don't want to give a bad experience to Google Chrome users but for Firefox users, that only support draft-29, they don't care because they already do it by making YouTube slow on Firefox: https://fortune.com/2018/07/25/youtube-slow-mozilla-firefox-chrome/

unixfox avatar Dec 04 '20 16:12 unixfox

In that case, we should use pre-existing libraries unless they are unstable or YouTube changes something. The only libraries which support h3-Q050 are chromium and lsquic (I'm still not sure why the draft doesn't list it in the http3 library list).

FireMasterK avatar Dec 04 '20 19:12 FireMasterK

image

Stable chrome seems to use draft-29 by default now.

FireMasterK avatar Dec 29 '20 10:12 FireMasterK

It depends on the server. On google.com : image

dalf avatar Dec 29 '20 10:12 dalf

Might be a region-specific change then, I still see it using draft-29 on google.com.

I tested on chrome on windows 10 with all flags reset.

FireMasterK avatar Dec 29 '20 10:12 FireMasterK

Sorry, the chrome://flags/#enable-quic was Enabled.

When I switch back to Default, I see h3-29 too.

dalf avatar Dec 29 '20 11:12 dalf