YouTube

[89z]

I cant seem to get the YouTube app to work. I tried these combinations:

• User cert
• System cert
• Frida
• Frida + user cert
• Frida + system cert

Each time I get this result:

There was a problem with the network [400]
RETRY

Note I am using Android API 24

[pimterry]

Hard to know what could cause this I'm afraid. It's not totally clear if the 400 here is from an HTTP response rejecting the apps request or something else. Any idea? If it's a real error response, then it's possible that something about HTTP Toolkit's proxying is being detected on the server side (e.g. TLS fingerprinting or similar) and the server is rejecting the request on that basis.

Can you see any intercepted traffic at all? If you can see a real 400 response then that would confirm this, and would give you something to test.

It's also quite possible that 400 is some other kind of error code, and this fairly is just due to certificate pinning or some other MitM detection.

I think seriously investigating that is likely to require manual reverse engineering to hunt down the specific error, and trace through exactly which code is causing it (and manually patch that, if possible). https://httptoolkit.tech/blog/android-reverse-engineering/ has a walk through.

Let me know if you find the underlying cause or any other clues, I'd be happy to help update the script or HTTP Toolkit more generally if there's something that's missing which could catch this.

[89z]

Hah, I just realized, you get that error even with no proxy. Whoops! I think I will try to update the YouTube version, or maybe use a higher Android API

Update: looks like it works as expected with API 26

[pimterry]

Update: looks like it works as expected with API 26

Ok, great! Good to know this wasn't a proxy/cert issue, thanks for the update :+1:

[89z]

Huh, something weird going on. I dont think I checked closely before. Its true, I dont get the errors with API 26, but Im not really capturing any requests either. I can capture images, but nothing else. No video, audio, JSON, anything.

[pimterry]

I've previously wondered whether Youtube and similar apps might be using QUIC & HTTP/3 experimentally, in a way that would avoid interception proxies. As far as I'm aware, there are no debugging proxies that support HTTP/3 yet (many don't even support HTTP/2) and because it's UDP it isn't even visible in many places (e.g. HTTP Toolkit's Android app doesn't forcibly redirect it, so if it ignore device proxy settings then it'll always be sent directly).

I think there's a chance that that's what's going on here.

The best way to work out if that's what's happening is to tether your device through your computer's network connection, and then use Wireshark or similar and inspect the raw traffic, looking for lots of UDP data on UDP port 443. If that's what's happening, it's going to be very difficult to inspect (because there's very very little HTTP/3 debugging support available today) but you can probably make this interceptable by blocking that traffic - since HTTP/3 is still experimental today I'm sure they fall back to TCP-based HTTP when it doesn't work.

Just a hypothesis, but worth looking into imo. Let me know if you find out any details either way, because this definitely affects my plans for prioritising HTTP/3 in future (blocked by lack of support elsewhere right now, but hopefully possible later this year if all goes well).

[89z]

you can probably make this interceptable by blocking that traffic

Sorry Im a dummy, how do I block UDP? That seems promising, so I would like to try that.

[pimterry]

You can probably do so with iptables, e.g. https://stackoverflow.com/a/47491895/68051. In this specific case, you're looking to drop all outgoing UDP traffic to port 443.

You'll need to make sure the mobile traffic is going through your machine first though, which probably means tethering the phone to your computer (assuming it's a physical device rather than an emulator). Once you've got that setup I'd definitely suggest taking a look with Wireshark first, since it'll definitively tell you whether or not this is the problem (lots of traffic for udp.port == 443 means yes, anything else means probably not). Without that it's hard to know whether or not any iptables rules you add are working.

[89z]

Just found this while making a YouTube API request:

{
  "QUIC": {
    "close_sessions_on_ip_change": true,
    "connection_options": "BWRS",
    "quic_version": "QUIC_VERSION_43"
  }
}

so I think that confirms it

[pimterry]

That makes sense! On Android I think this is something we could work around in theory by simply dropping outgoing UDP to port 443 in the Android app VPN. That needs testing, but I suspect that most/all QUIC implementations will fallback to HTTP/2 or 1 if QUIC starts failing like that.

Any chance you've got some time to prototype & test that out? I think in theory you just need something like if (udpheader.getDestinationPort() === 443) return; here and that would drop all that UDP immediately.

Alternatively, you might be able to avoid this issue with HTTP Toolkit rules. If Youtube is using alt-svc headers to detect & set up HTTP/3 (rather than hardcoding it as the default) then dropping those headers could disable it.

[89z]

Any chance you've got some time to prototype & test that out? I think in theory you just need something like if (udpheader.getDestinationPort() === 443) return; here and that would drop all that UDP immediately.

I don't really have a way to compile a Java program. I have java.exe on my system, but I think you need Gradle or something? I don't know. If you have a test APK I can try it out.

Alternatively, you might be able to avoid this issue with HTTP Toolkit rules. If Youtube is using alt-svc headers to detect & set up HTTP/3 (rather than hardcoding it as the default) then dropping those headers could disable it.

I tried removing the headers [1], I tried both of these:

set modify_headers /~q/Alt-Svc/
set modify_headers /~q/alt-svc/

but it didnt seem to change anything. Note that I am testing using Android API 31. I tried using API 30, but YouTube was missing, and API 29 and older seems to be broken.

1. https://docs.mitmproxy.org/stable/overview-features#modify-headers

[89z]

Just noticed this on the main YouTube site:

alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443";
ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"

the plot thickens...

[pimterry]

Yes, that's promising! That header is what I'm talking about. If the Youtube app is well behaved, it should only use HTTP/3 if it gets a response including that header. If you strip that header out everywhere, it shouldn't try to use HTTP/3.

Of course, it might remember from previous requests, or it might just ignore feature detection entirely and use HTTP/3 blindly, but it's worth a shot.

I don't know about mitmproxy, but to test that in HTTP Toolkit, you can either create a transform rule with "update response headers" put "alt-svc" there with a blank value, or you can set a response breakpoint for responses from youtube URLs, and remove the header manually.

I don't really have a way to compile a Java program. I have java.exe on my system, but I think you need Gradle or something? I don't know. If you have a test APK I can try it out.

I'm not going to have any time soon to put together a prototype for this for you in the short-term I'm afraid. If you're interested in playing around with this, you need to:

• Install the Android SDK (https://developer.android.com/about/versions/12/setup-sdk)
• Clone https://github.com/httptoolkit/httptoolkit-android
• Make the change mentioned above
• Run ./gradlew assembleRelease
• Install the APK it builds in app/build/outputs/apk/release/

Once you've got that set up, you can then modify the VPN any way you like in future, to redirect or drop arbitrary packets as you like, or log data (to adb logcat) to trace down low-level packets directly. There's some background here: https://httptoolkit.tech/blog/inspecting-android-http/

[89z]

create a transform rule

I don't know how to do that. I opened the HTTP Toolkit program, but I don't see any option for that. I checked the sidebar, and the menus as well.

[pimterry]

I don't know how to do that. I opened the HTTP Toolkit program, but I don't see any option for that. I checked the sidebar, and the menus as well.

Mock page -> Add a new rule -> On the right pick "Transform the real request or response automatically" from the dropdown.

In this case, you then want to change the "Pass through the real response headers" option to "Update the response headers", and put alt-svc in there with a blank value.

That does require Pro. If you don't have Pro then for quick testing you can just do the same with breakpoints manually.

[89z]

Mock page

Sorry to be difficult, but I don't see any "Mock page" either

[pimterry]

It should be in the sidebar:

If your HTTP Toolkit isn't showing that, then you're somehow running a very very old server version which doesn't support it. That shouldn't happen (it autoupdates in the background every time HTTP Toolkit opens) but perhaps that's not working for some reason. If so, can you hover over the 'HTTP' icon in the top left, and share the version numbers shown there here please?

[89z]

I think the "Transform the real request or response automatically" requires the Pro version

[pimterry]

Yes, as above:

That does require Pro. If you don't have Pro then for quick testing you can just do the same with breakpoints manually.

[89z]

I don't think I have the pro version. I think you gave me a month back in November, but I never used it, and I think it's expired now.

If you're essentially saying that YouTube support is a paid item, I respect that. However I don't have a job at this time, so I won't be able to pay for that. If you need to close this issue, I understand.

[pimterry]

If you're essentially saying that YouTube support is a paid item, I respect that.

It's not that YouTube support is paid. For starters, the above are just some suggestions for your issue that I haven't tested, so I have no idea if that will actually work, and also you can do everything above without Pro. You just need to use manual breakpoints instead. The only benefit of Pro in this case is that it saves you some time/inconvenience if it turns out that stripping alt-svc does indeed work and you want to do a lot of that later.

I'm happy to leave this issue open - it's a real issue, and maybe somebody else will find it and have suggestions, or maybe you'll find a workaround you can share here to help others interested in YouTube too. Personally though I'm not aware of any other promising options to investigate: I think that testing out stripping alt-svc or testing dropping UDP 443 packets in the Android app are probably your best options right now.