Sandmann79
Device not registering for 4K playback
Addon used
- [X] Amazon VOD (plugin.video.amazon-test)
- [ ] Browser Launcher (plugin.program.browser.launcher)
- Addon version: 0.9.5+Matrix.1
Account type
- [X] primevideo.com
- [ ] amazon.(com/co.uk/de/jp)
System Setup (please provide the following information):
- Hardware: Nvidia Shield Pro 2019
- OS version: Android TV 11 (Nvidia Experience 9.0.2)
- Kodi version number: 19.4 and 20.0 Alpha-1 (tried both)
Upload Logs
Debug log: https://paste.kodi.tv/inehevowow.kodi avod-login.log: https://paste.kodi.tv/pikeqeciwi.kodi avod-login-mfa.log: https://paste.kodi.tv/uniqokukix.kodi
Describe the bug
After enabling the 4K/HDR10/DV options in the addon and trying to play 4K episode, a popup appears telling you that the device is not registered at Amazon for 4K playback and that the user should login again from the addon.
After doing this, 4K playback still does not work. Also the primevideo.com web site does not show any additional devices that have registered.
Note: My Prime Video account is a Dutch one and in the browser I need to logon to primevideo.com
Hmm… I might have an idea on why this is no longer working, but I have no way to test this. I believe Amazon changed the way logins work, and instead of stopping on a certain page it goes automatically through everything on its own, which for the user is faster, but in our case blazes past the part we need to login properly.
I might come up with a solution, but it'd be a blind jump without anyone testing, I'd be developing without hardware.
@Varstahl I don't mind testing stuff for you and sending log files if required.
But isn't it strange that there are people on the Kodi forum for whom 4K seems to be working? Or do you think they managed to register their device earlier?
Possibly earlier. If they have the authentication already in place then it wouldn't matter, and the update would provide them with new features.
I'll get to it, but it's going to take a while to find the time needed to work on it.
So I just tried this with a VPN. The sign in button on primevideo.com is just a redirect to the amazon.XY page of that country?
In this case the code wouldnt work, because it relies on adding params to the URL that will trigger an oauth login. That oauth login returns a token that can be used to register a device and get the real token for playback.
https://github.com/Sandmann79/xbmc/blob/24b1505aed816135c1d99d32a6fd1ad0a9e2bc37/plugin.video.amazon-test/resources/lib/network.py#L633
If you add params to the primevideo.com domain it will just ignore them and redirect you to the standard amazon login for a browser. The oauth params wont be added properly, and you wont get the token for registering a device.
I have two ideas how this could be solved: You could open the primevideo page, wait for the redirect, read the URL and add the params to that. That should work, but it might be a bit messy. You probably also need to scrape out the amazon domain, and use that for oauth and registering instead of primevideo.com.
The other idea is more speculation, but could you just add all the other amazon domains to the list? e.g. amazon.nl, amazon.it, etc. Registering a device should work on any platform, not just Android. And by registering a device you can also get cookies for browser based playback.
Sadly I dont have a primevideo.com account so all I can do is guess.
@StollD Thanks for taking the time to help investigate.
You are right, when I click the signin button on primevideo.com I am redirected to this URL:
https://www.amazon.nl/ap/signin?openid.pape.max_auth_age=0&openid.return_to=https%3A%2F%2Feu.primevideo.com%2Fauth%2Freturn%2Fref%3Dav_auth_ap%3F_t%3Dsgw_GZJDh4uTqF0yXlObj9eb7soia51mRCtoM_PEhqX0iAAAAAQAAAABivcBwcmF3AAAAAPgWC9WfHH8iB-olH_E9xQ%26location%3D%2F%3Fref_%253Ddv_auth_ret&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&accountStatusPolicy=P1&openid.assoc_handle=amzn_prime_video_sso_nl&openid.mode=checkid_setup&siteState=260-8255159-8463919&language=nl_NL&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0
I wouldn't mind adding params to this URL in order to get my device registered, since it's probably a one-time action anyway. But I have no clue what changes I need to make to the URL to get this to work :)
Regarding your second possible solution, maybe @Varstahl or @Sandmann79 can comment on this?
So I just tried this with a VPN. The sign in button on primevideo.com is just a redirect to the amazon.XY page of that country?
Not necessarily, depends on the country. You are usually redirected to a domain. The "problem" is that one of the URLs has the openid.oa2 parameters:
https://github.com/Sandmann79/xbmc/blob/24b1505aed816135c1d99d32a6fd1ad0a9e2bc37/plugin.video.amazon-test/resources/lib/network.py#L675-L678
Which at the moment skips through. So my plan is to alter the getURL function to provide a redirection callback, so that we can proceed or interrupt the flow as needed, from here:
https://github.com/Sandmann79/xbmc/blob/24b1505aed816135c1d99d32a6fd1ad0a9e2bc37/plugin.video.amazon-test/resources/lib/network.py#L185-L187
Instead of forwarding the allowRedirects, on a callback present we manually call back the given closure for every redirect, and continue/abort looping through as needed. This offers the best compromise because it allows us to operate without knowing the details of every country served by PrimeVideo (there's a ton), and could work in our favour in the future, if something like this appears again.
With that done, when we have the correct match, endpoint and URL, we can just inject what we need and abort the redirection loop early, which should make logins faster and save bandwidth on top of the device registration.
That's my plan, didn't exactly have the time to implement it yet, though.
Edit: sorry, forgot a response:
I wouldn't mind adding params to this URL in order to get my device registered, since it's probably a one-time action anyway. But I have no clue what changes I need to make to the URL to get this to work :)
If you can run a browser and register your domain that way, you can extract the cookies and inject them in a file in user data. No need to change the code for that. Storage location details in the wiki.
The params skip through because its not doing an oauth login, its just doing a regular login.
For UHD playback to work, you need to register a device with Amazon. Doing that allows you to generate a token that you use instead of the cookies. You can still use the cookies, but they won't give you UHD manifests, Amazon will treat you like you are a browser.
To register a device you need the code from openid.oa2.authorization_code. But that value will not be returned if you are doing a regular login. You need to specifically request it, by adding theese params to the login request.
https://github.com/Sandmann79/xbmc/blob/24b1505aed816135c1d99d32a6fd1ad0a9e2bc37/plugin.video.amazon-test/resources/lib/network.py#L610-L628
Some of them are already in the URL that primevideo.com will redirect you to, but not all. And since these values are appended to an URL that discards them, the login flow is the same as it was before, you just get cookie auth and no registered device, because the code to register a device was never requested.
I tried to implement my first idea (parsing the amazon domain from the primevideo redirect and using it to log in) into my minimal plugin. I have tried the requests I added in the python interpreter, but not the full plugin.
https://github.com/StollD/plugin.video.tmsp-amazon/commit/3d65d55d0554690d600fa905620d8fda6ccc6250
@MauriceW67 If you are feeling adventurous, this might work for you. Or it might not work at all.
@StollD I tried to replace the auth.py file inside the plugin with you "hack" version on my Shield and then logging on
Unfortunately it throws an error:
2022-07-01 16:39:59.758 T:14163 ERROR
Ah, I see, makes sense. I'll factor that information in.
@Varstahl Did you make any progress yet? Not trying to pressure you, just curious :)
Nah, fell sick end of week. It's all in my head but need to actually write it down.
@StollD How would I go about testing your "hack"? You mentioned something about the Python interpreter? Is this possible on an Nvidia Shield running Kodi?
Replacing the auth.py file inside your plugin didn't work obviously :)
Replacing the file is the correct way, my idea just doesnt work. I only tested the code to determine the amazon.xy domain from the primevideo redirect, not if that domain will actually allow login the same way as amazon.com/uk/de
Replacing the file is the correct way, my idea just doesnt work. I only tested the code to determine the amazon.xy domain from the primevideo redirect, not if that domain will actually allow login the same way as amazon.com/uk/de
Ok, thanks for trying. It was worth a shot :)
I don't know if it will help, but I made a screenshot of the debug screen of the Prime Video app on my Shield:
https://i.postimg.cc/BQyF7ncR/Prime-Video-20220709-103313.jpg
Interestingly, the authEndpoint URL points to api.amazon.co.uk, but all the other endpoints point to api.amazonvideo.com.
As you can see my geoLocation is detected as NL.
@Varstahl I found sort of a semi-workaround:
After I noticed on the debug screen of the app that my authEndpoint URL pointed to amazon.co.uk, I tried to login from the addon while connected to the UK using NordVPN on my Shield. This seemed to work because after jumping through all the verification hoops (including a text message on my phone with an approval URL), a new Android device became visible in my Prime Video account on the UK version of the web page.
I was then able to play videos in 4K, but the addon seemed to use my "UK" profile, because my watchlist was empty for example. Also, the show Reacher - Season 1 shows up after a search, but does not show any episodes. However other shows such as The Boys and Wheel of Time work just fine.
I then checked my device list on the primevideo.com web site without a VPN and noticed that the newly registered device had also become visible there.
So I thought I would relog into my account in the addon without the VPN. This resulted in my watchlist becoming available again, but unfortunately the shows that worked before in 4K were now back to 1080p.
So it does work kind of, but it's not ideal :)
Maybe this info will help with your research.
@MauriceW67 After testing, I think 4K\HDR currently only works under the TLD login. I tried logging in in the UK or US (Add-On Settings - Connection - Amazon Account Region) and it doesn't seem to need to be under the corresponding region ip. For example, if you are not in the US, you can still login in to Amazon.com. Prime Video accounts can be logged in under TLD (except for Japan, which is a completely separate account system). But TLD and Prime Video have different content libraries, so we still need to wait for the developers to fix PV 4K\HDR support.
@Liqianyu You are probably right. But like I said I was able to play 4K content using my NL PV account without VPN. As long as I have first logged on with VPN (connected to UK in my case) which made the device registration work.
Unfortunately this means the addon now "sees" the video content of the UK version of PV. But for shows that are available in both UK and NL, I can still play them in 4K, including Dutch subtitles.
So, I've been working on this all day, and I've got a prototype working. The bad news is, I have no idea if it works (probably doesn't at all). As far as I could see, this whole ordeal revolves around the need to set openid.assoc_handle to amzn_piv_android_v2_[country], but the problem is that those are only available for the usual 4. The rest has amzn_prime_video_sso_[country].
If any of you wants to try, you can replace your network.py with this version. Just don't keep your hopes up.
If any of you have the hardware and the knowledge to install the PrimeVideo app on Android and give me the network trace of the login, feel free to share, because atm I don't have enough to work with.
@Varstahl I tried the new network.py, but as you already expected it doesn't work. I was able to login without errors, but 4K video was not available and no device was registered in my PV account.
Regarding the network trace, I was able to capture one on my Shield using the app PCAPdroid. However, all traffic is of course TLS encrypted. Is that still useful or do you need a decrypted capture?
Regarding the network trace, I was able to capture one on my Shield using the app PCAPdroid. However, all traffic is of course TLS encrypted. Is that still useful or do you need a decrypted capture?
Ah yeah, if we MitM it probably won't work because of CA verification 🤦♂️. Yeah, I would need it decrypted to be able to tell which variables are used and what endpoints, in sequence. Dang it.
The app that I mentioned (PCAPdroid) does come with an MITM addon, which I was able to install successfully.
However it also requires a CA certificate to be installed in the Shield's CA store and I cannot figure out how to do this.
I have found solutions such as these:
adb shell am start -a "android.intent.action.VIEW" -d "file:///sdcard/Download/PCAPdroid_CA.crt" -t "application/x-x509-ca-cert"
adb shell am start -n com.android.certinstaller/.CertInstallerMain -a android.intent.action.VIEW -t application/x-x509-ca-cert -d file:///sdcard/Download/PCAPdroid_CA.crt
But they all give me a toast message on the Shield saying "Couldn't install because the certificate file couldn't be read".
Do you have a clue on how to install a certificate on the Shield?
Took me some time, saw your message but was in the middle of a 4+ hours meeting. I'm not very experienced with Nvidia Shield, it's one of those objects that'd be cool to have but that I don't really want to buy, together with Chromecasts and Firesticks, so it was kind of a hard sell for me to actually buy one for this addon's development alone. I would think that the reason it doesn't work is explained in 3.4.1 of the PCAPdroid documentation. It's one of the things that made it hard for me to debug this stuff, on top of the fact that my main (and only?) PrimeVideo running platform is Tizen.
I would assume that those apps, by default, reject all third party certificates. Assuming one takes the time to unpack the apk, modify the XMLs to allow outern CAs and repack the apk, then it should work. I've done something similar in the past, ~I just don't know where to start with this though.~ Scratch that, after unpacking the APK with apktool, the specific file we're looking for seems to be res/xml/network_security_config.xml:
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
<base-config cleartextTrafficPermitted="false">
<trust-anchors>
<certificates src="system" />
</trust-anchors>
</base-config>
<debug-overrides>
<trust-anchors>
<certificates src="user" />
<certificates src="@raw/mockwebserver_root_ca_cert" />
</trust-anchors>
</debug-overrides>
<domain-config>
<domain includeSubdomains="true">localhost</domain>
<domain includeSubdomains="true">127.0.0.1</domain>
<trust-anchors>
<certificates src="@raw/mockwebserver_root_ca_cert" />
</trust-anchors>
</domain-config>
</network-security-config>
So we could add all the various TLDs to the include list and repack. Or we could add a brand new CA to the XML, by adding a new certificate, like @raw/totally_not_mitm_cert, place the .pem file res/raw, and add it to res/values/public.xml, original/META-INF/MANIFEST.MF and original/META-INF/TEST_SIG.SF (we'd need to re-sign the files though? not sure if necessary). Or we could overwrite the res/raw/mockwebserver_root_ca_cert.pem with the CA from PCAPdroid instead, but still would need signing I think (maybe).
Have I ever done it? No. Does it sounds fun? Possibly :D
I think that adding TLDs is possibly the safest bet, but I don't know how the SHA signatures work. Could removing it from the TEST_SIG.SF work? Does the MANIFEST.MF entry need to match the file? Probably, I'd assume, but I'm not an android expert, and it's not something I'll be able to discover at 2am on a working day, hehe.
Feel free to read more/experiment with any of these. Might get us closer, or not, what do I know, I'm a tired old man.
APK resign
Just found this, might work or not, I have no idea:
- Remove the folder META-INF
- Repack
- Use the jarsigner and zipalign with your new keystore
Basically just remove the signatures, create a new keystore to sign the apk with, and resign it. I remember doing this very thing many moons ago, but I'd be hard press to remember the specifics at this moment.
@Varstahl My test result is the same as MauriceW67. And to MITM grab the packets, I think need to modify the application or Root. I don't know if I can use other devices and methods instead.
Japan region login is a bit strange, as my European Prime subscription can be logged in the Japan region. As far as I know Amazon Japan uses a separate account system. At least amazon.co.jp other regional accounts can't log in. Does this mean that Prime Video accounts are globally interoperable even for the Japan region?
@Varstahl Thanks for looking into this further.
I did find an older guide on how to capture traffic using PCAP Remote:
https://www.exandroid.dev/2021/03/21/capture-all-android-network-traffic/
However, when I tried to make the changes to the APK and recompiling it, it would no longer install on the Shield. I used the APKEasyTool instead of apktool though, so that may have something to do with it.
The changes to the APK you are suggesting are going a bit over my head though :) Maybe if I spend more time learning/investigating I would start to understand, but for the moment it's not clear to me.
It would be easier to maybe use an Android phone instead of a Shield, since that would be easier to root and/or install certificates on. But I don't think Android phones support 4K playback?
@MauriceW67 If the phone is available, you can go directly through the Android emulator. Some emulators have Root access built in. then there are some methods and software to bypass SSL Pinning, for example https://github.com/Fuzion24/JustTrustMe Or decompile and modify the app, for example https://github.com/felHR85/Uncertify
Shield TV should be able to use SHIELD Developer OS Images to get Root. I think it's best to test it via mobile first, because odds are Android will use a very similar login.
@Varstahl The Fire TV Stick 4K is now on sale for Prime members for 27 euros. Or the 4K Max version for 37 euros if you're interested :).
I'd even be willing to donate one if you think it would help you debug the issue.
https://www.amazon.it/s?k=firestick&i=prime-day&__mk_it_IT=%C3%85M%C3%85%C5%BD%C3%95%C3%91&crid=1N4APZY6WMFN7&sprefix=firestick%2Cprime-day%2C44&ref=nb_sb_noss_2
Does this mean that Prime Video accounts are globally interoperable even for the Japan region?
Amazon's accounts are global, you can use your account on any of amazon's websites. The streaming services might be geolocked depending on where you're accessing what. PrimeVideo is global, so you can technically access it from everywhere, but the content and the languages you see depend on the georegion where you're locked into.
If the phone is available, you can go directly through the Android emulator
I was thinking that I couldn't do it before for 4k testing, but then I just realized that I don't need to, we know 4k is working o if I can just test the endpoints then that would kind of solve the problem. I have a couple emulators I can try it on, I'll see if it works (after work ofc).
@Varstahl The Fire TV Stick 4K is now on sale for Prime members for 27 euros. Or the 4K Max version for 37 euros if you're interested :)
hnng… is this the day where I give in? Honestly I don't know if it would help, at least in this specific regard. ~I'll think it over. Possibly past the Prime Day, hahaha, wouldn't be uncharacteristic of me.~ Fuck it, I gave in, I'll receive it tomorrow.
hnng… is this the day where I give in? Honestly I don't know if it would help, at least in this specific regard. ~I'll think it over. Possibly past the Prime Day, hahaha, wouldn't be uncharacteristic of me.~ Fuck it, I gave in, I'll receive it tomorrow.
LOL, sorry didn't mean to make you spend extra money... Just thought I'd mention it since the discount is pretty substantial. Just noticed that the Chromecast with Google TV is also discounted to 35 euros (at least in Italy, in NL there is no discount).