proshunsuke
日向坂46メッセージ取得用の refresh_token が取得できない
概要
refresh_token の取得手順 の mitmproxy で解析中に https://api.kh.glastonr.net/v2/signin へのリクエストが見つからないため refresh_token が取得できない
情報
関係ないものもあるかもしれませんが、調査に関する情報を記載します。
- 実行環境
- PC の OS: macOS Sequoia (intel)
- スマートフォンの OS: iOS 18.0
- 日向坂46メッセージのバージョン: 3.1.6
- 以前取得した refresh_token を用いて colmsg を実行すると問題なくデータ取得ができる
- 2024年8月に日向坂46メッセージアプリが大型アップデートしており、その影響を受けている可能性がある
- mitmproxy のログに TLS のバージョンによるエラーが発生している
Client TLS handshake failed. Client and mitmproxy cannot agree on a TLS version to use. You may need to adjust mitmproxy's tls_version_client_min option.
I'm facing the same issue.
I think it only captures the /update_token only, not /signin.
The app looks like enabled SSL pinning. The update should be a month before with UI amendment.
This feature would only trust the application stated SSL Certification rather than the MITM certification.
What if you are using iOS, it's very difficult to capture. But, Android is still possible.
Also, another stuff is the refresh_token would be updated when you switch login between device. It means when you get the refresh_token from an Android device, but you often use your iPhone for reading messages as usual, the refresh_token would be change at the moment of iPhone login.
This is my observation.
Thank you for issue and investigating.
It seems that getting the refresh_token has become difficult on iOS.
Even on Android, it is difficult to use mitmproxy on devices running version 7 or later, making it hard to retrieve the refresh_token. Moreover, as mentioned in https://github.com/proshunsuke/colmsg/issues/108#issuecomment-2381116518 , even if you set up an environment with Android 6 or earlier and use mitmproxy to get the refresh_token, it would be inconvenient since the refresh_token would be used in a different environment from your usual device.
Currently, it is difficult for new users to use colmsg on both iOS and Android (though existing colmsg users who have already got the refresh_token will not be affected).
Unfortunately, I don't have a complete solution for this issue at the moment.
For now, I will add a notice about this issue at the top of the repository.
I think, for now, the best approach is to avoid updating or downgrading the app if you're using Android.
I've tested on both my iOS and Android phones and successfully retrieved the refresh_token via MITM on older app versions (below version 3.0.0).
iOS 櫻坂46メッセージ & 乃木坂46メッセージ: Version 3.1.6.33 日向坂46メッセージ: Version 2.16.01.224 (still works)
Android All versions are still working 櫻坂46メッセージ: Version 1.13.00.180 乃木坂46メッセージ: Version 1.11.01.224 日向坂46メッセージ: Version 2.14.00.180
So, yeah, they enabled SSL pinning after the major UI update in August 2024. Luckily, we can still use the older app versions.
I have confirmed that the refresh_token can be obtained by using an emulator running in Genymotion on Windows and Intel-based Macs.
In Genymotion, start a device with Android 6.0 or below, and download the "日向坂46 メッセージ" app version below 3 from APKPure.
After that, you can obtain the refresh_token using mitmproxy as usual.
I haven't figured out a way to run an Android 6.0 or earlier emulator on an Apple Silicon Mac, so the same method doesn't work. If you have any ideas, I'd appreciate your help.
Can we use access_token to download as many as possible as a temporary solution? access_token expires after an hour iirc which is enough to download quite a lot of messages
Force updated has been released. Since 2400-2800, Dec.18, 2024, both iOS and Android devices would be requested to use latest version.
So, from Dec.19, 2024, it is highly impossible to retrieve refresh_token by using mitm.
Can we use access_token to download as many as possible as a temporary solution? access_token expires after an hour iirc which is enough to download quite a lot of messages
Yes, but if you have already retrieved the access_token.
According the flow of application,
refresh_token is using for updating the latest access_token in for get data. (Typical OAuth2.0 application flow)
So, when you have owned refresh_token and well protected this refresh_token, it would let you get access_token.
But, the things we discussed above are related to SSL pinning. In other words, the case all communications are protected with the SSL certification. It's very difficult to capture the communication between the application and server.
Yup, I got the notification too. I guess this is the end for us?
Maybe. It's becoming those good old days
Cheers mates! 🍻
Thanks @proshunsuke to contribute this project. 本当にありがとうございます！
Can we use access_token to download as many as possible as a temporary solution? access_token expires after an hour iirc which is enough to download quite a lot of messages
Yes, but if you have already retrieved the
access_token.According the flow of application,
refresh_tokenis using for updating the latestaccess_tokenin for get data. (Typical OAuth2.0 application flow) So, when you have ownedrefresh_tokenand well protected thisrefresh_token, it would let you getaccess_token.But, the things we discussed above are related to SSL pinning. In other words, the case all communications are protected with the SSL certification. It's very difficult to capture the communication between the application and server.
So, it means that despite having an access token, we still can't do anything because of SSL pinning?
Yup, I got the notification too. I guess this is the end for us?
Maybe. It's becoming those
good old daysCheers mates! 🍻
Thanks @proshunsuke to contribute this project. 本当にありがとうございます！
It's kinda sad, though. I really need this for when a member graduates or for monthly backups. I guess this might be the end for us—or maybe not. I hope the Chinese Weibo fans can do something about it.
So, it means that despite having an access token, we still can't do anything because of SSL pinning?
Because access_token would be expired, but refresh_token seems only updated when application execute transfer or login flow.
So, once you have refresh_token, it just use is to get access_token from /login endpoint.
Without SSL pinning, we could easily capture either refresh_token or access_token. Under SSL pinning, we can not see this communication from mitm
So, it means that despite having an access token, we still can't do anything because of SSL pinning?
Because
access_tokenwould be expired, butrefresh_tokenseems only updated when application execute transfer or login flow.So, once you have
refresh_token, it just use is to getaccess_tokenfrom/loginendpoint. Without SSL pinning, we could easily capture eitherrefresh_tokenoraccess_token. Under SSL pinning, we can not see this communication frommitm
Yeah, I mean, as long as we have the refresh_token and don't do anything like transfer or login flow, we can still do it, right?
Yeah, I mean, as long as we have the
refresh_tokenand don't do anything like transfer or login flow, we can still do it, right?
Seems like that.
Thank you for notifying about the forced update.
As explained, once a valid refresh_token is retrieved, it can be used to receive messages. However, the token will be updated through the login flow.
Due to the forced update, it will become difficult to retrieve a new refresh_token after Dec. 19. If a valid refresh_token remains after Dec. 19, it will likely still allow messages to be received. However, I’m not sure how future updates will affect this.
I have confirmed that the refresh_token can be obtained by using an emulator running in Genymotion on Windows and Intel-based Macs.
In Genymotion, start a device with Android 6.0 or below, and download the "日向坂46 メッセージ" app version below 3 from APKPure.
After that, you can obtain the refresh_token using mitmproxy as usual.
I haven't figured out a way to run an Android 6.0 or earlier emulator on an Apple Silicon Mac, so the same method doesn't work. If you have any ideas, I'd appreciate your help.
I'm trying to use Genymotion on my Linux and I'm encountering an error like this. Do I need to use Android 7.0 or higher?
Wait... maybe it's just a technical problem on Flutter...
Flutter applications still don't automatically use the system's proxy, unless the developer adds this functionality by creating custom Android & iOS plugins that provide this information.
https://blog.nviso.eu/2022/08/18/intercept-flutter-traffic-on-ios-and-android-http-https-dio-pinning/
Wait... maybe it's just a technical problem on Flutter...
Flutter applications still don't automatically use the system's proxy, unless the developer adds this functionality by creating custom Android & iOS plugins that provide this information.
https://blog.nviso.eu/2022/08/18/intercept-flutter-traffic-on-ios-and-android-http-https-dio-pinning/
Oh? So this Mobame app uses Flutter? I just found out...
I tried using Reqable with the 'Bypass All SSL' feature yesterday, yet it still did not capture the API hits.
I am able to get the token by running the latest APK on an Android Emulator. Do a scan using Cheat Engine on the emulator process. It takes some time to try an error to inspect the memory.
I am able to get the token by running the latest APK on an Android Emulator. Do a scan using Cheat Engine on the emulator process. It takes some time to try an error to inspect the memory.
can you provide which emulator you are using and maybe the process? I tried with nox and seems not working
I am able to get the token by running the latest APK on an Android Emulator. Do a scan using Cheat Engine on the emulator process. It takes some time to try an error to inspect the memory.
can you provide which emulator you are using and maybe the process? I tried with nox and seems not working
I am using LDPlayer 9. The process name is Ld9BoxHeadless. At the log in confirmation screen, search for the refresh token from the memory.
@ktsh2024
Do a scan using Cheat Engine on the emulator process
I'm trying to do this. Would you mind sharing screenshots of Cheat Engine and LDPlayer 9 when you identified the refresh token using Cheat Engine?
@ktsh2024
Do a scan using Cheat Engine on the emulator process
I'm trying to do this. Would you mind sharing screenshots of Cheat Engine and LDPlayer 9 when you identified the refresh token using Cheat Engine?
On the login confirmation screen, I searched for "refresh_token" (the double quotation marks are necessary), then right-clicked on the result and selected "Browse this memory region," where I found my refresh_token.
@ktsh2024
Do a scan using Cheat Engine on the emulator process
I'm trying to do this. Would you mind sharing screenshots of Cheat Engine and LDPlayer 9 when you identified the refresh token using Cheat Engine?
On the login confirmation screen, I searched for "refresh_token" (the double quotation marks are necessary), then right-clicked on the result and selected "Browse this memory region," where I found my refresh_token.
Hello, I have followed your instructions to locate the refresh_token. After selecting Browse this memory region, what steps should I take next to obtain the refresh_token?
@ktsh2024
Do a scan using Cheat Engine on the emulator process
I'm trying to do this. Would you mind sharing screenshots of Cheat Engine and LDPlayer 9 when you identified the refresh token using Cheat Engine?
On the login confirmation screen, I searched for "refresh_token" (the double quotation marks are necessary), then right-clicked on the result and selected "Browse this memory region," where I found my refresh_token.
Hello, I have followed your instructions to locate the refresh_token. After selecting Browse this memory region, what steps should I take next to obtain the refresh_token?
After selecting 'Browse this memory region,' the Memory Viewer pops up, and you need to locate your refresh_token in the ascii pane, which is written in the format: "refresh_token":"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx".
@ktsh2024
Do a scan using Cheat Engine on the emulator process
I'm trying to do this. Would you mind sharing screenshots of Cheat Engine and LDPlayer 9 when you identified the refresh token using Cheat Engine?
On the login confirmation screen, I searched for "refresh_token" (the double quotation marks are necessary), then right-clicked on the result and selected "Browse this memory region," where I found my refresh_token.
Hello, I have followed your instructions to locate the refresh_token. After selecting Browse this memory region, what steps should I take next to obtain the refresh_token?
After selecting 'Browse this memory region,' the Memory Viewer pops up, and you need to locate your refresh_token in the ascii pane, which is written in the format: "refresh_token":"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx".
Thank you, I get refresh token now!
After selecting 'Browse this memory region,' the Memory Viewer pops up, and you need to locate your refresh_token in the ascii pane, which is written in the format: "refresh_token":"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx".
When I download Nogizaka msg, it shows:[31m[colmsg error][0m: HTTP status client error (400 Bad Request) for url (https://api.n46.glastonr.net/v2/update_token). Have you encountered this issue? (colmsg-v3.2.1) (Window 10)
After selecting 'Browse this memory region,' the Memory Viewer pops up, and you need to locate your refresh_token in the ascii pane, which is written in the format: "refresh_token":"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx".
When I download, it shows:[31m[colmsg error][0m: HTTP status client error (400 Bad Request) for url (https://api.n46.glastonr.net/v2/update_token). Have you encountered this issue?
Yes, I did encounter this issue, but updating colmsg to version 3.2.1 resolved it. The version of your colmsg seems to be the latest, but please double-check to ensure it is truly up-to-date.