anyproxy icon indicating copy to clipboard operation
anyproxy copied to clipboard

抓取一个APP失败案例

Open fxp opened this issue 6 years ago • 10 comments

希望了解具体怎么调试这个问题。

Which platform are you running AnyProxy

Mac 10.14

The version of the AnyProxy

4.0.12

Your expected behavior of AnyProxy

运行 anyproxy -i --ws-intercept 启动的服务 EWA - English speaking course by LEARNING ENGLISH LAB PTE LTD 可以抓取到应用内的网络访问

The actual behavior of AnyProxy

应用停止在开机页面,抓取到的请求都是CONNECT的类型。 证书安装到了login和system里,也选了always trust 虽然每次启动都还是会报ROOT CA NOT INSTALLED YET。但其他应用,如Twitter,Google这样HTTPS的网页都可以打开抓取到body,应该证明证书安装成功了吧?

The log of the error

后台看到的是这样的 后台截图 会有一些类似api.appewa.com,graph.facebook.com的部分是灰色的。

$ anyproxy -i --ws-intercept [AnyProxy WARN][2018-12-25 16:05:00]: ROOT CA NOT INSTALLED YET ? The rootCA is not trusted yet, install it to the trust store now? Yes [AnyProxy Log][2018-12-25 16:05:06]: About to trust the root CA, this may requires your password Password: [AnyProxy Log][2018-12-25 16:05:09]: Root CA install, you are ready to intercept the https now [AnyProxy Log][2018-12-25 16:05:09]: The root CA file path is: /Users/xxx/.anyproxy/certificates/rootCA.crt [AnyProxy Log][2018-12-25 16:05:09]: Http proxy started on port 8001 [AnyProxy Log][2018-12-25 16:05:09]: web interface started on port 8002 [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request graph.facebook.com [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request graph.facebook.com [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for graph.facebook.com established [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for graph.facebook.com established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request gate.hockeyapp.net [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for gate.hockeyapp.net established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request sentry.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for sentry.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request sentry.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for sentry.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request sentry.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for sentry.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request sentry.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for sentry.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request sentry.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for sentry.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request sentry.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for sentry.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request sentry.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for sentry.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request sentry.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for sentry.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request sentry.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for sentry.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request sentry.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for sentry.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request graph.facebook.com [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request www.gstatic.com [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for graph.facebook.com established [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for www.gstatic.com established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request play.googleapis.com [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for play.googleapis.com established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request p52-buy.itunes.apple.com [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for p52-buy.itunes.apple.com established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request api2.branch.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for api2.branch.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request api.appewa.com [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for api.appewa.com established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request api.appewa.com [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for api.appewa.com established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request api2.branch.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request device-provisioning.googleapis.com [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for api2.branch.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request api.appewa.com [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for device-provisioning.googleapis.com established [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request graph.facebook.com [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for api.appewa.com established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request graph.facebook.com [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request api2.branch.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for graph.facebook.com established [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for graph.facebook.com established [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for api2.branch.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request device-provisioning.googleapis.com [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request api.appewa.com [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for device-provisioning.googleapis.com established [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for api.appewa.com established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request api2.branch.io [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for api2.branch.io established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request events.appsflyer.com [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for events.appsflyer.com established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request events.appsflyer.com [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for events.appsflyer.com established [AnyProxy Log][2018-12-25 16:05:14]: received https CONNECT request itunes.apple.com [AnyProxy Log][2018-12-25 16:05:14]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:14]: [internal https]proxy server for itunes.apple.com established [AnyProxy Log][2018-12-25 16:05:15]: received https CONNECT request itunes.apple.com [AnyProxy Log][2018-12-25 16:05:15]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:15]: [internal https]proxy server for itunes.apple.com established [AnyProxy Log][2018-12-25 16:05:15]: received https CONNECT request api2.branch.io [AnyProxy Log][2018-12-25 16:05:15]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:15]: [internal https]proxy server for api2.branch.io established [AnyProxy Log][2018-12-25 16:05:15]: received https CONNECT request t.appsflyer.com [AnyProxy Log][2018-12-25 16:05:15]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:15]: [internal https]proxy server for t.appsflyer.com established [AnyProxy Log][2018-12-25 16:05:15]: received https CONNECT request api2.branch.io [AnyProxy Log][2018-12-25 16:05:15]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:15]: [internal https]proxy server for api2.branch.io established [AnyProxy Log][2018-12-25 16:05:15]: received https CONNECT request events.appsflyer.com [AnyProxy Log][2018-12-25 16:05:15]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:15]: [internal https]proxy server for events.appsflyer.com established [AnyProxy Log][2018-12-25 16:05:15]: received https CONNECT request api2.branch.io [AnyProxy Log][2018-12-25 16:05:15]: received https CONNECT request events.appsflyer.com [AnyProxy Log][2018-12-25 16:05:15]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:15]: [internal https]proxy server for api2.branch.io established [AnyProxy Log][2018-12-25 16:05:15]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:15]: [internal https]proxy server for events.appsflyer.com established [AnyProxy Log][2018-12-25 16:05:15]: received https CONNECT request api2.branch.io [AnyProxy Log][2018-12-25 16:05:15]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:15]: received https CONNECT request events.appsflyer.com [AnyProxy Log][2018-12-25 16:05:15]: [internal https]proxy server for api2.branch.io established [AnyProxy Log][2018-12-25 16:05:15]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:15]: [internal https]proxy server for events.appsflyer.com established [AnyProxy Log][2018-12-25 16:05:15]: received https CONNECT request events.appsflyer.com [AnyProxy Log][2018-12-25 16:05:15]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:15]: [internal https]proxy server for events.appsflyer.com established [AnyProxy Log][2018-12-25 16:05:15]: received https CONNECT request cloudconfig.googleapis.com [AnyProxy Log][2018-12-25 16:05:15]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:15]: [internal https]proxy server for cloudconfig.googleapis.com established [AnyProxy Log][2018-12-25 16:05:15]: received https CONNECT request api.appewa.com [AnyProxy Log][2018-12-25 16:05:15]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:15]: [internal https]proxy server for api.appewa.com established [AnyProxy Log][2018-12-25 16:05:15]: received https CONNECT request ssl.google-analytics.com [AnyProxy Log][2018-12-25 16:05:15]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:15]: [internal https]proxy server for ssl.google-analytics.com established [AnyProxy Log][2018-12-25 16:05:29]: received https CONNECT request graph.facebook.com [AnyProxy Log][2018-12-25 16:05:29]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:29]: [internal https]proxy server for graph.facebook.com established [AnyProxy Log][2018-12-25 16:05:30]: received https CONNECT request app-measurement.com [AnyProxy Log][2018-12-25 16:05:30]: received https CONNECT request gate.hockeyapp.net [AnyProxy Log][2018-12-25 16:05:30]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:30]: [internal https]proxy server for app-measurement.com established [AnyProxy Log][2018-12-25 16:05:30]: will forward to local https server [AnyProxy Log][2018-12-25 16:05:30]: [internal https]proxy server for gate.hockeyapp.net established

fxp avatar Dec 25 '18 08:12 fxp

@fxp 部分应用可能自己会创建独立的https链接,不走系统的请求,此时无法抓到请求包,如果已有其他app能抓到包,说明证书是安装成功了

codingfishman avatar Jan 01 '19 15:01 codingfishman

@codingfishman 但是用Charles是可以抓到的,所以可能是自签名CA的问题?

fxp avatar Jan 02 '19 05:01 fxp

同样的问题怎么处理啊 1、部分https的请求可以抓到,部分不行 received https CONNECT request api.****.com,应用报了出现SSL错误,无法与服务器建立连接 2、fiddler抓出问题的域名请求是ok的

有没有老铁知道怎么处理呢?在线等

zrzs avatar May 17 '19 07:05 zrzs

会不会是应用自己做了证书识别呢?

Best Regards 冯小平 - Chopin Feng 软件开发工程师

email: [email protected] tel/wechat: +86 134-8889-2615 twitter: @fengxiaoping facebook: feng.xiaoping homepage: www.fengxiaoping.com

On Fri, May 17, 2019 at 3:01 PM Most_Wanted [email protected] wrote:

同样的问题怎么处理啊 1、部分https的请求可以抓到,部分不行 received https CONNECT request api.****.com,应用报了出现SSL错误,无法与服务器建立连接 2、fiddler抓出问题的域名请求是ok的

有没有老铁知道怎么处理呢?在线等

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/alibaba/anyproxy/issues/453?email_source=notifications&email_token=AAAN5RIAWNZBHHYDWYVUPELPVZJ3PA5CNFSM4GMDWJ4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVT5QFQ#issuecomment-493344790, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAN5RO2LOZW6IL2GQWN77DPVZJ3PANCNFSM4GMDWJ4A .

fxp avatar May 17 '19 07:05 fxp

如何判断呢 @fxp

zrzs avatar May 17 '19 07:05 zrzs

我的经验是,如果这个设备其他APP中确定是HTTPS的请求被完整的截取下来了,那八成就是这个问题。@Nick说的那种APP自己创建HTTPS链接,不走系统请求的,我也没想好怎么测试。

Best Regards 冯小平 - Chopin Feng 软件开发工程师

email: [email protected] tel/wechat: +86 134-8889-2615 twitter: @fengxiaoping facebook: feng.xiaoping homepage: www.fengxiaoping.com

On Fri, May 17, 2019 at 3:11 PM Most_Wanted [email protected] wrote:

如何判断呢 @fxp https://github.com/fxp

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/alibaba/anyproxy/issues/453?email_source=notifications&email_token=AAAN5RM5MMSUOX6IUU4CWDDPVZLAVA5CNFSM4GMDWJ4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVT6D3I#issuecomment-493347309, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAN5ROSU3RSIENUHHL7J7DPVZLAVANCNFSM4GMDWJ4A .

fxp avatar May 17 '19 07:05 fxp

@fxp 什么叫不走系统的请求呢? 为什么fiddler都可以正确抓取到,只是fiddler不灵活支持二次开发

zrzs avatar May 17 '19 08:05 zrzs

@fxp 什么叫不走系统的请求呢? 为什么fiddler都可以正确抓取到,只是fiddler不灵活支持二次开发

fiddler能抓到内容?

fxp avatar May 17 '19 14:05 fxp

遇到了同样的问题,请问您这边的问题有解决方案了吗?

wangyuqi7 avatar Jun 01 '21 05:06 wangyuqi7

同问,也是无法抓取应用数据

artibix avatar Feb 13 '24 15:02 artibix