google-play-scraper icon indicating copy to clipboard operation
google-play-scraper copied to clipboard

[BUG] app() doesn't return any app data , only return review text

Open NupuraW opened this issue 2 years ago • 9 comments

google_play_scraper.1.1.0

Describe the bug app_meta = app( 'com.webserveis.batteryinfo', lang='en', # defaults to 'en' country='us' # defaults to 'us' ) doesn't return any data about app. all the fields are None or null

Code Copy and paste the code that have issue.

Expected behavior should return data such as title, developer ,price, description,rating ,intstalls

Additional context Add any other context about the problem here.

NupuraW avatar Jun 24 '22 02:06 NupuraW

@NupuraW, are you sure you are using the latest version of Google Play Scraper? I just tested it for 10k apps and everything works fine for me. Maybe Google is rolling out an update that is causing the problems, but then I can't fix it until I have the problem too.

kluhan avatar Jun 24 '22 17:06 kluhan

I'm using the latest version. 1.1.0.

NupuraW avatar Jun 24 '22 18:06 NupuraW

does it depend on anything else to pull data? can i use proxy for getting data for an app?

NupuraW avatar Jun 24 '22 19:06 NupuraW

no, as far as i know there are no dependencies except python >= 3.6.

no, currently there is no way to use a proxy, but you can open a feature request, maybe more people need such a feature. a workaround would be to simply redirect all tcp traffic to a proxy. you could try a tool like redsocks https://github.com/darkk/redsocks if you have a suitable proxy server. another option would be to try a public VPN provider. please let me know if you are successful with this or if you find another solution.

kluhan avatar Jun 24 '22 21:06 kluhan

I'm having simmilar issue (everything is none) even on the latest version 1.2.0 This only happens when crawling apps for australia (others like us, slovakia, brazil are okay) for example id: "com.arzfoodservice" country "au" https://play.google.com/store/apps/details?id=com.arzfoodservice&hl=en&gl=au

I am crawling from slovakia

crowley285 avatar Jul 07 '22 10:07 crowley285

@crowley285 tried right now "com.arzfoodservice" country "au" with 1.2.0 and works OK for me

rossoft avatar Jul 07 '22 11:07 rossoft

@rossoft Thanks for verifying Interesting, do you think that google doesn't send the same data when crawling from different countries? Like in my case for australia when crawling from slovakia.

crowley285 avatar Jul 07 '22 11:07 crowley285

@crowley285 no idea, do you see the page from same laptop via browser? just to check it is not a network problem

rossoft avatar Jul 07 '22 11:07 rossoft

The browser on the same computer opens the page without problems The scraper also downloads the page but is unable to parse anything from it I'm attaching what is returned by: /google_play_scraper/features/app.py:15 (dom = get(url)) dom.txt

crowley285 avatar Jul 11 '22 06:07 crowley285

I'm having similar issue (everything is none) even on the latest version 1.2.2 and I solved it by overwrite Detail in ElementSpecs change all ElementSpec(4, ...) by ElementSpec(5, ...) because for me app date in ds:5 not in ds:4

I think Google changes the content of the pages according to the IP address of the users My IP address is inside Iran

ma9h9di avatar Aug 21 '22 15:08 ma9h9di

@ma9h9di please check you are really using 1.2.2, as this change ElementSpec(4) to ElementSpec(5) is precisely what is included in 1.2.2

rossoft avatar Aug 22 '22 09:08 rossoft

I pulled the latest changes and the issue seems to be resolved. So far i did not encounter any issues.

crowley285 avatar Aug 22 '22 12:08 crowley285

@kluhan & @rossoft huge thanks to you help resolve this issue.

JoMingyu avatar Aug 30 '22 12:08 JoMingyu