Scweet icon indicating copy to clipboard operation
Scweet copied to clipboard

Scrape can't get anything

Open G-walk opened this issue 2 years ago • 8 comments

I made the same modification as issue #163, now the scrape can search but can't return anything... Is it my problem or Twitter's?

ps: It worked well just 2 days ago.

image

G-walk avatar May 07 '23 09:05 G-walk

I have faced the same issue. I modified scweet.py a bit, and it works. the solution is present in #163

yisyed avatar May 07 '23 15:05 yisyed

https://github.com/Altimis/Scweet/issues/163#issuecomment-1531799351

yisyed avatar May 07 '23 15:05 yisyed

Thank you for your reply. I have actually seen your solution in #163, althougth I tried the solution by another user NicerWang instead of your's. It worked well for me two days ago, but today it just can't get anything. I think I have solved the login problem and the issue I meet now is something else.

I will try your method later, but just to make sure, your Scweet still works well now, right?

G-walk avatar May 07 '23 15:05 G-walk

My code still works. One solution to figure out what's going on is to try 'headless = False'. This will show you what your code is doing in the background. data = scrape(..., headless=False)

yisyed avatar May 07 '23 15:05 yisyed

I use your method and find what the true issue is. I used my own Twitter account to log in, and Twitter recognized my account as suspicious. After the code input my email, password and user name, Twitter asked to input a confirmation code. Scweet of course can't do that. This is the true reason why I can't get anything.

I may type in the confirmation code manually, but the log in proccess is too fast that I can't do that... Anyway, thank you very much. It's glad to know Scweet still works : )

G-walk avatar May 07 '23 16:05 G-walk

Make a burner account for such activities and remove 2FA. You cannot enter confirmation code or have any type of controll on the browser as the browser is controlled by the code (robot).

yisyed avatar May 07 '23 16:05 yisyed

I made the same modification as issue #163, now the scrape can search but can't return anything... Is it my problem or Twitter's?

ps: It worked well just 2 days ago.

image

I had the same issue but managed to find a fix and posted it over over at #167

YGH16 avatar May 30 '23 10:05 YGH16

check the solution I provided in https://github.com/Altimis/Scweet/issues/169#issuecomment-1640205875

baqachadil avatar Jul 18 '23 13:07 baqachadil