twitter-scraper icon indicating copy to clipboard operation
twitter-scraper copied to clipboard

Scrape restricted twitter accounts

Open DonaldTsang opened this issue 6 years ago • 4 comments

Can this scraper scrape tweets like https://twitter.com/jackiec57 ?

To remove the need to click "show sensitive content anyways", it would need to...

  1. Login just like https://gist.github.com/momota10/969d904b4cad239da2a5c00df1ad87e7
  2. Go to https://twitter.com/settings/safety
  3. Check the "show sensitive" box in <input name="search-settings-nsfw" checked="" class="SearchSettings-toggleInput SearchSettings-toggleInput--sensitive" type="checkbox">
  4. click "submit" in <button id="settings_save" class="EdgeButton EdgeButton--primary EdgeButton--medium" type="submit" disabled="">Save changes</button>

DonaldTsang avatar Mar 08 '18 06:03 DonaldTsang

It isn't designed to scrape accounts with sensitive contents.

justinlittman avatar Mar 08 '18 16:03 justinlittman

@justinlittman but it can be modified to do so, as it uses Selenium. All it has to do is login, no?

DonaldTsang avatar May 03 '18 07:05 DonaldTsang

Yes, I believe so. However, I don't believe that you can hydrate protected tweets from Twitter's API (though I could be mistaken).

justinlittman avatar May 03 '18 13:05 justinlittman

I think you can hydrate protected tweets from API if your account follows the protected account (but i'm also not sure about this and need to double check in case this changed).

Protected tweets don't appear at all in searches though, so you can't display from:protected_account.

It is possible to "login" and get tweets that may be hidden as "sensitive" - I have that half working in my fork.

igorbrigadir avatar Feb 07 '19 20:02 igorbrigadir