twitter_scraping icon indicating copy to clipboard operation
twitter_scraping copied to clipboard

Scraping "restricted" accounts

Open DonaldTsang opened this issue 6 years ago • 0 comments

Accounts like https://twitter.com/jackiec57 are unscrapible in search,
unless the user has logged into the browser, and turned off sensitivity.
Scrapy can't scrape it as it does not allow cookies, can Selenium?

Some ideas to reduce the repetitive need to click "show" on restricted accounts:

  1. Login https://gist.github.com/momota10/969d904b4cad239da2a5c00df1ad87e7
  2. Go to https://twitter.com/settings/safety
  3. Check "show sensitive content" in <input name="search-settings-nsfw" checked="" class="SearchSettings-toggleInput SearchSettings-toggleInput--sensitive" type="checkbox">
  4. Click "submit" <button id="settings_save" class="EdgeButton EdgeButton--primary EdgeButton--medium" type="submit" disabled="">Save changes</button>

DonaldTsang avatar Mar 08 '18 06:03 DonaldTsang