backend icon indicating copy to clipboard operation
backend copied to clipboard

add topic ingestion function for PushShift verified twitter

Open rahulbot opened this issue 4 years ago • 2 comments

Same as the reddit one (https://github.com/berkmancenter/mediacloud/issues/598). I think this has already happened, but we aren't tracking it on an issue. We want to be able to search PushShift.io verified twitter archive to discover tweets with links to add to a topic.

rahulbot avatar Jan 08 '20 16:01 rahulbot

On the front end we have a small wrapper around this API, which could be built on or repurposed to support this plugin. Relevant code is in pushshift/reddit.py in the front end code. We use this to show a preview of content while adding the "Verified Twitter" topic to your topic.

rahulbot avatar Mar 04 '20 19:03 rahulbot

re-assigning this to eric. eric and I had a good first talk about getting him started on this work

first step for eric will be to pull out the topics_mine/posts/init.py, topics_mine/posts/csv_generic.py, topics_base/posts.py, and tests/test_csv_generic.py files into a separate repo and get them running independently from media cloud.

once that is working, he will work on implementing a pushshift_twitter.py package that will implement pushshift verified twitter in the separate repo. when that is working, I will integrated back into the main codebase.

hroberts avatar Aug 21 '20 20:08 hroberts