backend
backend copied to clipboard
add topic ingestion function for PushShift verified twitter
Same as the reddit one (https://github.com/berkmancenter/mediacloud/issues/598). I think this has already happened, but we aren't tracking it on an issue. We want to be able to search PushShift.io verified twitter archive to discover tweets with links to add to a topic.
On the front end we have a small wrapper around this API, which could be built on or repurposed to support this plugin. Relevant code is in pushshift/reddit.py
in the front end code. We use this to show a preview of content while adding the "Verified Twitter" topic to your topic.
re-assigning this to eric. eric and I had a good first talk about getting him started on this work
first step for eric will be to pull out the topics_mine/posts/init.py, topics_mine/posts/csv_generic.py, topics_base/posts.py, and tests/test_csv_generic.py files into a separate repo and get them running independently from media cloud.
once that is working, he will work on implementing a pushshift_twitter.py package that will implement pushshift verified twitter in the separate repo. when that is working, I will integrated back into the main codebase.