backend
backend copied to clipboard
mediacloud
Media Cloud is an open source, open data platform that allows researchers to answer quantitative questions about the content of online media.
One of the last big chunks of code that need porting to Python from Perl. (Split off of #679)
Aashka pointed out that in multi-platform world they will want to know how many total matching posts their were. Ie. not just how many links were in the tweets, but...
Some of our topics get too big (👀 elections). @hroberts added support for filtering for stories in a snapshot that have at least: * 1 inlink * 10 twitter shares...
We are getting lots of download errors from malformed urls. Some are fixable with more aggressive url fixing of things like '>http://foo.bar'. Mostly the problem is feeds giving us relative...
I'm trying to debug some user support requests, but right now calls to story list are going super slow. The below log shows return times from 1 minute to 3...
The run_remotely() call to extract_and_vector from topics-fetch-link was mysteriously hanging occasionally, eventually causing the fetch link queue to shrink. I changed the run_remotely() call to an add_to_queue() call followed by...
> Now that we consume 3rd party APIs as part of our core pipeline, we need some system for monitoring them to find out when they are down. This should...
solr imports are very slow. here are the last ten imports and the size of the import queue: ``` mediacloud=# select * from solr_imports order by solr_imports_id desc limit 10;...
To support labelling attention charts in a topic with top stories (https://github.com/mitmedialab/MediaCloud-Web-Tools/issues/1659), we need to be able to search for the top story within an arbitrary date range. This means...
The ["Africa Newsroom (Arabic)"](https://sources.mediacloud.org/#/sources/651063) source content isn't parsing correctly. Aashka's SDG topic found this one via spidering. For example, story #1307690431 ([original URL](https://tools.mediacloud.org/api/stories/1307701607/raw.html)) should have a bunch of about SDGs...
Metadata
Owner
Metadata
Media Cloud is an open source, open data platform that allows researchers to answer quantitative questions about the content of online media.