Linas Valiukas
Linas Valiukas
Given that disks on which we run our services are pre-shot or at least very old, and the disk health review + replacement is still ongoing, we need to improve...
>> Does adding select insert_platform_source_pair( 'twitter', 'brandwatch' ); here in mediawords.sql entail a migration, and should I also delete select insert_platform_source_pair( 'twitter', 'crimson_hexagon' );? > I think it does -...
Hey Hal! I know you're superbusy with other stuff, but could you have a quick look? In https://github.com/mediacloud/backend/issues/729 (and possibly https://github.com/mediacloud/backend/issues/725), it looks like `MediaWords::DBI::Stories::attach_story_data_to_stories()` used to reset `$list_field` key...
(Moved from #725.) More confusingly - asking to page with more rows than 100 seems to make the story_tags disaster in results. This code returns a story 105831 with story_tags...
Hey Jason! The AP crawler that you wrote (and Hal adapted) crashes pretty often with the following: ``` 2020-01-29T20:23:40.670845000Z INFO crawler_ap.ap: Found new story (guid: 3816eb886cea605916a1f879a5d516d8, version: 0), 2020-01-29T20:23:40.868011000Z INFO...
Me: > I think the crawler fetches quite a few (most? all?) downloads twice: fetchers constantly complain with *Object ID %d already exists, will store a new version or overwrite...
> Now that we consume 3rd party APIs as part of our core pipeline, we need some system for monitoring them to find out when they are down. This should...
Probably will be revived (#395), but we can always `git revert` it.
Hi, `/api/explorer/words/count` doesn't appear to like it too much when it receives Hindi UTF-8 response from `/api/v2/wc/list`, e.g. try: ``` https://api.mediacloud.org/api/v2/wc/list?q=%28%22%E0%A4%86%E0%A4%B0%E0%A5%8D%E0%A4%A5%E0%A4%BF%E0%A4%95+%E0%A4%B8%E0%A4%B6%E0%A4%95%E0%A5%8D%E0%A4%A4%E0%A4%BF%E0%A4%95%E0%A4%B0%E0%A4%A3%22+OR+%22%E0%A4%86%E0%A4%B0%E0%A5%8D%E0%A4%A5%E0%A4%BF%E0%A4%95+%E0%A4%B6%E0%A4%95%E0%A5%8D%E0%A4%A4%E0%A4%BF%22+OR+%22%E0%A4%B2%E0%A4%98%E0%A5%81+%E0%A4%B5%E0%A5%8D%E0%A4%AF%E0%A4%BE%E0%A4%AA%E0%A4%BE%E0%A4%B0%22+OR+%22%E0%A4%B2%E0%A4%98%E0%A5%81+%E0%A4%B5%E0%A5%8D%E0%A4%AF%E0%A4%B5%E0%A4%B8%E0%A4%BE%E0%A4%AF%22%29+AND+%28%28+tags_id_media%3A%289325106%29%29%29&num_words=100&sample_size=1000&include_stopwords=0&include_stats=0&ngram_size=1&fq=publish_day%3A%5B2019-01-01T00%3A00%3A00Z+TO+2019-06-30T00%3A00%3A00Z%5D&key= ``` Relevant `dokku/mc-explorer:latest` log: ``` [19:58:47][DEBUG] mediacloud.api api.py:_query:426 |...
Don't complain about invoices that both weren't paid and weren't supposed to be paid at a given year
While running `shaibos-totals`: ``` Invoice 'invoice_lvs00014' hasn't been paid in the year 2015, skipping invoice_lvs00014: unpaid Invoice 'invoice_lvs00015' hasn't been paid in the year 2015, skipping invoice_lvs00015: unpaid ```