backend

backend copied to clipboard

Podcast transcoding fails for some episodes because: ``` $ docker service logs $(docker service ls | grep podcast-transcribe-episode-temporal-worker | awk '{ print $1 }') mediacloud_podcast-transcribe-episode-temporal-worker.1.bi957ibrx176@bd-misc | INFO podcast_transcribe_episode.workflow: Fetching, transcoding,...

pypt

Don't allow API calls with sort=random + last_id

1

Post-Colorado Mystery, make the API return an error when these parameters are used together in calls to `stories/list` endpoint: `sort=random` and `processed_stories_id>0`.

jtotoole

Fix disk space "leak" in extract-and-vector, limit disk usage on all services

2

`extract-and-vector` workers tend to fill up `/var/tmp` with gigabytes of pretty much identical files which are of the size of either 0 or 3332489: ``` $ docker exec -it 689b33c92426...

pypt

Implement workflow to merge feeds into each other

1

So, now that we came up with lists of media sources / feeds to be merged into each other (#799), let's try doing the actual merging. Given that: * We...

pypt

Bumps [pip](https://github.com/pypa/pip) from 21.0 to 21.1. Changelog Sourced from pip's changelog. 21.1 (2021-04-24) Process Start installation scheme migration from distutils to sysconfig. A warning is implemented to detect differences between...

dependabot[bot]

This represents a first pass at the code for feed merging, as yet untested because I was having trouble getting the `Dockerfile` to build (and consequently running the Python interpreter...

jtotoole

Back up PostgreSQL databases

9

Given that disks on which we run our services are pre-shot or at least very old, and the disk health review + replacement is still ongoing, we need to improve...

pypt

fix jieba cache issue

4

references https://github.com/mediacloud/backend/issues/809

jtotoole

jtotoole

Add topic mine support for the Pushshift verified twitter archive

6

This adds support for pulling data from the Pushshift verified twitter archive. A couple things of note: - Implemented using Elasticsearch's scroll API for paging support. - This makes a...

epenn