backend icon indicating copy to clipboard operation
backend copied to clipboard

story list call super slow right now

Open rahulbot opened this issue 4 years ago • 0 comments

I'm trying to debug some user support requests, but right now calls to story list are going super slow. The below log shows return times from 1 minute to 3 minutes just to fetch 500 stories matching a pretty simple query. Is there some load on the system causing this?

[14:54:36][DEBUG] mediacloud.api api.py:_query:424 | query GET to https://api.mediacloud.org/api/v2/stories_public/list with {'q': '(Wuhan AND coronavirus) AND (( tags_id_media:(58722749)))', 'fq': 'publish_day:[2020-01-09T00:00:00Z TO 2020-02-09T00:00:00Z]', 'last_processed_stories_id': 0, 'rows': 500, 'sort': 'processed_stories_id', 'wc': 0, 'feeds_id': None, 'show_feeds': 0} and None
[14:57:39][DEBUG] mediacloud.api api.py:_query:477 | Profiling: 183.70073699951172s for GET to https://api.mediacloud.org/api/v2/stories_public/list (with {'q': '(Wuhan AND coronavirus) AND (( tags_id_media:(58722749)))', 'fq': 'publish_day:[2020-01-09T00:00:00Z TO 2020-02-09T00:00:00Z]', 'last_processed_stories_id': 0, 'rows': 500, 'sort': 'processed_stories_id', 'wc': 0, 'feeds_id': None, 'show_feeds': 0, 'key': 'MY_KEY'} / null)
[14:57:40][DEBUG] mediacloud.api api.py:_query:424 | query GET to https://api.mediacloud.org/api/v2/stories_public/list with {'q': '(Wuhan AND coronavirus) AND (( tags_id_media:(58722749)))', 'fq': 'publish_day:[2020-01-09T00:00:00Z TO 2020-02-09T00:00:00Z]', 'last_processed_stories_id': 1899484217, 'rows': 500, 'sort': 'processed_stories_id', 'wc': 0, 'feeds_id': None, 'show_feeds': 0} and None
[15:00:12][DEBUG] mediacloud.api api.py:_query:477 | Profiling: 152.50264596939087s for GET to https://api.mediacloud.org/api/v2/stories_public/list (with {'q': '(Wuhan AND coronavirus) AND (( tags_id_media:(58722749)))', 'fq': 'publish_day:[2020-01-09T00:00:00Z TO 2020-02-09T00:00:00Z]', 'last_processed_stories_id': 1899484217, 'rows': 500, 'sort': 'processed_stories_id', 'wc': 0, 'feeds_id': None, 'show_feeds': 0, 'key': 'MY_KEY'} / null)
[15:00:12][DEBUG] mediacloud.api api.py:_query:424 | query GET to https://api.mediacloud.org/api/v2/stories_public/list with {'q': '(Wuhan AND coronavirus) AND (( tags_id_media:(58722749)))', 'fq': 'publish_day:[2020-01-09T00:00:00Z TO 2020-02-09T00:00:00Z]', 'last_processed_stories_id': 1901312586, 'rows': 500, 'sort': 'processed_stories_id', 'wc': 0, 'feeds_id': None, 'show_feeds': 0} and None
[15:00:59][DEBUG] mediacloud.api api.py:_query:477 | Profiling: 46.87873411178589s for GET to https://api.mediacloud.org/api/v2/stories_public/list (with {'q': '(Wuhan AND coronavirus) AND (( tags_id_media:(58722749)))', 'fq': 'publish_day:[2020-01-09T00:00:00Z TO 2020-02-09T00:00:00Z]', 'last_processed_stories_id': 1901312586, 'rows': 500, 'sort': 'processed_stories_id', 'wc': 0, 'feeds_id': None, 'show_feeds': 0, 'key': 'MY_KEY'} / null)
[15:00:59][DEBUG] mediacloud.api api.py:_query:424 | query GET to https://api.mediacloud.org/api/v2/stories_public/list with {'q': '(Wuhan AND coronavirus) AND (( tags_id_media:(58722749)))', 'fq': 'publish_day:[2020-01-09T00:00:00Z TO 2020-02-09T00:00:00Z]', 'last_processed_stories_id': 1903306788, 'rows': 500, 'sort': 'processed_stories_id', 'wc': 0, 'feeds_id': None, 'show_feeds': 0} and None
[15:01:47][DEBUG] mediacloud.api api.py:_query:477 | Profiling: 48.112025022506714s for GET to https://api.mediacloud.org/api/v2/stories_public/list (with {'q': '(Wuhan AND coronavirus) AND (( tags_id_media:(58722749)))', 'fq': 'publish_day:[2020-01-09T00:00:00Z TO 2020-02-09T00:00:00Z]', 'last_processed_stories_id': 1903306788, 'rows': 500, 'sort': 'processed_stories_id', 'wc': 0, 'feeds_id': None, 'show_feeds': 0, 'key': 'MY_KEY'} / null)
[15:01:48][DEBUG] mediacloud.api api.py:_query:424 | query GET to https://api.mediacloud.org/api/v2/stories_public/list with {'q': '(Wuhan AND coronavirus) AND (( tags_id_media:(58722749)))', 'fq': 'publish_day:[2020-01-09T00:00:00Z TO 2020-02-09T00:00:00Z]', 'last_processed_stories_id': 1905010327, 'rows': 500, 'sort': 'processed_stories_id', 'wc': 0, 'feeds_id': None, 'show_feeds': 0} and None
[15:02:32][DEBUG] mediacloud.api api.py:_query:477 | Profiling: 44.4689998626709s for GET to https://api.mediacloud.org/api/v2/stories_public/list (with {'q': '(Wuhan AND coronavirus) AND (( tags_id_media:(58722749)))', 'fq': 'publish_day:[2020-01-09T00:00:00Z TO 2020-02-09T00:00:00Z]', 'last_processed_stories_id': 1905010327, 'rows': 500, 'sort': 'processed_stories_id', 'wc': 0, 'feeds_id': None, 'show_feeds': 0, 'key': 'MY_KEY'} / null)

rahulbot avatar Feb 10 '20 20:02 rahulbot