Investigate and correct Work Merge UI `503` errors

[jimchamp]

Evidence / Screenshot (if possible)

The work merge UI was very difficult to use today on account of several asynchronous rating, edition, and list fetches failing with 503 errors. After a page refresh, everything would usually load without fail (for me, anyway).

Note that until all records are fetched, the merge button is disabled.

Relevant url?

Steps to Reproduce

1. Select a number of works from a search result view and click the merge link in the ILE blue bar
2. Wait for all of the records to load.

• Actual: After some time, some of our API calls may fail with 503s. A retry dialog will appear.
• Expected: All of the records load fairly quickly, and the merge button is enabled.

Details

• **Logged in (Y/N)?**Y
• **Browser type/version?**Any
• **Operating system?**Any
• Environment (prod/dev/local)? prod

Proposal & Constraints

For now, I think that we should investigate why the 503s are occurring, and if the same API requests are failing for other pages in the site. Having these errors in Sentry would be helpful as well --- I didn't notice any of these errors there when I checked today.

Related files

Resources that I've seen return 503s:

/works/{OLID}/editions.json -> https://github.com/internetarchive/openlibrary/blob/master/openlibrary/plugins/openlibrary/api.py#L347 /works/{OLID}/ratings.json -> https://github.com/internetarchive/openlibrary/blob/master/openlibrary/plugins/openlibrary/api.py#L115 /works/{OLID}/bookshelves.json -> https://github.com/internetarchive/openlibrary/blob/master/openlibrary/plugins/openlibrary/api.py#L266 /works/{OLID}/lists.json?limit=0 -> https://github.com/internetarchive/openlibrary/blob/master/openlibrary/plugins/openlibrary/lists.py#L174

Stakeholders

@mheiman @seabelis @cdrini @mekarpeles

[cdrini]

@seabelis @mheiman I know we were seeing this a lot a few days ago; is this still popping up a lot?

[mheiman]

It was happening on every merge on Friday/Saturday -- now it seems to strike about 10% of the time. So, still a problem, but not completely disruptive.

[mekarpeles]

Here is a concrete example I am hitting (unless it's been merged) https://openlibrary.org/works/merge?records=OL16283010W,OL19642672W,OL21525711W,OL21780035W,OL21836646W,OL21836832W,OL21844199W,OL21847252W&primary=OL16283010W

[mekarpeles]

Hopefully db indices fixed this issue. Please re-open if we hit this again!