integreat-cms icon indicating copy to clipboard operation
integreat-cms copied to clipboard

Translating pages with many paragraphs via SUMM.AI runs into rate limiting

Open timobrembeck opened this issue 2 years ago • 0 comments

Describe the Bug

Translating a page which has many paragraphs (or bullet list items) via SUMM.AI may be impossible due to their rate limiting of 15 requests per second (in reality, I managed to send more requests than documented, but at least this is the official limit they're claiming).

Steps to Reproduce

  1. Create page with many HTML segments (<p> or <li> tags)
  2. Translate page into Easy German via SUMM.AI
  3. See error (perhaps)

Expected Behavior

The SUMM.AI API client should inspect the response, and if it is the error which indicates that too many request were sent (HTTP status 429, see docs), delay all other parallel tasks, wait for one minute, resend the failed request(s) and continue.

Actual Behavior

Currently, the client does not respect the quota and just sends as many requests as there are HTML segments. Also, it does not stop if it encounters one error - it just finishes all other requests which causes the server to crash and return a less meaningful error message (just internal server error in HTML instead of JSON).

Additional Information

timobrembeck avatar Oct 05 '22 21:10 timobrembeck