haystack icon indicating copy to clipboard operation
haystack copied to clipboard

Fix: Add rate limit to OpenAIAnswerGenerator, change token computation

Open Timoeller opened this issue 2 years ago • 1 comments

Related Issues

  • fixes #3068
  • A related PR adjusting the temperatur param as well: https://github.com/deepset-ai/haystack/pull/3073

Proposed Changes:

  1. Added a rate limit since otherwise the API will crash eventually.
  2. Added a try catch around the API call since OpenAIs API sometimes returns errors
  3. Changed prompt length computation, since one has to take the output length into account as well

How did you test it?

manual verification

Checklist

Timoeller avatar Aug 19 '22 18:08 Timoeller

Sorry, I realize I won't be able to work on this the next week. I can either look into it afterward or you start taking over?

Timoeller avatar Aug 25 '22 16:08 Timoeller

Closing this in favor of https://github.com/deepset-ai/haystack/pull/3398

Timoeller avatar Oct 21 '22 09:10 Timoeller