running into rate limit with OpenAI's GPT-3 API when used during evaluation

[Timoeller]

Is your feature request related to a problem? Please describe. OpenAI has some rate limits on their API access:

• 20 requests per minute for free access
• 60 rpm in pay as you go initially, then it ramps up to 3000 later

see https://help.openai.com/en/articles/5955598-is-api-usage-subject-to-any-rate-limits for details.

Describe the solution you'd like I see 3 solutions:

1. get paid plan and wait a bit (3000 requests per minute should be fine for most cases)
2. hack a time.sleep(X) in here
3. haystack implements an optional rate limit in the eval methods