haystack
haystack copied to clipboard
deepset-ai
Add enviroment variable for openai timeout, backoff and max retries
Is your feature request related to a problem? Please describe. In version 1, haystack allows to set three environment variables to manage:
- OPENAI_TIMEOUT
- OPENAI_MAX_RETRIES
- OPENAI_BACKOFF
It's useful also for the versione 2.0
@nickprock, I've been locking for this environment variables on the v1.x and v1.25.x branches and cannot find any reference to them, could you give me more info please.
In 1.25 there are these environment variables
# Any remote API (OpenAI, Cohere etc.)
HAYSTACK_REMOTE_API_BACKOFF_SEC = "HAYSTACK_REMOTE_API_BACKOFF_SEC"
HAYSTACK_REMOTE_API_MAX_RETRIES = "HAYSTACK_REMOTE_API_MAX_RETRIES"
HAYSTACK_REMOTE_API_TIMEOUT_SEC = "HAYSTACK_REMOTE_API_TIMEOUT_SEC"
HAYSTACK_PROMPT_TEMPLATE_ALLOWED_FUNCTIONS = "HAYSTACK_PROMPT_TEMPLATE_ALLOWED_FUNCTIONS"
https://github.com/deepset-ai/haystack/blob/v1.25.5/haystack/environment.py
I talked with @vblagoje on Discord here: https://discord.com/channels/993534733298450452/1232706764294717511
@nickprock the discord invitation does not work. Could you send a new one please.
Not really getting anywhere, and I will be out for the next few days, so feel free to proceed with this implementation. :)
Hey @masci just wanted to give you a heads up that deepset Cloud + Sol will want this since we used this extensively for client pipelines that timed out when using bigger models like GPT4.
