haystack-core-integrations icon indicating copy to clipboard operation
haystack-core-integrations copied to clipboard

Add Request Timeout Option to AnthropicGenerator for API Calls

Open hollandmwx opened this issue 8 months ago • 0 comments

Is your feature request related to a problem? Please describe. I am unable to set a request timeout for API calls when using the AnthropicGenerator class from Haystack's integration with the Anthropic API. Currently, there is no option to define a timeout for requests, which can be problematic in scenarios where network latency or server issues cause long delays.

Describe the solution you'd like I would like to have a timeout parameter added to the AnthropicGenerator class that can be passed through the generation_kwargs. Alternatively, the class could be updated to support two timeout configurations provided by the underlying Anthropic API:

  1. Global Timeout Setting: Set a default timeout for all requests when initializing the Anthropic client. client = Anthropic(timeout=20.0)

  2. Per-request Timeout Override: Override the timeout for each request individually.

client.with_options(timeout=5.0).messages.create(
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello, Claude"}],
    model="claude-3-5-sonnet-latest",
)

Describe alternatives you've considered As a workaround, I considered inheriting the class and modifying it directly to add timeout functionality, but this approach requires custom development and isn't optimal for users who want an out-of-the-box solution.

hollandmwx avatar Mar 21 '25 11:03 hollandmwx