Allow control over LLM call timeout

[johnsonr]

Slow LLM calls should not cause ReadTimeoutException.

Per ptr in Discord:

We are building a few PoCs and have hit a blocker that we have also experienced with Spring AI.

If we are sending a non-streaming request to a LLM that is more than trivial, the LLMs "thinking time" causes a ReadTimeoutException within Netty, which is used by Spring for HTTP Requests.

We were able to get around this issue in SpringAI by exposing the following as a Bean:

 public Config()
    {
        httpClient = HttpClient.create().responseTimeout( Duration.ofSeconds( 120 ) )  // 120 second read timeout
                .option( ChannelOption.CONNECT_TIMEOUT_MILLIS, 10000 );

        builder = RestClient.builder().requestFactory( new ReactorClientHttpRequestFactory( httpClient ) );
    }

    @Bean
    public OpenAiApi openAiApi()
    {
        return OpenAiApi.builder().apiKey( openAiApiKey ).restClientBuilder( builder ).build();
    }

However, there seems to be no similar ability in Embabel. A timeout was introduced recently to LllmOptions, but this is a timeout to the overall request - and is wrapped around the HTTP call.

[corneil]

👍