Support async call by return `Mono<ChatResponse>`

[XhstormR]

trafficstars

Please do a quick search on GitHub issues first, the feature you are about to request might have already been requested.

Expected Behavior

The call method of ChatClient supports returning Mono objects to achieve asynchronous calls.

For example：

public interface AsyncChatClient extends ModelClient<Prompt, ChatResponse> {
    @Override
    Mono<ChatResponse> call(Prompt prompt);
}

Current Behavior

Currently, the call method of AzureOpenAiChatClient does not support asynchronous calls. Only the stream method can return a Flux async object, but Flux represents multiple results. I need Mono object which it represent a single result.

Context

I used com.azure:azure-ai-openai SDK before. This SDK provides two type client: OpenAIClient and OpenAIAsyncClient. OpenAIAsyncClient provides complete Mono async support for a single result. But when I migrated to Spring AI, I found that Mono support was not provided.

[markpollack]

Thanks for your input.

[andresssantos]

Any updates on this subject?

[alvarosevilla95]

bumping this, an async endpoint for non-stream responses would be quite useful!

[markpollack]

Hi @alvarosevilla95 @andresssantos we are hoping to address this gap, as well as a non reactive stream based method, by RC1, but unsure if we will make it is time and it may have to wait until a 1.1 release.

[markpollack]

We will have to wait until a post GA release to fill this gap.