Inconsistency in API Method Implementation in OpenAi Chat Model

[RikJux]

Overview

The class OpenAiChatModel:

• extends the abstract class AbstractFunctionCallSupport (implementing methods callWithFunctionSupport and callWithFunctionSupportStream)
• implements the two interfaces ChatModel and StreamingChatModel (defining the call and stream methods).

OpenAiChatModel implements the call method relying on the callWithFunctionSupport method, as expected. However, the implementation of the stream method does not rely on callWithFunctionSupportStream as one would expect.

Issue Description

This inconsistency in the implementation of the stream method makes it challenging to customize the existing models. When the call method uses callWithFunctionSupport, it provides a standard, extendable approach that can be easily adapted or overridden for custom behavior. However, the custom implementations of the stream method bypass callWithFunctionSupportStream, resulting in a fragmented and non-uniform codebase. This discrepancy hampers the ability to make consistent enhancements or apply uniform customizations across different models.

Current Implementation

The model classes implement the call method relying on the callWithFunctionSupport method, as expected. However, the implementation of the stream method does not rely on callWithFunctionSupportStream as one would expect.

OpenAiChatModel

@Override
public ChatResponse call(Prompt prompt) {
    ChatCompletionRequest request = createRequest(prompt, false);
    return this.retryTemplate.execute(ctx -> {
        ResponseEntity<ChatCompletion> completionEntity = this.callWithFunctionSupport(request);
        // other code
    });
}

@Override
public Flux<ChatResponse> stream(Prompt prompt) {
    ChatCompletionRequest request = createRequest(prompt, true);
    return this.retryTemplate.execute(ctx -> {
        Flux<OpenAiApi.ChatCompletionChunk> completionChunks = this.openAiApi.chatCompletionStream(request);
       // other code
    });
}

Suggested Change

Refactor the stream method in each of the model classes to use the callWithFunctionSupportStream method, similar to how the call method uses callWithFunctionSupport.

Note

This problem may also be present in other chat models implementing AbstractFunctionCallSupport