Fix unit tests, trigger completion callback when connection is closed, avoid NPE in client timeout

[alexis779]

This Pull Request handles gracefully errors that can happen while using OpenAI API compatible backends, such as predibase and replicate.

It also fixes some consistently failing unit tests.

Case 1

The problem

When using langchain4j-open-ai library with an Open AI compatible API, such as predibase, we would get a NullPointerException when the client times out. This can be caused by predibase warming up Lora adapter.

2024-07-17T15:58:56.682-07:00 DEBUG 3479431 --- [OpenAI proxy] [edibase.com/...] d.a.openai4j.StreamingRequestExecutor    : onFailure()

java.io.InterruptedIOException: timeout
	at okhttp3.internal.connection.RealCall.timeoutExit(RealCall.kt:398) ~[okhttp-4.12.0.jar:na]
	at okhttp3.internal.connection.RealCall.callDone(RealCall.kt:360) ~[okhttp-4.12.0.jar:na]
	at okhttp3.internal.connection.RealCall.noMoreExchanges$okhttp(RealCall.kt:325) ~[okhttp-4.12.0.jar:na]
	at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:209) ~[okhttp-4.12.0.jar:na]
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:517) ~[okhttp-4.12.0.jar:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na]
	at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na]
Caused by: java.io.IOException: Canceled
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:72) ~[okhttp-4.12.0.jar:na]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) ~[okhttp-4.12.0.jar:na]
	at dev.ai4j.openai4j.RequestLoggingInterceptor.intercept(RequestLoggingInterceptor.java:41) ~[openai4j-0.18.0.jar:na]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) ~[okhttp-4.12.0.jar:na]
	at dev.ai4j.openai4j.GenericHeaderInjector.intercept(GenericHeaderInjector.java:27) ~[openai4j-0.18.0.jar:na]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) ~[okhttp-4.12.0.jar:na]
	at dev.ai4j.openai4j.GenericHeaderInjector.intercept(GenericHeaderInjector.java:27) ~[openai4j-0.18.0.jar:na]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) ~[okhttp-4.12.0.jar:na]
	at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201) ~[okhttp-4.12.0.jar:na]
	... 4 common frames omitted

The fix

Do not log the response when null, which can happen upon a client timeout.

The reproduction case

• name: predibase
• url: https://serving.app.predibase.com/.../deployments/v2/llms/mistral-7b-instruct-v0-3/v1
• token: ...
• model: ""

Case 2

The problem

If the server does not send a DONE event at the end of the Server Side Event stream, the client gets stuck for ever.

The fix

Notify the completion callback upon closing the response instead. This finite state machine transition always happens in the SSE stream lifecycle.

The reproduction case

• backend: replicate
• url: https://openai-proxy.replicate.com/v1
• token: ...
• model: mistralai/mistral-7b-instruct-v0.2

Unit tests

Some unit tests consistently fail.

Deprecated OpenAI model

The problem

GPT_4_0314 is now deprecated

Caused by: dev.ai4j.openai4j.OpenAiHttpException: {
    "error": {
        "message": "The model `gpt-4-0314` has been deprecated, learn more here: https://platform.openai.com/docs/deprecations",
        "type": "invalid_request_error",
        "param": null,
        "code": "model_not_found"
    }
}

The fix

Just remove this model entry.

Race condition

To test that a request is effectively cancelled, rely on asynchronous callbacks instead of sleeping. Cancel the response once the request started. Use a CompletableFuture to wait for this action. Validate that the request did not complete or error by failing the test within associated callbacks.