quarkus-langchain4j
quarkus-langchain4j copied to clipboard
quarkiverse
Setting max-retries=0 causes infinite retries
If you set, say, quarkus.langchain4j.openai.max-retries=0, it will attempt to retry indefinitely.
The correct setting for "no retries" is 1, but given the property name, some people might think that max-retries=0 means "just try once", which isn't the case. Instead, it causes the retry loop to immediately break through the upper bound, and because the end condition is based on == it will probably only stop after an integer overflow...
I'll probably fix this in two places, once in the upstream class dev.langchain4j.internal.RetryUtils to reject 0 max-retries, and also at the level of validating Quarkus configuration.
To be honest, I think we should not retry at all by default and promote the use of Smallrye Fault Tolerance.
But in any case max-retries=0 should indeed not do infinite retries
To be honest, I think we should not retry at all by default and promote the use of Smallrye Fault Tolerance.
How would one do that, annotate some application methods with MP FT annotations and set the max-retries to 1 to avoid the clashing of two approaches? The problem is that an application developer cannot annotate the methods of the ChatLanguageModel interface directly
My idea is that we should have retries on the LangChain4j level turned off by default and push people to use FT with AiService.
For ChatLanguageModel we obviously can't do anything of that sort.
Ok, so perhaps I could send a PR that turns off the LangChain4j-level retries for chat models by default (change the default to 1) and make it clear in the configuration javadocs that if one uses AiServices, the preferred way to achieve FT is to use MP FT annotations on the AI service?
(This may not be reasonable to do in some cases - for example, if some of the used tools aren't idempotent)