quarkus-langchain4j
quarkus-langchain4j copied to clipboard
Cannot use streamed responses with blocking tools
Let's imagine an AI Service returning a Multi<String>
. If the generation of the response uses tools, these tools cannot be blocking (basically, cannot use Panache or Hibernate).
I tried:
- @RunOnVirtualThread (the magic bullet(tm))
- @Blocking
-
.runSubscriptionOn(...)
None of them works.
I think we need to find a proper way to handle that use case.