quarkus-langchain4j icon indicating copy to clipboard operation
quarkus-langchain4j copied to clipboard

Cannot use streamed responses with blocking tools

Open cescoffier opened this issue 5 months ago • 1 comments

Let's imagine an AI Service returning a Multi<String>. If the generation of the response uses tools, these tools cannot be blocking (basically, cannot use Panache or Hibernate).

I tried:

  • @RunOnVirtualThread (the magic bullet(tm))
  • @Blocking
  • .runSubscriptionOn(...)

None of them works.

I think we need to find a proper way to handle that use case.

cescoffier avatar Sep 17 '24 11:09 cescoffier