quarkus-langchain4j
quarkus-langchain4j copied to clipboard
quarkiverse
Dev UI page for testing of AI services
Currently we have a Chat model page in our Dev UI that injects and uses a raw chat model. That's nice but not extremely useful. Users might want to test AI services, including the ability to use tools, and RAG, etc.
The main challenge with using AI services is creating a UI that allows working with generic Java objects.
Showing a POJO result should be relatively easy (serialize it into JSON, or call toString()?), but what if the service takes POJOs as arguments? We would need to invent a way to allow the user to describe the instance, serialize it, send it to the JSON-RPC endpoint, and then transform it into the real object...
It will probably be quite tricky to implement, but the potential usefulness is huge IMO.
I agree it would be useful, but I am very sceptical we could get it right because of the complication of serializing and deserializaing the input and output.
The easier option would be to generate at least a "dummy" AI Service that takes and produces only strings, but it still can have tools and a content retriever, so you can test tools and RAG with it.
@mkouba had the same challenges for gRPC. And we just asked the users to write the json object. Not great but good enough.
In the OSGi world, we had something named metatype that was allowing generating dedicated form for complex object. A possibility would be to see if we can do something similar based on an openapi / json schema descriptor. But the first approach is probably good enough for now.
The easier option would be to generate at least a "dummy" AI Service that takes and produces only strings, but it still can have tools and a content retriever, so you can test tools and RAG with it.
Just for the record, I've recently added that (the chat page now picks up retrieval augmentors and tools). Let's see if it's good enough, or if we get pushed to add support for full-blown AI services.