quarkus-langchain4j
quarkus-langchain4j copied to clipboard
quarkiverse
Dev UI chat page: support images in messages
Hm, I'm trying to implement attaching images using the Vaadin upload component and then converting to base64 in the frontend and passing it to a jsonrpc invocation, but I'm hitting some kind of limitation where the websocket for JsonRPC invocations can only handle messages up to 100 kB, anything bigger crashes the websocket :(
@phillip-kruger have you perhaps tried anything similar before?
(For clarity, the intended use case is creating an image-based chat message to pass to gpt-4-turbo or any other model that has computer vision capabilities)
In that case just use normal post, not the ws. You will have to use another comment or build your own
Yeah that will make things somewhat more complicated because the image will arrive completely independently of the accompanying text message, but I'll give it a try
Hm, I now realize, if I want to add a custom endpoint beyond JSON-RPC (either a REST endpoint or a reactive route) then I have to add some kind of HTTP extension, and we recently removed that to make quarkus-langchain4j suitable for CLI applications :(
We don't have that as a dependency of the extension (neither the deployment, nor the runtime artifact). Quarkus apparently brings it in when the user runs dev mode, but we don't have any compile-time dependency on it (when compiling the extension)
Ok I see. Maybe I can add something to the Dev UI to make this possible. How urgent is this ?
It would be cool to have the ability to post images for demoing on conferences (the first is in the half of May), but even if it doesn't work out I still have enough material to present :)
Simply making the jsonrpc endpoint accept messages over 100 kB isn't possible? I don't know where the limitation is coming from. But lifting it would be the easiest solution.
Yea I am not sure where that comes from. It's not something I set in Dev UI so must be some default of Vertx Web Sockets. If you can find and increase that, go for it !