how to load local llm in quarkus

[oneanime]

hello，how to load local llm in quarkus

[geoand]

cc @cescoffier @maxandersen

We don't currently anything for this, but we will soon have a comprehensive story around this

[cescoffier]

Which model are you looking at @oneanime ?

[oneanime]

Which model are you looking at @oneanime ?

https://huggingface.co/BlinkDL/rwkv-5-world

[rsvoboda]

quarkus-langchain4j provides 2 options to run large language models locally

• https://docs.quarkiverse.io/quarkus-langchain4j/dev/jlama.html
• https://docs.quarkiverse.io/quarkus-langchain4j/dev/llama3.html

@geoand @oneanime I think this could be closed

[geoand]

Yeah, definitely