djl icon indicating copy to clipboard operation
djl copied to clipboard
verified publisher icon deepjavalibrary

Does DJL support using large models? For example, Qwen

Open geekwenjie opened this issue 5 months ago • 2 comments

geekwenjie avatar Jul 29 '25 03:07 geekwenjie

The best way to serve Qen model is to use DJLServing

You can use DJL as a client side to run inference, see: https://github.com/deepjavalibrary/djl/blob/master/extensions/genai/src/test/java/ai/djl/genai/openai/ChatCompletionTest.java#L59-L63

frankfliu avatar Jul 29 '25 04:07 frankfliu

thank you

geekwenjie avatar Jul 29 '25 04:07 geekwenjie