mindcraft icon indicating copy to clipboard operation
mindcraft copied to clipboard
verified publisher icon kolbytn

Self-host model w/ sglang and vllm

Open BlankCheng opened this issue 10 months ago • 2 comments

Hi, is there any clue how to empower the agent backend LLM with sglang or vllm? Thanks!

BlankCheng avatar Feb 24 '25 00:02 BlankCheng

You would have to code in that as an API endpoint yourself, it isn't supported currently.

Sweaterdog avatar Feb 24 '25 04:02 Sweaterdog

Thanks! I get it run.

BlankCheng avatar Feb 24 '25 05:02 BlankCheng