gorilla icon indicating copy to clipboard operation
gorilla copied to clipboard

How to add a model that is not in vllm

Open shizhediao opened this issue 6 months ago • 5 comments

Hi,

I am evaluating a new model which is not in vllm. How can I generate responses with this model since I find vllm is the only way for generate()?

Thank you!

shizhediao avatar Aug 04 '24 22:08 shizhediao