airllm icon indicating copy to clipboard operation
airllm copied to clipboard

Published 20 hours ago • verified publisher icon lyogavin

docker based or BareMetal serving

Open dhandhalyabhavik opened this issue 1 year ago • 1 comments

Wondering if any plans to implement to enable servings,

similar to vllm serving, it should support OpenAI compatible chat endpoints.

dhandhalyabhavik avatar Nov 10 '24 14:11 dhandhalyabhavik

I would like to get any kind of loaded serving on endpoint, what is the way to serve it on endpoint?

gnusupport avatar Jan 06 '25 16:01 gnusupport