FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

Published 20 hours ago • verified publisher icon lm-sys

OpenAI API compatible serving would deserve its own project.

Open XReyRobert opened this issue 1 year ago • 1 comments

Hi all, thanks for your work.

Openai api serving is a great feature that would IMHO deserve a project by its own to get more traction.. Not sure if there are already existing projects in this area?

XReyRobert avatar May 06 '23 13:05 XReyRobert

@XReyRobert I guess https://github.com/hyperonym/basaran might be such a project. It directly works on huggingface models.

lslslslslslslslslslsls avatar May 07 '23 08:05 lslslslslslslslslslsls

Glad that you love these features.

  • FastChat is not only about Vicuna. It will support more models and features in the future. This OpenAI-compatible serving is definitely one thing we will push.
  • @XReyRobert Thanks for sharing this. FastChat also works with Hugging Face models but additionally provides better chatting templates.

merrymercy avatar May 08 '23 08:05 merrymercy