vllm
vllm copied to clipboard
[Server] use get_conversation_template to make model template
we can use get_conversation_template method, then we don't need to specified chat_template, when new models add, we don't need to change our code, because the get_conversation_template method will first find the correct model adapter ,then use it's get_default_conv_template to get conv template. which is easy to use than before.
def get_conversation_template(model_path: str) -> Conversation:
adapter = get_model_adapter(model_path)
return adapter.get_default_conv_template(model_path)
@zhuohan123
A lots of models still not merge to FastChat yet, but some of them may have the same Conversation templeate of previous, I don't think this is a flexible change and get_conv_template(chat_template) should be better for developer.