vllm icon indicating copy to clipboard operation
vllm copied to clipboard

[Server] use get_conversation_template to make model template

Open akxxsb opened this issue 1 year ago • 1 comments

we can use get_conversation_template method, then we don't need to specified chat_template, when new models add, we don't need to change our code, because the get_conversation_template method will first find the correct model adapter ,then use it's get_default_conv_template to get conv template. which is easy to use than before.

def get_conversation_template(model_path: str) -> Conversation:
    adapter = get_model_adapter(model_path)
    return adapter.get_default_conv_template(model_path)

akxxsb avatar Jul 04 '23 09:07 akxxsb

@zhuohan123

akxxsb avatar Jul 04 '23 10:07 akxxsb

A lots of models still not merge to FastChat yet, but some of them may have the same Conversation templeate of previous, I don't think this is a flexible change and get_conv_template(chat_template) should be better for developer.

gesanqiu avatar Jul 19 '23 09:07 gesanqiu