Support StableVicuna

[iRanadheer]

I've seen you are supporting the openai 3.5 turbo model, and I couldnt find how to call using an api. perhaps I couldn't find the instructions?

Also, do you have any plans on supporting the stablevicuna model?

[rwl4]

The reference to gpt 3.5 turbo you saw is actually the model conversation style which could be used on any model, and it's not completed. I actually added full support for finetuning models using a subset of OpenAI's ChatML here: https://github.com/lm-sys/FastChat/pull/644

[merrymercy]

For gpt-3.5 / gpt-4, please use this option https://github.com/lm-sys/FastChat/blob/a879340a813cc4b304b598d0e61b13f4ffebbd61/fastchat/serve/gradio_web_server.py#L591-L598

For stablevicuna, please help us add it https://github.com/lm-sys/FastChat/blob/main/docs/arena.md#how-to-add-a-new-model

[iRanadheer]

@merrymercy

The process of adding models to the system is not difficult, but it becomes complicated when each model has its own "end-of-sequence" token that it was trained with. To solve this issue, a template for each version of the model can be created. For example, since the stablevicuna model was trained using vicuna v0, adding a template for v0 should solve the issue, as long as the model name argument does not contain the string "vicuna". Otherwise, a different template (V1.1) will be called. see below

def get_default_conv_template(model_name):
    model_name = model_name.lower()
    if "vicuna" in model_name or "output" in model_name:
        return conv_vicuna_v1_1

For the wizard-vicuna-13b model, since it was created using v1.1, we can use the existing template.

[merrymercy]

We did some refactoring to make adding new models easier. Please help us add the support for wizard-vicuna-13b and StableVicuna. You can see https://github.com/lm-sys/FastChat/pull/1019 for example. For the name conflict, you can register a model_adapter StableVicunaAdapter prior to the VicunaAdatper so the stable vicuna has a higher priority. Then you can get the correct template for stable vicuna.