Error - ValueError: Invalid style: None when running with langchain

[davzucky]

First, thank you for this great library. It is working well and the basic are working.

We are trying now to use it with langchain and replace openai API with the one of fastchat using this document.

However, in langchain when you set the model to gtp-x it changes automatically to chat mode. This will call the endpoint /v1/chat/completions

This will try to extract the prompt parameter for the model using get_gen_params, which will return the ChatGPTAdapter as the model is gpt-x. is set the conversation template for chatgpt. This model set sep_style=None which is the root cause of the error.

I wrote a simple test that show the error

def test_get_gen_params_gpt4():
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who won the world series in 2020?"},
        {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
        {"role": "user", "content": "Where was it played?"}
    ]
    gen_params = get_gen_params("gpt-4", messages=messages, temperature=0.9, max_tokens=10, top_p=1.0, echo=False, stream=False, stop=["\n"])
    assert gen_params["model"] == "gpt-4"    

Question ?

In the mode we are using for langchain we are simulating gpt model using the model name; however, in the backend, the model can be anything. This is why I'm thinking the openai server should not query the model worker to ask for which real model they are using. At the moment, you only replace the model name, however you should have a notion of model name and real model. If you are happy with that I could have a look at it

[davzucky]

I found that using full model name like gpt-3.5-turbo-0301 or gpt-4-0314 unlock my problem, however the root cause is still here. you can find the full list of model here