chatml support

[phiweger]

Is it possible to query an index with the (new) chatml syntax? like:

[
    {
        "role": "system", 
        "content" : "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.\nKnowledge cutoff: 2021-09-01\nCurrent date: 2023-03-02"
    },
    {
        "role": "user", 
        "content" : "How are you?"
    },
    {
        "role": "assistant", 
        "content" : "I am doing well"
    },
    {
        "role": "user", 
        "content" : "What is the mission of the company OpenAI?"
    }
]

[jerryjliu]

If you use a chatgpt llm, then we'll use this message format! https://github.com/jerryjliu/gpt_index/blob/main/examples/vector_indices/SimpleIndexDemo-ChatGPT.ipynb

[phiweger]

Thank you! But how would I translate the above example into a query (which has to be a single string, right?) -- just concatenate the strings in the dict above?

[logan-markewich]

@phiweger using the ChatGPTLLMPredictor class, the messages get built like that for you under the good 💪 so single string inputs are fine

[phiweger]

awesome! do I need to use this predictor class for both, index construction AND query? @logan-markewich

[logan-markewich]

Technically, for list and vector indexes, the llm is not used during construction, so it's only needed at query() time @phiweger

But, passing it to both won't hurt either.

Also, the ChatLLMPredictor class might actually be deprecated, so the examples at the top of Jerry's notebook will work best now