adamyodinsky
adding support for claude, azure, llama2 and cohere
Hi @adamyodinsky,
Noticed you're only calling OpenAI. I'm working on litellm (simple library to standardize LLM API Calls - https://github.com/BerriAI/litellm) and was wondering if we could be helpful.
Added support for Claude, Cohere, Azure and Llama2 (via Replicate) by replacing the ChatOpenAI completion call with a litellm completion call. The code is pretty similar to the OpenAI class - as litellm follows the same pattern as the openai-python sdk.
Would love to know if this helps.
Happy to add additional tests / update documentation, if the initial PR looks good to you.
@krrishdholakia
First of all, really appreciate the PR and the initiative mate! this is awsome!
the PR looks ok, but it seems missing some stuff.
- token window control. as different models have different window size, diff token encoders, we will need to match a window token controller mechanism to each model.
- how do i choose which model I use? Which vendor?
P.S - there is another project I'm working on where litellm can be useful, I'll check how I can integrate it. also, the langchain guys may find it useful as well. maybe you should holla them as well.
@adamyodinsky
re: choosing which model - you can do that by the model parameter
E.g.
completion(model="gpt-3.5-turbo"...)
Could you explain more / point me to example of what you mean regarding token window control? I don't think i saw how you were doing it today
Hey @adamyodinsky, can you say more about the token window controller mechanism. What are you looking for here?
@krrishdholakia I have an Azure hosted GPT4 model, will that work with your code out-of-the-box?
